Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients

From: Antoine Riard <antoine.riard@gmail•com>
To: Igor Cota <igor@codexapertus•com>
Cc: Bitcoin Protocol Discussion
	<bitcoin-dev@lists•linuxfoundation.org>,
	"lightning-dev\\\\@lists.linuxfoundation.org"
	<lightning-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients
Date: Sat, 9 May 2020 03:22:52 -0400	[thread overview]
Message-ID: <CALZpt+FH10XgPsGZn_kgzrzObjvpnB-j3sh8C=ytBXatPzacYg@mail.gmail.com> (raw)
In-Reply-To: <CAJx8jdwHF3wF+q+JafuDhMtca45Wb2vS_gvB0yw71jNGz=ZHzA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 11702 bytes --]

Hi Igor,

Thanks for sharing about what it's technically possible to do for a
full-node on phone, specially with regards to lower grade devices.

I do see 2 limitations for sleeping nodes:
- a lightning specific one, i.e you need to process block data real-time in
case of incoming HTLC you need to claim on chain or a HTLC timeout. There
is a bunch of timelocks implications in LN,  with regards to CSV,
CLTV_DELTA, incoming policy, outgoing policy, ... and you can't really
afford to be late without loosing a payment. I don't see timelocks being
increase, that would hinder liquidity.
- a p2p bandwidth concern, even if this new class of nodes turn as public
ones, they would still have a heavy sync period due to be fallen-behind
during the day, so you would have huge bandwidth spikes every a timezone
falls asleep and a risk of choking upload links of stable full-nodes.

I think assume-utxo may be interesting in the future in case of long-fork
detection, you may be able to download a utxo-set on the fly, and fall-back
to a full-node. But that would be only an emergency measure, not a regular
cost on the backbone network.

Antoine

Le jeu. 7 mai 2020 à 12:41, Igor Cota <igor@codexapertus•com> a écrit :

> Hi Antoine et al,
>
> Maybe I'm completely wrong, missing some numbers, and it's maybe fine to
>> just rely on few thousands of full-node operators being nice and servicing
>> friendly millions of LN mobiles clients. But just in case it may be good to
>> consider a reasonable alternative.
>>
>
>
>> So you may want to separate control/data plane, get filters from CDN and
>> headers as check-and-control directly from the backbone network. "Hybrid"
>> models should clearly be explored.
>
>
> For some months now I've been exploring the feasibility of running full
> nodes on everyday phones [1]. One of my first thoughts was how to avoid the
> phones mooching off the network. Obviously due to battery, storage and
> bandwidth constraints it is not reasonable to expect pocket full nodes to
> serve blocks during day time.
>
> Huge exception to this is the time we are asleep and our phones are
> connected to wifi and charging. IMO this is a huge untapped resource that
> would allow mobile nodes to earn their keep. If we limit full node
> operation to sleepy night time the only constraining resource is storage:
> 512 gb of internal storage in phones is quite rare, probably about $100 for
> an SD card with full archival node capacity but phones with memory card
> slots rarer still - no one is going to bother.
>
> So depending on their storage capacity phone nodes could decide to store
> and serve just a randomly selected range of blocks during their nighttime
> operation. With trivial changes to P2P they could advertise the blocks they
> are able to serve.
> If there comes a time that normal full nodes feel DoS'ed they can
> challenge such nodes to produce the blocks they advertise and ban them as
> moochers if they fail to do so. Others may elect to be more charitable and
> serve everyone.
>
> These types of nodes would truly be part-timing since they only carry a
> subset of the blockchain and work while their operator is asleep. Probably
> should be called part-time or Sleeper Nodes™.
>
> They could be user friendly as well, with Assume UTXO they could be
> bootstrapped quickly and while they do the IBD in the background instead of
> traditional pruning they can keep the randomly assigned bit of blockchain
> to later serve the network.
>
> Save for the elderly, all the people I know could run such a node, and I
> don't live in a first world country.
>
> There is also the feel-good kumbaya aspect of American phone nodes serving
> the African continent while the Americans are asleep, Africans and
> Europeans serving the Asians in kind. By plugging in our phones and going
> to sleep we could blanket the whole world in (somewhat) full nodes!
>
> Cheers,
> Igor
>
> [1] https://icota.github.io/
>
> On Tue, 5 May 2020 at 12:18, Antoine Riard <antoine.riard@gmail•com>
> wrote:
>
>> Hi,
>>
>> (cross-posting as it's really both layers concerned)
>>
>> Ongoing advancement of BIP 157 implementation in Core maybe the
>> opportunity to reflect on the future of light client protocols and use this
>> knowledge to make better-informed decisions about what kind of
>> infrastructure is needed to support mobile clients at large scale.
>>
>> Trust-minimization of Bitcoin security model has always relied first and
>> above on running a full-node. This current paradigm may be shifted by LN
>> where fast, affordable, confidential, censorship-resistant payment services
>> may attract a lot of adoption without users running a full-node. Assuming a
>> user adoption path where a full-node is required to benefit for LN may
>> deprive a lot of users, especially those who are already denied a real
>> financial infrastructure access. It doesn't mean we shouldn't foster node
>> adoption when people are able to do so, and having a LN wallet maybe even a
>> first-step to it.
>>
>> Designing a mobile-first LN experience opens its own gap of challenges
>> especially in terms of security and privacy. The problem can be scoped as
>> how to build a scalable, secure, private chain access backend for millions
>> of LN clients ?
>>
>> Light client protocols for LN exist (either BIP157 or Electrum are used),
>> although their privacy and security guarantees with regards to
>> implementation on the client-side may still be an object of concern
>> (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted fee
>> estimation). That said, one of the bottlenecks is likely the number of
>> full-nodes being willingly to dedicate resources to serve those clients.
>> It's not about _which_ protocol is deployed but more about _incentives_ for
>> node operators to dedicate long-term resources to client they have lower
>> reasons to care about otherwise.
>>
>> Even with cheaper, more efficient protocols like BIP 157, you may have a
>> huge discrepancy between what is asked and what is offered. Assuming 10M
>> light clients [0] each of them consuming ~100MB/month for filters/headers,
>> that means you're asking 1PB/month of traffic to the backbone network. If
>> you assume 10K public nodes, like today, assuming _all_ of them opt-in to
>> signal BIP 157, that's an increase of 100GB/month for each. Which is
>> consequent with regards to the estimated cost of 350GB/month for running an
>> actual public node. Widening full-node adoption, specially in term of
>> geographic distribution means as much as we can to bound its operational
>> cost.
>>
>> Obviously,  deployment of more efficient tx-relay protocol like Erlay
>> will free up some resources but it maybe wiser to dedicate them to increase
>> health and security of the backbone network like deploying more outbound
>> connections.
>>
>> Unless your light client protocol is so ridiculous cheap to rely on
>> niceness of a subset of node operators offering free resources, it won't
>> scale. And it's likely you will always have a ratio disequilibrium between
>> numbers of clients and numbers of full-node, even worst their growth rate
>> won't be the same, first ones are so much easier to setup.
>>
>> It doesn't mean servicing filters for free won't work for now, numbers of
>> BIP157 clients is still pretty low, but what is worrying is  wallet vendors
>> building such chain access backend, hitting a bandwidth scalability wall
>> few years from now instead of pursuing better solutions. And if this
>> happen, maybe suddenly, isn't the quick fix going to be to rely on
>> centralized services, so much easier to deploy ?
>>
>> Of course, it may be brought that actually current full-node operators
>> don't get anything back from servicing blocks, transactions, addresses...
>> It may be replied that you have an indirect incentive to participate in
>> network relay and therefore guarantee censorship-resistance, instead of
>> directly connecting to miners. You do have today ways to select your
>> resources exposure like pruning, block-only or being private but the wider
>> point is the current (non?)-incentives model seems to work for the base
>> layer. For light clients data, are node operators going to be satisfied to
>> serve this new *class* of traffic en masse ?
>>
>> This doesn't mean you won't find BIP157 servers, ready to serve you with
>> unlimited credit, but it's more likely their intentions maybe not aligned,
>> like spying on your transaction broadcast or block fetched. And you do want
>> peer diversity to avoid every BIP157 servers being on few ASNs for
>> fault-tolerance. Do people expect a scenario a la Cloudflare, where
>> everyone connections is to far or less the same set of entities ?
>>
>> Moreover, the LN security model diverges hugely from basic on-chain
>> transactions. Worst-case attack on-chain a malicious light client server
>> showing a longest, invalid, PoW-signed chain to double-spend the user. On
>> LN, the *liveliness* requirement means the entity owning your view of the
>> chain can lie to you on whether your channel has been spent by a revoked
>> commitment, the real tip of the blockchain or even dry-up block
>> announcement to trigger unexpected behavior in the client logic. A
>> malicious light client server may just drop any filters/utxos spends, what
>> your LN client should do in this case ? [1]
>>
>> Therefore, you may want to introduce monetary compensation in exchange of
>> servicing filters. Light client not dedicating resources to maintain the
>> network but free-riding on it, you may use their micro-payment capabilities
>> to price chain access resources [3]. This proposition may suit within the
>> watchtower paradigm, where another entity is delegated some part of
>> protocol execution, alleviating client onliness requirement. It needs
>> further analysis but how your funds may be compromised by a watchtower are
>> likely to be the same scenario that how a chain validation provider can
>> compromise you. That said, how do you avoid such "chain access" market
>> turning as an oligopoly is an open question. You may "bind" them to
>> internet topology or ask for fidelity bonds and create some kind of
>> scarcity but still...
>>
>> Maybe I'm completely wrong, missing some numbers, and it's maybe fine to
>> just rely on few thousands of full-node operators being nice and servicing
>> friendly millions of LN mobiles clients. But just in case it may be good to
>> consider a reasonable alternative.
>>
>> Thanks Gleb for many points exposed here but all mistakes are my own.
>>
>> Cheers,
>>
>> Antoine
>>
>> [0] UTXO set size may be a bottleneck, but still if you have 2 channels
>> by clients that's 20M utxos, just roughly ~x3 than today.
>>
>> [1] And committing filters as part of headers may not solve everything as
>> an attacker can just delay or slow announcements to you, so you still need
>> network access to at least one honest node.
>>
>> [2]  It maybe argue that distinction client-vs-peer doesn't hold because
>> you may start as a client and start synchronizing the chain, relaying
>> blocks, etc. AFAIK, there is no such hybrid implementation and that's not
>> what you want to run in a mobile.
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
>
> --
> *Igor Cota*
> Codex Apertus Ltd
>

[-- Attachment #2: Type: text/html, Size: 13963 bytes --]