From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id B5E3AC0859; Wed, 6 May 2020 09:40:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A3E1E8869A; Wed, 6 May 2020 09:40:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2GKQnNd5Olxj; Wed, 6 May 2020 09:40:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by hemlock.osuosl.org (Postfix) with ESMTPS id BB4D488699; Wed, 6 May 2020 09:40:19 +0000 (UTC) Received: by mail-pl1-f178.google.com with SMTP id b6so249046plz.13; Wed, 06 May 2020 02:40:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=OqhD+GagQxLb7KVji29P74KOaYiVDYfTpFUziRG+GWk=; b=pZqXfR7QTCftYj1/PPImlPEfwSvTGKp17iUSNFl0Beki+Qad4QsGTzdgPtoP+XJHDD 31c4OrZ2j0Z555gu2V8xbWG34AxqMKneqdd+gjKh+U0lAgcnqoqWKd5i47ULzpXgOzNP ws5DqlPX96mhQfMtNgmpOxzUgZ+WuHty8vVsAz/aX+8IINKO4MExL7xs7kEnscChFJ2v LYH7o0TI15WhnkN1mQdwkqnuNP30Fkm7YWxMqYGkpRh6q3i01VOsJR3lWjfROLQdZPKy C1IhQywTa2WYgLnDdVViY3qoy3zv18tlgYtRlQiPmS0GpoH2d/AIgSXhD1K1tSwgPbA0 Cvmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OqhD+GagQxLb7KVji29P74KOaYiVDYfTpFUziRG+GWk=; b=M8k158gg2zOVaB71ZMXDutXC3FDh8fgck4UKK5upl+HVaZ4JFPWoUO4SlpkwAt6SIm 7LxKML92XKrWRF2l22jYbAQd9INnhyzcYCkzPLaZsZpL0tJTSxrAFOrCxEG/0aBbuk0Z dwkx4FM4H0gTtUidG5BAKErWiliFgprRJuwxBt+DBwkcmMyoWumlOZt+yCI1oESzhglC zvF7/1a9+zmXYLOLimdKA6ILd93FsQx42cg/0NWNr87zjO/syzYs15YTacbWgWg4UShO yzxySof/Brez8DJf7ySKs7dMw6KkjxMBtK/b83IJQiqLVtb7gBu8Z9NJ9qg6fcmKeAfT ZmIg== X-Gm-Message-State: AGi0PuYg8KFZBy12J1xOtBUHGQOnaM3pi8I32CW6QoTL8M6bTNWewvYa Lf55J1LaqVsClBPnja5YT9sHNQldtyIwHF/CorY= X-Google-Smtp-Source: APiQypJ/Ypm2XtO4DSsEqBZMcq+W36g551pUENimz3f87q77DJlDsGGurDVKeF7WKm78a+L7wIJb9Di4niF4CivFQ/4= X-Received: by 2002:a17:902:8ec1:: with SMTP id x1mr6931760plo.325.1588758019215; Wed, 06 May 2020 02:40:19 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Antoine Riard Date: Wed, 6 May 2020 05:40:06 -0400 Message-ID: To: Olaoluwa Osuntokun Content-Type: multipart/alternative; boundary="0000000000005e977e05a4f78dde" X-Mailman-Approved-At: Wed, 06 May 2020 13:25:00 +0000 Cc: Bitcoin Protocol Discussion , "lightning-dev\\\\@lists.linuxfoundation.org" Subject: Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2020 09:40:21 -0000 --0000000000005e977e05a4f78dde Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > As a result, the entire protocol could be served over something like HTTP, taking advantage of all the established CDNs and anycast serving infrastructure, Yes it's moving the issue of being a computation one to a distribution one. But still you need the bandwidth capacities. What I'm concerned is the trust model of relying on few-establish CDNs, you don't want to make it easy to have "headers-routing" hijack and therefore having massive channel closure or time-locks interference due to LN clients not seeing the last few block. So you may want to separate control/data plane, get filters from CDN and headers as check-and-control directly from the backbone network. "Hybrid" models should clearly be explored. Web-of-trust style of deployments should be also envisioned, you may get huge scaling improvement, assuming client may be peers between themselves and the ones belonging to the same social entity should be able to share the same chain view without too much risk. > Piggy backing off the above idea, if the data starts being widely served over HTTP, then LSATs[1][2] can be used to add a lightweight payment mechanism by inserting a new proxy server in front of the filter/header infrastructure. Yeah, I hadn't time to read the spec yet but that was clearly something like LSATs I meaned speaking about monetary compensation to price resources. I just hope it isn't too much tie to HTTP because you may want to read/write over other communication channels like tx-broadcast-over-radio to solve first-hop privacy. Le mar. 5 mai 2020 =C3=A0 20:31, Olaoluwa Osuntokun a = =C3=A9crit : > Hi Antoine, > > > Even with cheaper, more efficient protocols like BIP 157, you may have = a > > huge discrepancy between what is asked and what is offered. Assuming 10= M > > light clients [0] each of them consuming ~100MB/month for > filters/headers, > > that means you're asking 1PB/month of traffic to the backbone network. = If > > you assume 10K public nodes, like today, assuming _all_ of them opt-in = to > > signal BIP 157, that's an increase of 100GB/month for each. Which is > > consequent with regards to the estimated cost of 350GB/month for runnin= g > > an actual public node > > One really dope thing about BIP 157+158, is that the protocol makes servi= ng > light clients now _stateless_, since the full node doesn't need to perfor= m > any unique work for a given client. As a result, the entire protocol coul= d > be served over something like HTTP, taking advantage of all the establish= ed > CDNs and anycast serving infrastructure, which can reduce syncing time > (less latency to > fetch data) and also more widely distributed the load of light clients > using > the existing web infrastructure. Going further, with HTTP/2's server-push > capabilities, those serving this data can still push out notifications fo= r > new headers, etc. > > > Therefore, you may want to introduce monetary compensation in exchange = of > > servicing filters. Light client not dedicating resources to maintain th= e > > network but free-riding on it, you may use their micro-payment > > capabilities to price chain access resources [3] > > Piggy backing off the above idea, if the data starts being widely served > over HTTP, then LSATs[1][2] can be used to add a lightweight payment > mechanism by inserting a new proxy server in front of the filter/header > infrastructure. The minted tokens themselves may allow a user to purchase > access to a single header/filter, a range of them in the past, or N heade= rs > past the known chain tip, etc, etc. > > -- Laolu > > [1]: https://lsat.tech/ > [2]: https://lightning.engineering/posts/2020-03-30-lsat/ > > > On Tue, May 5, 2020 at 3:17 AM Antoine Riard > wrote: > >> Hi, >> >> (cross-posting as it's really both layers concerned) >> >> Ongoing advancement of BIP 157 implementation in Core maybe the >> opportunity to reflect on the future of light client protocols and use t= his >> knowledge to make better-informed decisions about what kind of >> infrastructure is needed to support mobile clients at large scale. >> >> Trust-minimization of Bitcoin security model has always relied first and >> above on running a full-node. This current paradigm may be shifted by LN >> where fast, affordable, confidential, censorship-resistant payment servi= ces >> may attract a lot of adoption without users running a full-node. Assumin= g a >> user adoption path where a full-node is required to benefit for LN may >> deprive a lot of users, especially those who are already denied a real >> financial infrastructure access. It doesn't mean we shouldn't foster nod= e >> adoption when people are able to do so, and having a LN wallet maybe eve= n a >> first-step to it. >> >> Designing a mobile-first LN experience opens its own gap of challenges >> especially in terms of security and privacy. The problem can be scoped a= s >> how to build a scalable, secure, private chain access backend for millio= ns >> of LN clients ? >> >> Light client protocols for LN exist (either BIP157 or Electrum are used)= , >> although their privacy and security guarantees with regards to >> implementation on the client-side may still be an object of concern >> (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted = fee >> estimation). That said, one of the bottlenecks is likely the number of >> full-nodes being willingly to dedicate resources to serve those clients. >> It's not about _which_ protocol is deployed but more about _incentives_ = for >> node operators to dedicate long-term resources to client they have lower >> reasons to care about otherwise. >> >> Even with cheaper, more efficient protocols like BIP 157, you may have a >> huge discrepancy between what is asked and what is offered. Assuming 10M >> light clients [0] each of them consuming ~100MB/month for filters/header= s, >> that means you're asking 1PB/month of traffic to the backbone network. I= f >> you assume 10K public nodes, like today, assuming _all_ of them opt-in t= o >> signal BIP 157, that's an increase of 100GB/month for each. Which is >> consequent with regards to the estimated cost of 350GB/month for running= an >> actual public node. Widening full-node adoption, specially in term of >> geographic distribution means as much as we can to bound its operational >> cost. >> >> Obviously, deployment of more efficient tx-relay protocol like Erlay >> will free up some resources but it maybe wiser to dedicate them to incre= ase >> health and security of the backbone network like deploying more outbound >> connections. >> >> Unless your light client protocol is so ridiculous cheap to rely on >> niceness of a subset of node operators offering free resources, it won't >> scale. And it's likely you will always have a ratio disequilibrium betwe= en >> numbers of clients and numbers of full-node, even worst their growth rat= e >> won't be the same, first ones are so much easier to setup. >> >> It doesn't mean servicing filters for free won't work for now, numbers o= f >> BIP157 clients is still pretty low, but what is worrying is wallet vend= ors >> building such chain access backend, hitting a bandwidth scalability wall >> few years from now instead of pursuing better solutions. And if this >> happen, maybe suddenly, isn't the quick fix going to be to rely on >> centralized services, so much easier to deploy ? >> >> Of course, it may be brought that actually current full-node operators >> don't get anything back from servicing blocks, transactions, addresses..= . >> It may be replied that you have an indirect incentive to participate in >> network relay and therefore guarantee censorship-resistance, instead of >> directly connecting to miners. You do have today ways to select your >> resources exposure like pruning, block-only or being private but the wid= er >> point is the current (non?)-incentives model seems to work for the base >> layer. For light clients data, are node operators going to be satisfied = to >> serve this new *class* of traffic en masse ? >> >> This doesn't mean you won't find BIP157 servers, ready to serve you with >> unlimited credit, but it's more likely their intentions maybe not aligne= d, >> like spying on your transaction broadcast or block fetched. And you do w= ant >> peer diversity to avoid every BIP157 servers being on few ASNs for >> fault-tolerance. Do people expect a scenario a la Cloudflare, where >> everyone connections is to far or less the same set of entities ? >> >> Moreover, the LN security model diverges hugely from basic on-chain >> transactions. Worst-case attack on-chain a malicious light client server >> showing a longest, invalid, PoW-signed chain to double-spend the user. O= n >> LN, the *liveliness* requirement means the entity owning your view of th= e >> chain can lie to you on whether your channel has been spent by a revoked >> commitment, the real tip of the blockchain or even dry-up block >> announcement to trigger unexpected behavior in the client logic. A >> malicious light client server may just drop any filters/utxos spends, wh= at >> your LN client should do in this case ? [1] >> >> Therefore, you may want to introduce monetary compensation in exchange o= f >> servicing filters. Light client not dedicating resources to maintain the >> network but free-riding on it, you may use their micro-payment capabilit= ies >> to price chain access resources [3]. This proposition may suit within th= e >> watchtower paradigm, where another entity is delegated some part of >> protocol execution, alleviating client onliness requirement. It needs >> further analysis but how your funds may be compromised by a watchtower a= re >> likely to be the same scenario that how a chain validation provider can >> compromise you. That said, how do you avoid such "chain access" market >> turning as an oligopoly is an open question. You may "bind" them to >> internet topology or ask for fidelity bonds and create some kind of >> scarcity but still... >> >> Maybe I'm completely wrong, missing some numbers, and it's maybe fine to >> just rely on few thousands of full-node operators being nice and servici= ng >> friendly millions of LN mobiles clients. But just in case it may be good= to >> consider a reasonable alternative. >> >> Thanks Gleb for many points exposed here but all mistakes are my own. >> >> Cheers, >> >> Antoine >> >> [0] UTXO set size may be a bottleneck, but still if you have 2 channels >> by clients that's 20M utxos, just roughly ~x3 than today. >> >> [1] And committing filters as part of headers may not solve everything a= s >> an attacker can just delay or slow announcements to you, so you still ne= ed >> network access to at least one honest node. >> >> [2] It maybe argue that distinction client-vs-peer doesn't hold because >> you may start as a client and start synchronizing the chain, relaying >> blocks, etc. AFAIK, there is no such hybrid implementation and that's no= t >> what you want to run in a mobile. >> >> _______________________________________________ >> Lightning-dev mailing list >> Lightning-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >> > --0000000000005e977e05a4f78dde Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
> As a result, the entire protocol = could be served over something like HTTP, taking advantage of all the estab= lished CDNs and anycast serving infrastructure,

Yes it's m= oving the issue of being a computation one to a distribution one. But still= you need the bandwidth capacities. What I'm concerned is the trust mod= el of relying on few-establish CDNs, you don't want to make it easy to = have "headers-routing" hijack and therefore having massive channe= l closure or time-locks interference due to LN clients not seeing the last = few block. So you may want to separate control/data plane, get filters from= CDN and headers as check-and-control directly from the backbone network. &= quot;Hybrid" models should clearly be explored.

Web-= of-trust style of deployments should be also envisioned, you may get huge s= caling improvement, assuming client may be peers between themselves and the= ones belonging to the same social entity should be able to share the same = chain view without too much risk.

> Piggy backing off the above i= dea, if the data starts being widely served
over HTTP, then LSATs[1][2] = can be used to add a lightweight payment
mechanism by inserting a new pr= oxy server in front of the filter/header
infrastructure.

Yeah, I hadn't time to read the spec yet but that was clearly someth= ing like LSATs I meaned speaking about monetary compensation to price resou= rces. I just hope it isn't too much tie to HTTP because you may want to= read/write over other communication channels like tx-broadcast-over-radio = to solve first-hop privacy.

Le=C2=A0mar. 5 mai 2020 = =C3=A0=C2=A020:31, Olaoluwa Osuntokun <laolu32@gmail.com> a =C3=A9crit=C2=A0:
Hi Antoine,

>= Even with cheaper, more efficient protocols like BIP 157, you may have a> huge discrepancy between what is asked and what is offered. Assuming= 10M
> light clients [0] each of them consuming ~100MB/month for filt= ers/headers,
> that means you're asking 1PB/month of traffic to t= he backbone network. If
> you assume 10K public nodes, like today, as= suming _all_ of them opt-in to
> signal BIP 157, that's an increa= se of 100GB/month for each. Which is
> consequent with regards to the= estimated cost of 350GB/month for running
> an actual public node
One really dope thing about BIP 157+158, is that the protocol makes se= rving
light clients now _stateless_, since the full node doesn't nee= d to perform
any unique work for a given client. As a result, the entire= protocol could
be served over something like HTTP, taking advantage of = all the established
CDNs and anycast serving infrastructure, which can r= educe syncing time (less latency to
fetch data) and also more widely dis= tributed the load of light clients using
the existing web infrastructure= . Going further, with HTTP/2's server-push
capabilities, those servi= ng this data can still push out notifications for
new headers, etc.
<= br>> Therefore, you may want to introduce monetary compensation in excha= nge of
> servicing filters. Light client not dedicating resources to = maintain the
> network but free-riding on it, you may use their micro= -payment
> capabilities to price chain access resources [3]

Pi= ggy backing off the above idea, if the data starts being widely served
o= ver HTTP, then LSATs[1][2] can be used to add a lightweight payment
mech= anism by inserting a new proxy server in front of the filter/header
infr= astructure. The minted tokens themselves may allow a user to purchase
ac= cess to a single header/filter, a range of them in the past, or N headerspast the known chain tip, etc, etc.

-- Laolu

[1]: https://lsat.tech/
[2]: https://lightning.engineering/posts/2020-03-30-lsat/


On Tu= e, May 5, 2020 at 3:17 AM Antoine Riard <antoine.riard@gmail.com> wrote:
= Hi,

(cross-posting as it's really both layers = concerned)

Ongoing advancement of BIP 157 implementa= tion in Core maybe the opportunity to reflect on the future of light client= protocols and use this knowledge to make better-informed decisions about w= hat kind of infrastructure is needed to support mobile clients at large sca= le.

Trust-minimization of Bitcoin security model has alwa= ys relied first and above on running a full-node. This current paradigm may= be shifted by LN where fast, affordable, confidential, censorship-resistan= t payment services may attract a lot of adoption without users running a fu= ll-node. Assuming a user adoption path where a full-node is required to ben= efit for LN may deprive a lot of users, especially those who are already de= nied a real financial infrastructure access. It doesn't mean we shouldn= 't foster node adoption when people are able to do so, and having a LN = wallet maybe even a first-step to it.

Designing a mobile-= first LN experience opens its own gap of challenges especially in terms of = security and privacy. The problem can be scoped as how to build a scalable,= secure, private chain access backend for millions of LN clients ?

<= /div>
Light client protocols for LN exist (either BIP157 or Electrum ar= e used), although their privacy and security guarantees with regards to imp= lementation on the client-side may still be an object of concern (aggressiv= e tx-rebroadcast, sybillable outbound peer selection, trusted fee estimatio= n). That said, one of the bottlenecks is likely the number of full-nodes be= ing willingly to dedicate resources to serve those clients. It's not ab= out _which_ protocol is deployed but more about _incentives_ for node opera= tors to dedicate long-term resources to client they have lower reasons to c= are about otherwise.

Even with cheaper, more efficient pr= otocols like BIP 157, you may have a huge discrepancy between what is asked= and what is offered. Assuming 10M light clients [0] each of them consuming= ~100MB/month for filters/headers, that means you're asking 1PB/month o= f traffic to the backbone network. If you assume 10K public nodes, like tod= ay, assuming _all_ of them opt-in to signal BIP 157, that's an increase= of 100GB/month for each. Which is consequent with regards to the estimated= cost of 350GB/month for running an actual public node. Widening full-node = adoption, specially in term of geographic distribution means as much as we = can to bound its operational cost.

Obviously,=C2=A0 deplo= yment of more efficient tx-relay protocol like Erlay will free up some reso= urces but it maybe wiser to dedicate them to increase health and security o= f the backbone network like deploying more outbound connections.

Unless your light client protocol is so ridiculous cheap to rely on= niceness of a subset of node operators offering free resources, it won'= ;t scale. And it's likely you will always have a ratio disequilibrium b= etween numbers of clients and numbers of full-node, even worst their growth= rate won't be the same, first ones are so much easier to setup.
It doesn't mean servicing filters for free won't work f= or now, numbers of BIP157 clients is still pretty low, but what is worrying= is=C2=A0 wallet vendors building such chain access backend, hitting a band= width scalability wall few years from now instead of pursuing better soluti= ons. And if this happen, maybe suddenly, isn't the quick fix going to b= e to rely on centralized services, so much easier to deploy ?

=
Of course, it may be brought that actually current full-node operators= don't get anything back from servicing blocks, transactions, addresses= ... It may be replied that you have an indirect incentive to participate in= network relay and therefore guarantee censorship-resistance, instead of di= rectly connecting to miners. You do have today ways to select your resource= s exposure like pruning, block-only or being private but the wider point is= the current (non?)-incentives model seems to work for the base layer. For = light clients data, are node operators going to be satisfied to serve this = new *class* of traffic en masse ?

This doesn't mean you won'= t find BIP157 servers, ready to serve you with unlimited credit, but it'= ;s more likely their intentions maybe not aligned, like spying on your tran= saction broadcast or block fetched. And you do want peer diversity to avoid= every BIP157 servers being on few ASNs for fault-tolerance. Do people expe= ct a scenario a la Cloudflare, where everyone connections is to far or less= the same set of entities ?

Moreover, the LN security model diverges= hugely from basic on-chain transactions. Worst-case attack on-chain a mali= cious light client server showing a longest, invalid, PoW-signed chain to d= ouble-spend the user. On LN, the *liveliness* requirement means the entity = owning your view of the chain can lie to you on whether your channel has be= en spent by a revoked commitment, the real tip of the blockchain or even dr= y-up block announcement to trigger unexpected behavior in the client logic.= A malicious light client server may just drop any filters/utxos spends, wh= at your LN client should do in this case ? [1]

Therefore, you may wa= nt to introduce monetary compensation in exchange of servicing filters. Lig= ht client not dedicating resources to maintain the network but free-riding = on it, you may use their micro-payment capabilities to price chain access r= esources [3]. This proposition may suit within the watchtower paradigm, whe= re another entity is delegated some part of protocol execution, alleviating= client onliness requirement. It needs further analysis but how your funds = may be compromised by a watchtower are likely to be the same scenario that = how a chain validation provider can compromise you. That said, how do you a= void such "chain access" market turning as an oligopoly is an ope= n question. You may "bind" them to internet topology or ask for f= idelity bonds and create some kind of scarcity but still...

Maybe I&= #39;m completely wrong, missing some numbers, and it's maybe fine to ju= st rely on few thousands of full-node operators being nice and servicing fr= iendly millions of LN mobiles clients. But just in case it may be good to c= onsider a reasonable alternative.

Thanks Gleb for many points expose= d here but all mistakes are my own.

Cheers,

=
Antoine

[0] UTXO set size may be a bottleneck, but still if you= have 2 channels by clients that's 20M utxos, just roughly ~x3 than tod= ay.

[1] And committing filters as part of headers may not solve ever= ything as an attacker can just delay or slow announcements to you, so you s= till need network access to at least one honest node.

[2]=C2=A0 It m= aybe argue that distinction client-vs-peer doesn't hold because you may= start as a client and start synchronizing the chain, relaying blocks, etc.= AFAIK, there is no such hybrid implementation and that's not what you = want to run in a mobile.

_______________________________________________
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/ma= ilman/listinfo/lightning-dev
--0000000000005e977e05a4f78dde--