[Bitcoin-development] soft-fork block size increase (extension blocks)

[Bitcoin-development] soft-fork block size increase (extension blocks)

[Adam Back]

Hi Gavin

Sorry for slow response & broken threading - mailbox filled up & only
saw your response on archive.

I do earnestly think opt-in block-size increases are politically
cleaner (gives different people different sized blocks by their own
volition without forcing a compromise) and less risky than hard forks.
Particularly if a hard-fork were really provoked without clear and
wide consensus - dragons lay there.

> Then ask the various wallet developer how long it would take them to update
> their software to support something like this,

I don't think thats any particular concern, extension block payments
are forwards and backwards compatible.  Businesses who are keen to
have more transactions, would make it their problem to implement in
their wallet, or ask the wallet vendor/maintainer they're working with
to do it.  Nothing breaks if they dont use it.  The people that have
the need for it will work on it.  Market at work.  If it turns out
they dont really have a need for it, just projected huge numbers for
their business plan that say dont materialise, well no foul.

> and do some UI mockups of what the experience would look like for users.

I am not a UX guy, but for example it might be appropriate for tipping
services or other micropayments to use an extension block.  Or small
retail payments.  They can choose what address they use.  Merchants,
integrators etc can do likewise.
It gives plenty enough scope that people can work with useful
trade-offs while others work on lightning.

> If there are two engineering solutions to a problem, one really simple, and
> one complex, why would you pick the complex one?

Because the more complex one is safer, more flexible, more future
proof and better for decentralisation (and so as a bonus and might
actually get done without more months of argument as its less
contentious because it gives users choice to opt-in).  Bitcoin itself
is complex, a central ledger is simpler but as we know uninteresting
which is to say this is a security tradeoff.

Obviously I do appreciate KISS as a design principle, and utility of
incremental improvements, but this is a security trade-off we're
discussing here.  I am proposing a way to not weaken security, while
getting what you think is important - access to more TPS with a higher
centralisation tradeoff (for those who opt-in to it, rather than for
everyone whether that tradeoff is strongly against their interests or
not).

The decentralisation metrics are getting worse, not better, see Greg
Maxwell's comments
http://www.reddit.com/r/Bitcoin/comments/37vg8y/is_the_blockstream_company_the_reason_why_4_core/crqg381

This would not by those metrics be a good moment in history to make
the situation worse.

> Especially if the complex solution has all of the problems of the simple
> one (20MB extension blocks are just as "dangerous" as 20MB main blocks,
> yes? If not, why not?)

Not at all, thats the point.  Bitcoin has a one-size fits all
blocksize.  People can pool mine the 8MB extension block, while solo
or GBT mining the 1MB block.  Thats more centralising than staying at
1MB (because to get the fees from the extension block some people
without that kind of bandwidth are pool mining 8/9th of the lower
security/decentralisation transactions.  But its less centralising
than a fixed blocksize of 9MB (1+8 for apples/apples) because
realistically if those transactions are not spam, they would've
happened offchain, and offchain until we get lightning like systems
up, means central systems which are worse than the slight
centralisation of 8MB blocks being single servers and prone to custody
& security failure.  I think you made that point yourself in a recent
post also.

Sound good? ;)  Seriously I think its the least bad idea I've heard on
this topic.

As an aside, a risk with using companies as a sounding board, is that
you can get a misleading sense of consensus.  Did they understand the
tradeoff between security (decentralisation) and blocksize.  Did they
care?  Do they represent users interests?  Would they have "voted"
instead for extension blocks if it was presented in similar terms?  (I
have to imagine they might have preferred extension blocks given the
better story if you gloss over complexity and tradeoffs).

Adam

Re: [Bitcoin-development] soft-fork block size increase (extension blocks)

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3153 bytes --]

Hi Adam,

I have more experience than Gavin of building consumer wallets, so I'll
make an attempt to answer your questions.

> Then ask the various wallet developer how long it would take them to
> update
> > their software to support something like this,
>
> I don't think thats any particular concern


I am a wallet developer and I am telling you that it is.


> Businesses who are keen to
> have more transactions, would make it their problem to implement in
> their wallet, or ask the wallet vendor/maintainer they're working with
> to do it.  Nothing breaks if they dont use it.


I don't see how this is the case. If an exchange supports extension blocks
and I withdraw from that to a wallet that doesn't, the money will never
arrive from my perspective. Yet the exchange will claim they sent it and
they will wash their hands of the matter. Disaster.

I am not a UX guy
>

But I am. I've designed both consumer and engineering UI's at Google, and
also more recently for Lighthouse.

Attempting to explain to a user why they sent money that didn't show up on
the other end is a non starter. It's bad enough when things take a long
time to confirm or bugs cause propagation failures. Doing it
deliberately is not going to work. Payments *must* be reliable and wallets
*must* be compatible with each other.

This is one reason why a Lightning style approach also isn't going to work
any time soon. For example, it would require people to abandon Bitcoin
addresses. I pushed for that before, around the P2SH time, and Gavin
correctly intuited that the community wasn't ready for it yet. I'm not sure
much has changed.


> Because the more complex one is safer, more flexible, more future
> proof and better for decentralisation


I disagree with all of those points. I find Lightning/Stroem etc to be more
dangerous, less flexible, and worse for decentralisation. I explain why
here:

https://medium.com/@octskyward/the-capacity-cliff-586d1bf7715e

You mentioned decentralisation metrics. Gregory's post is ignoring one of
the most important decentralisation metrics, which is number of wallets
made by independent developers. That has got dramatically better over time.
It would get worse if wallets became more complex very suddenly.


> As an aside, a risk with using companies as a sounding board, is that
> you can get a misleading sense of consensus.


From what I can tell Blockstream employees are just ignoring those
companies entirely, which will give you a radically more distorted view of
the consensus. As companies providing services to our community have
serious economic weight, it stands to reason that their opinions would
matter a great deal. Yet on this mailing list I see zero effort to even
recognise their concerns, let alone care about them.
  <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>
Anyway, let me repeat again to make it clear - as someone who has spent
five years writing SPV wallets, I am not on board with extension blocks or
any other Rube Goldberg contraption that exists purely to work around
theoretical objections by Blockstream employees+Peter Todd, which is what
this feels like to me.

[-- Attachment #2: Type: text/html, Size: 4642 bytes --]

Re: [Bitcoin-development] soft-fork block size increase (extension blocks)

[Adam Back]

Mike wrote:
>> Businesses who are keen to
>> have more transactions, would make it their problem to implement in
>> their wallet, or ask the wallet vendor/maintainer they're working with
>> to do it.  Nothing breaks if they dont use it.
>
>
> I don't see how this is the case. If an exchange supports extension blocks
> and I withdraw from that to a wallet that doesn't, the money will never
> arrive from my perspective. Yet the exchange will claim they sent it and
> they will wash their hands of the matter. Disaster.

To be clear in case you are missing part of the mechanism.: it is
forward and backwards compatible meaning a 1MB address can receive
payments from an 8MB address (at reduced security if it has software
that doesnt understand it) and a 1MB address can pay an 8MB address by
paying to an OP_TRUE that has meaning to the extension block nodes.

A 1MB client wont even understand the difference between a 1MB and 8MB
out payment.  An 8MB client will understand and pay 1MB addresses in a
different way (moving the coin back to the 1MB chain).

So its opt-in and incrementally deployable.  Exchanges could encourage
their users to use wallets that support 8MB blocks, eg by charging a
fee for 1MB transactions.  If 1MB blocks experience significant fee
pressure, this will be persuasive.  Or they could chose not to and eat
the cost.  This is all normal market adoption of a new cheaper
technical option (in this case with a tradeoff of reduced
security/more centralisation for those opting in to it).

>> Because the more complex one is safer, more flexible, more future
>> proof and better for decentralisation
>
> I disagree with all of those points. I find Lightning/Stroem etc to be more
> dangerous, less flexible, and worse for decentralisation. I explain why
> here:

Extension blocks & lightning are unrelated things.

While I understand the need for being practical, there is IMO, amongst
engineering maxims something as far as being too pragmatic,
dangerously pragmatic even.  We cant do stuff in bitcoin that has bad
carry costs, nor throw out the baby with the bathwater.

The situation is just that we are facing a security vs volume tradeoff
and different people will have different requirements and comfort
zones.  If I am not misremembering, I think you've sided typically
with the huge block, big data center only end of the spectrum.  What I
am proposing empowers you to do experiments in that direction without
getting into a requirements conflict with people who value more
strongly the bitcoin properties arising from it being robustly
decentralised.

I am not sure personally where the blocksize discussion comes out - if
it stays as is for a year, in a wait and see, reduce spam, see
fee-pressure take effect as it has before, work on improving improve
decentralisation metrics, relay latency, and do a blocksize increment
to kick the can if-and-when it becomes necessary and in the mean-time
try to do something more long-term ambitious about scale rather than
volume.  Bitcoin without scale improvements probably wont get the
volume people would like.  So scale is more important than volume; and
security (decentralisation) is important too.  To the extreme analogy
we could fix scale tomorrow by throwing up a single high perf
database, but then we'd break the security properties arising from
decentralisation.  We should improve both within an approximately safe
envelope IMO.

Adam

Re: [Bitcoin-development] soft-fork block size increase (extension blocks)

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1984 bytes --]

>
> (at reduced security if it has software that doesnt understand it)


Well, yes. Isn't that rather key to the issue?  Whereas by simply
increasing the block size, SPV wallets don't care (same security and
protocol as before) and fully validating wallets can be updated with a very
small code change.


> A 1MB client wont even understand the difference between a 1MB and 8MB
> out payment.


Let's say an old client makes a payment that only gets confirmed in an
extension block. The wallet will think the payment is unconfirmed and show
that to the user forever, no?

Can you walk through the UX for each case?


> If I am not misremembering, I think you've sided typically
> with the huge block, big data center only end of the spectrum.


It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure
how this meme has taken hold amongst you guys, as I am the guy who wrote
the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability

It says:

*The core Bitcoin network can scale to much higher transaction rates than
are seen today, assuming that nodes in the network are primarily running on
high end servers rather than desktops. *


By "much higher rates" I meant VISA scale and by "high end server" I meant
high end by today's standards not tomorrows. There's a big difference
between a datacenter and a single server! By definition a single server is
not a datacenter, although it would be conventional to place it in
one. But even
with the most wildly optimistic growth imaginable, I couldn't foresee a
time when you needed more than a single machine to keep up with the
transaction stream.

And we're not going to get to VISA scale any time soon: I don't think I've
ever argued we will. If it does happen it would presumably be decades away.
Again, short of some currently unimagined killer app.

So I don't believe I've ever argued this, and honestly I kinda feel people
are putting words in my mouth.

[-- Attachment #2: Type: text/html, Size: 4129 bytes --]

Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

[Raystonn .]

[-- Attachment #1.1: Type: text/plain, Size: 3352 bytes --]

> And we're not going to get to VISA scale any time soon

No, not at these block size limits.  The closer we get to the maximum block size, the slower we grow the average block size toward it.  Number of transactions per day is of course highly correlated with average block size.  Based on these graphs we can expect that hitting 1 million transactions per day will be impossible without raising the maximum block size.


https://blockchain.info/charts/avg-block-size?showDataPoints=false&show_header=true&daysAverageString=7&timespan=all&scale=1&address=



https://blockchain.info/charts/n-transactions?showDataPoints=false&timespan=all&show_header=true&daysAverageString=7&scale=1&address= 



From: Mike Hearn 
Sent: Monday, June 01, 2015 11:01 AM
To: Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

  (at reduced security if it has software that doesnt understand it) 

Well, yes. Isn't that rather key to the issue?  Whereas by simply increasing the block size, SPV wallets don't care (same security and protocol as before) and fully validating wallets can be updated with a very small code change.

  A 1MB client wont even understand the difference between a 1MB and 8MB
  out payment. 

Let's say an old client makes a payment that only gets confirmed in an extension block. The wallet will think the payment is unconfirmed and show that to the user forever, no?

Can you walk through the UX for each case?

  If I am not misremembering, I think you've sided typically
  with the huge block, big data center only end of the spectrum.  

It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure how this meme has taken hold amongst you guys, as I am the guy who wrote the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability


It says:

  The core Bitcoin network can scale to much higher transaction rates than are seen today, assuming that nodes in the network are primarily running on high end servers rather than desktops. 


By "much higher rates" I meant VISA scale and by "high end server" I meant high end by today's standards not tomorrows. There's a big difference between a datacenter and a single server! By definition a single server is not a datacenter, although it would be conventional to place it in one. But even with the most wildly optimistic growth imaginable, I couldn't foresee a time when you needed more than a single machine to keep up with the transaction stream. 


And we're not going to get to VISA scale any time soon: I don't think I've ever argued we will. If it does happen it would presumably be decades away. Again, short of some currently unimagined killer app.


So I don't believe I've ever argued this, and honestly I kinda feel people are putting words in my mouth.


--------------------------------------------------------------------------------
------------------------------------------------------------------------------



--------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[-- Attachment #1.2: Type: text/html, Size: 8113 bytes --]

[-- Attachment #2: image[2].png --]
[-- Type: image/png, Size: 53618 bytes --]

[-- Attachment #3: image[12].png --]
[-- Type: image/png, Size: 50593 bytes --]

Re: [Bitcoin-development] soft-fork block size increase (extension blocks)

[Tom Harding]

On 6/1/2015 10:21 AM, Adam Back wrote:
> if it stays as is for a year, in a wait and see, reduce spam, see
> fee-pressure take effect as it has before, work on improving improve
> decentralisation metrics, relay latency, and do a blocksize increment
> to kick the can if-and-when it becomes necessary and in the mean-time
> try to do something more long-term ambitious about scale rather than
> volume.

What's your estimate of the lead time required to kick the can,
if-and-when it becomes necessary?

The other time-series I've seen all plot an average block size.  That's
misleading, because there's a distribution of block sizes.  If you bin
by retarget interval and plot every single block, you get this

http://i.imgur.com/5Gfh9CW.png

The max block size has clearly been in play for 8 months already.

Re: [Bitcoin-development] soft-fork block size increase (extension blocks)

[Richard Moore]

[-- Attachment #1: Type: text/plain, Size: 1740 bytes --]

Awesome graph!

It would be interesting if you could use your Excel voodoo (or whatever graph app you use) to add density as a heat map, making areas with more blocks red and regions with less blocks blue…


> On Jun 1, 2015, at 8:42 PM, Tom Harding <tomh@thinlink.com> wrote:
> 
> On 6/1/2015 10:21 AM, Adam Back wrote:
>> if it stays as is for a year, in a wait and see, reduce spam, see
>> fee-pressure take effect as it has before, work on improving improve
>> decentralisation metrics, relay latency, and do a blocksize increment
>> to kick the can if-and-when it becomes necessary and in the mean-time
>> try to do something more long-term ambitious about scale rather than
>> volume.
> 
> What's your estimate of the lead time required to kick the can,
> if-and-when it becomes necessary?
> 
> The other time-series I've seen all plot an average block size.  That's
> misleading, because there's a distribution of block sizes.  If you bin
> by retarget interval and plot every single block, you get this
> 
> http://i.imgur.com/5Gfh9CW.png
> 
> The max block size has clearly been in play for 8 months already.
> 
> 
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸><(((º>

Richard Moore ~ Founder
Genetic Mistakes Software inc.
phone: (778) 882-6125
email: ricmoo@geneticmistakes.com <mailto:ricmoo@geneticmistakes.com>
www: http://GeneticMistakes.com <http://geneticmistakes.com/>

[-- Attachment #2: Type: text/html, Size: 3314 bytes --]

Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

[Raystonn .]

[-- Attachment #1.1: Type: text/plain, Size: 3781 bytes --]

I also need to argue for increasing the default block limit to the full 1MB in the next release.  We’re already hitting that limit in bursts of transactions, which puts pressure on the average displayed in the below graphs.

From: raystonn@hotmail.com 
Sent: Monday, June 01, 2015 11:39 AM
To: Mike Hearn ; Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

> And we're not going to get to VISA scale any time soon

No, not at these block size limits.  The closer we get to the maximum block size, the slower we grow the average block size toward it.  Number of transactions per day is of course highly correlated with average block size.  Based on these graphs we can expect that hitting 1 million transactions per day will be impossible without raising the maximum block size.


https://blockchain.info/charts/avg-block-size?showDataPoints=false&show_header=true&daysAverageString=7&timespan=all&scale=1&address=



https://blockchain.info/charts/n-transactions?showDataPoints=false&timespan=all&show_header=true&daysAverageString=7&scale=1&address= 



From: Mike Hearn 
Sent: Monday, June 01, 2015 11:01 AM
To: Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

  (at reduced security if it has software that doesnt understand it) 

Well, yes. Isn't that rather key to the issue?  Whereas by simply increasing the block size, SPV wallets don't care (same security and protocol as before) and fully validating wallets can be updated with a very small code change.

  A 1MB client wont even understand the difference between a 1MB and 8MB
  out payment. 

Let's say an old client makes a payment that only gets confirmed in an extension block. The wallet will think the payment is unconfirmed and show that to the user forever, no?

Can you walk through the UX for each case?

  If I am not misremembering, I think you've sided typically
  with the huge block, big data center only end of the spectrum.  

It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure how this meme has taken hold amongst you guys, as I am the guy who wrote the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability


It says:

  The core Bitcoin network can scale to much higher transaction rates than are seen today, assuming that nodes in the network are primarily running on high end servers rather than desktops. 


By "much higher rates" I meant VISA scale and by "high end server" I meant high end by today's standards not tomorrows. There's a big difference between a datacenter and a single server! By definition a single server is not a datacenter, although it would be conventional to place it in one. But even with the most wildly optimistic growth imaginable, I couldn't foresee a time when you needed more than a single machine to keep up with the transaction stream. 


And we're not going to get to VISA scale any time soon: I don't think I've ever argued we will. If it does happen it would presumably be decades away. Again, short of some currently unimagined killer app.


So I don't believe I've ever argued this, and honestly I kinda feel people are putting words in my mouth.


--------------------------------------------------------------------------------
------------------------------------------------------------------------------



--------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[-- Attachment #1.2: Type: text/html, Size: 9530 bytes --]

[-- Attachment #2: image[2].png --]
[-- Type: image/png, Size: 53618 bytes --]

[-- Attachment #3: image[12].png --]
[-- Type: image/png, Size: 50593 bytes --]

Re: [Bitcoin-development] soft-fork block size increase(extensionblocks)

[Raystonn .]

[-- Attachment #1: Type: text/plain, Size: 4502 bytes --]

Wow.  That email was delayed by the list for quite some time.  It was sent on 6/1.
From: Raystonn . 
Sent: Monday, June 01, 2015 12:02 PM
To: Mike Hearn ; Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase(extensionblocks)

I also need to argue for increasing the default block limit to the full 1MB in the next release.  We’re already hitting that limit in bursts of transactions, which puts pressure on the average displayed in the below graphs.

From: raystonn@hotmail.com 
Sent: Monday, June 01, 2015 11:39 AM
To: Mike Hearn ; Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

> And we're not going to get to VISA scale any time soon

No, not at these block size limits.  The closer we get to the maximum block size, the slower we grow the average block size toward it.  Number of transactions per day is of course highly correlated with average block size.  Based on these graphs we can expect that hitting 1 million transactions per day will be impossible without raising the maximum block size.


https://blockchain.info/charts/avg-block-size?showDataPoints=false&show_header=true&daysAverageString=7&timespan=all&scale=1&address=



https://blockchain.info/charts/n-transactions?showDataPoints=false&timespan=all&show_header=true&daysAverageString=7&scale=1&address= 



From: Mike Hearn 
Sent: Monday, June 01, 2015 11:01 AM
To: Adam Back 
Cc: Bitcoin Dev 
Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

  (at reduced security if it has software that doesnt understand it) 

Well, yes. Isn't that rather key to the issue?  Whereas by simply increasing the block size, SPV wallets don't care (same security and protocol as before) and fully validating wallets can be updated with a very small code change.

  A 1MB client wont even understand the difference between a 1MB and 8MB
  out payment. 

Let's say an old client makes a payment that only gets confirmed in an extension block. The wallet will think the payment is unconfirmed and show that to the user forever, no?

Can you walk through the UX for each case?

  If I am not misremembering, I think you've sided typically
  with the huge block, big data center only end of the spectrum.  

It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure how this meme has taken hold amongst you guys, as I am the guy who wrote the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability


It says:

  The core Bitcoin network can scale to much higher transaction rates than are seen today, assuming that nodes in the network are primarily running on high end servers rather than desktops. 


By "much higher rates" I meant VISA scale and by "high end server" I meant high end by today's standards not tomorrows. There's a big difference between a datacenter and a single server! By definition a single server is not a datacenter, although it would be conventional to place it in one. But even with the most wildly optimistic growth imaginable, I couldn't foresee a time when you needed more than a single machine to keep up with the transaction stream. 


And we're not going to get to VISA scale any time soon: I don't think I've ever argued we will. If it does happen it would presumably be decades away. Again, short of some currently unimagined killer app.


So I don't believe I've ever argued this, and honestly I kinda feel people are putting words in my mouth.


--------------------------------------------------------------------------------
------------------------------------------------------------------------------



--------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development



--------------------------------------------------------------------------------
------------------------------------------------------------------------------



--------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[-- Attachment #2: Type: text/html, Size: 10526 bytes --]

Re: [Bitcoin-development] soft-fork block size increase(extensionblocks)

[Tom Harding]

"Increase DEFAULT_BLOCK_MAX_SIZE to 1MB"
https://github.com/bitcoin/bitcoin/pull/6231

On 6/17/2015 8:28 AM, Raystonn . wrote:
> Wow.  That email was delayed by the list for quite some time.  It was 
> sent on 6/1.
>
>> I also need to argue for increasing the default block limit to the 
>> full 1MB in the next release. We’re already hitting that limit in 
>> bursts of transactions, which puts pressure on the average displayed 
>> in the below graphs.
>
>