[bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 6460 bytes --]

1. Summary

Segregated Witness (SegWitness, SW) is being presented in the context of
Scaling Bitcoin.  It has useful attributes, notably addressing a major
malleability vector, but is not a short term scaling solution.


2. Definitions

Import Fee Event, ECE, TFM, FFM from previous email.

Older clients - Any software not upgraded to SW

Newer clients - Upgraded, SW aware software


Block size - refers to the core block economic resource limited by
MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
Requires a hard fork to change.

Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.  Not
changed by SW.


Extended transaction - Newer, upgraded version of transaction data format.

Extended block - Newer, upgraded version of block data format.


EBS - Extended block size.  Block size seen by newer clients.


3. Context of analysis

One proposal presents SW *in lieu of* a hard fork block size increase.
This email focuses directly on that.

Useful features outside block size context, such as anti-malleability or
fraud proof features, are not covered in depth.


4.1.  Observations on data structure formats and views

SW creates two *views* of each transaction and block.  SW has blocks and
extended blocks.  Similarly, there exists transactions and extended
transactions.

This view is rendered to clients depending on compatibility level.  Newer
clients see extended blocks and extended transactions.  Older clients see
blocks (limit 1M), and do not see extended blocks.  Older clients see
upgraded transactions as unsigned, anyone-can-pay transactions.

Each extended transaction exists in two states, one unsigned and one
signed, each of which passes validation as a valid bitcoin transaction.


4.2.  Observations on behavior of older transaction creation

Transactions created by older clients will not use the extended transaction
format.  All data is stored the standard 1M block as today.


4.3.  Observations on new block economic model

SW complicates block economics by creating two separate, supply limited
resources.

The core block economic resource is heavily contended.  Older clients use
core blocks exclusively.  Newer clients use core blocks more
conservatively, storing as much data as possible in extended blocks.

The extended block economic resource is less heavily contended, though that
of course grows over time as clients upgrade.

Because core blocks are more heavily contended, it is presumed that older
clients will pay a higher fee than newer clients (subject to elasticity
etc.).


5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must be
considered.

The current apparent proposal is to roll out Segregated Witness as a soft
fork, and keep block size at 1M.

The roll-out pace cannot simply be judged by soft fork speed - which is
months at best.  Analysis must the layers above:  Updating bitcoin-core
(JS) and bitcoinj (Java), and then the timelines to roll out those updates
to apps, and then the timeline to update those apps to create extended
transactions.

Overall, wallet software and programmer libraries must be upgraded to make
use of this new format, adding many more months (12+ in some stacks) to the
roll out timeline.  In the meantime, clients continue to contend entirely
for core block space.


5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with most
software, unlike SW.

A simple hard fork such as BIP 102 is automatically compatible with the
vast range of today's ecosystem software.

SW requires merchants to upgrade almost immediately, requires wallet and
other peripheral software upgrades to make use of.  Other updates are
opt-in and occur more slowly.  BIP 70 processors need some updates.

The number of LOC that must change for BIP 102 is very small, and the
problem domain well known, versus SW.


5.3.  Problem:   Due to pace, Fee Event not forestalled.

Even presuming SW is merged into Bitcoin Core tomorrow, this does not
address the risk of a Fee Event and associated Economic Change in the
coming months.


5.4.  Problem:   More complex economic policy, new game theory, new bidding
structure risks.

Splitting blocks into two pieces, each with separate and distinct behaviors
and resource values, creates *two fee markets.*

Having two pricing strata within each block has certainly feasible - that
is the current mining policy of (1) fee/KB followed by (2) priority/age.

Valuable or not - e.g. incentivizing older clients to upgrade - the fact
remains that SW creates a more-complex bidding structure by creating a
second economic resource.

*This is clearly a change to a new economic policy* with standard risks
associated with that.  Will that induce an Economic Change Event (see def
last email)?  *Unlikely*, due to slow rollout pace.


5.5.  Problem:  Current SW mining algorithm needs improvement

Current SW block template maker does a reasonable job, but makes some naive
assumptions about the fee market across an entire extended block.  This is
a mismatch with the economic reality (just described).

5.6.   Problem:  New, under-analyzed attack surfaces

Less significant and fundamental but still worth noting.

This is not a fundamental SW problem, but simply standard complexity risk
factors:  splitting the signatures away from transactions, and creating a
new apparently-unsigned version of the transaction opens the possibility of
some network attacks which cause some clients to degrade down from extended
block to core block mode temporarily.

There is a chance of a failure mode that fools older clients into thinking
fraudulent data is valid (judgement: unlikely vis hashpower but not
impossible)

6. Conclusions and recommendations

It seems unlikely that SW provides scaling in the short term, and SW
introduces new economics complexities.

A "short term bump" hard fork block size increase addresses economic and
ecosystem risks that SW does not.

Bump + SW should proceed in parallel, independent tracks, as orthogonal
issues.


7. Appendix - Other SW comments

Hard forks provide much stronger validation, and ensure the network
operates at a fully trustless level.

SW hard fork is preferred, versus soft fork.  Soft forking SW places a huge
amount of trust on miners to validate transaction signatures, versus the
rest of the network, as the network slowly upgrades to newer clients.

An SW hard fork could also add several zero-filled placeholders in a merkle
tree for future use.

[-- Attachment #2: Type: text/html, Size: 11277 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Matt Corallo]

[-- Attachment #1: Type: text/plain, Size: 8208 bytes --]

A large part of your argument is that SW will take longer to deploy than a hard fork, but I completely disagree. Though I do not agree with some people claiming we can deploy SW significantly faster than a hard fork, once the code is ready (probably a six month affair) we can get it deployed very quickly. It's true the ecosystem may take some time to upgrade, but I see that as a feature, not a bug - we can build up some fee pressure with an immediate release valve available for people to use if they want to pay fewer fees.

On the other hand, a hard fork, while simpler for the ecosystem to upgrade to, is a  1-2 year affair (after the code is shipped, so at least 1.5-2.5 from today if we all put off heads down and work). One thing that has concerned me greatly through this whole debate is how quickly people seem to think we can roll out a hard fork. Go look at the distribution of node versions on the network today and work backwards to get nearly every node upgraded... Even with a year between fork-version-release and fork-activation, we'd still kill a bunch of nodes and instead of reducing their security model, lead them to be outright robbed.

On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>1. Summary
>
>Segregated Witness (SegWitness, SW) is being presented in the context
>of
>Scaling Bitcoin.  It has useful attributes, notably addressing a major
>malleability vector, but is not a short term scaling solution.
>
>
>2. Definitions
>
>Import Fee Event, ECE, TFM, FFM from previous email.
>
>Older clients - Any software not upgraded to SW
>
>Newer clients - Upgraded, SW aware software
>
>
>Block size - refers to the core block economic resource limited by
>MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
>Requires a hard fork to change.
>
>Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE. 
>Not
>changed by SW.
>
>
>Extended transaction - Newer, upgraded version of transaction data
>format.
>
>Extended block - Newer, upgraded version of block data format.
>
>
>EBS - Extended block size.  Block size seen by newer clients.
>
>
>3. Context of analysis
>
>One proposal presents SW *in lieu of* a hard fork block size increase.
>This email focuses directly on that.
>
>Useful features outside block size context, such as anti-malleability
>or
>fraud proof features, are not covered in depth.
>
>
>4.1.  Observations on data structure formats and views
>
>SW creates two *views* of each transaction and block.  SW has blocks
>and
>extended blocks.  Similarly, there exists transactions and extended
>transactions.
>
>This view is rendered to clients depending on compatibility level. 
>Newer
>clients see extended blocks and extended transactions.  Older clients
>see
>blocks (limit 1M), and do not see extended blocks.  Older clients see
>upgraded transactions as unsigned, anyone-can-pay transactions.
>
>Each extended transaction exists in two states, one unsigned and one
>signed, each of which passes validation as a valid bitcoin transaction.
>
>
>4.2.  Observations on behavior of older transaction creation
>
>Transactions created by older clients will not use the extended
>transaction
>format.  All data is stored the standard 1M block as today.
>
>
>4.3.  Observations on new block economic model
>
>SW complicates block economics by creating two separate, supply limited
>resources.
>
>The core block economic resource is heavily contended.  Older clients
>use
>core blocks exclusively.  Newer clients use core blocks more
>conservatively, storing as much data as possible in extended blocks.
>
>The extended block economic resource is less heavily contended, though
>that
>of course grows over time as clients upgrade.
>
>Because core blocks are more heavily contended, it is presumed that
>older
>clients will pay a higher fee than newer clients (subject to elasticity
>etc.).
>
>
>5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must be
>considered.
>
>The current apparent proposal is to roll out Segregated Witness as a
>soft
>fork, and keep block size at 1M.
>
>The roll-out pace cannot simply be judged by soft fork speed - which is
>months at best.  Analysis must the layers above:  Updating bitcoin-core
>(JS) and bitcoinj (Java), and then the timelines to roll out those
>updates
>to apps, and then the timeline to update those apps to create extended
>transactions.
>
>Overall, wallet software and programmer libraries must be upgraded to
>make
>use of this new format, adding many more months (12+ in some stacks) to
>the
>roll out timeline.  In the meantime, clients continue to contend
>entirely
>for core block space.
>
>
>5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with
>most
>software, unlike SW.
>
>A simple hard fork such as BIP 102 is automatically compatible with the
>vast range of today's ecosystem software.
>
>SW requires merchants to upgrade almost immediately, requires wallet
>and
>other peripheral software upgrades to make use of.  Other updates are
>opt-in and occur more slowly.  BIP 70 processors need some updates.
>
>The number of LOC that must change for BIP 102 is very small, and the
>problem domain well known, versus SW.
>
>
>5.3.  Problem:   Due to pace, Fee Event not forestalled.
>
>Even presuming SW is merged into Bitcoin Core tomorrow, this does not
>address the risk of a Fee Event and associated Economic Change in the
>coming months.
>
>
>5.4.  Problem:   More complex economic policy, new game theory, new
>bidding
>structure risks.
>
>Splitting blocks into two pieces, each with separate and distinct
>behaviors
>and resource values, creates *two fee markets.*
>
>Having two pricing strata within each block has certainly feasible -
>that
>is the current mining policy of (1) fee/KB followed by (2)
>priority/age.
>
>Valuable or not - e.g. incentivizing older clients to upgrade - the
>fact
>remains that SW creates a more-complex bidding structure by creating a
>second economic resource.
>
>*This is clearly a change to a new economic policy* with standard risks
>associated with that.  Will that induce an Economic Change Event (see
>def
>last email)?  *Unlikely*, due to slow rollout pace.
>
>
>5.5.  Problem:  Current SW mining algorithm needs improvement
>
>Current SW block template maker does a reasonable job, but makes some
>naive
>assumptions about the fee market across an entire extended block.  This
>is
>a mismatch with the economic reality (just described).
>
>5.6.   Problem:  New, under-analyzed attack surfaces
>
>Less significant and fundamental but still worth noting.
>
>This is not a fundamental SW problem, but simply standard complexity
>risk
>factors:  splitting the signatures away from transactions, and creating
>a
>new apparently-unsigned version of the transaction opens the
>possibility of
>some network attacks which cause some clients to degrade down from
>extended
>block to core block mode temporarily.
>
>There is a chance of a failure mode that fools older clients into
>thinking
>fraudulent data is valid (judgement: unlikely vis hashpower but not
>impossible)
>
>6. Conclusions and recommendations
>
>It seems unlikely that SW provides scaling in the short term, and SW
>introduces new economics complexities.
>
>A "short term bump" hard fork block size increase addresses economic
>and
>ecosystem risks that SW does not.
>
>Bump + SW should proceed in parallel, independent tracks, as orthogonal
>issues.
>
>
>7. Appendix - Other SW comments
>
>Hard forks provide much stronger validation, and ensure the network
>operates at a fully trustless level.
>
>SW hard fork is preferred, versus soft fork.  Soft forking SW places a
>huge
>amount of trust on miners to validate transaction signatures, versus
>the
>rest of the network, as the network slowly upgrades to newer clients.
>
>An SW hard fork could also add several zero-filled placeholders in a
>merkle
>tree for future use.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 13290 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jameson Lopp]

[-- Attachment #1: Type: text/plain, Size: 9891 bytes --]

On Wed, Dec 16, 2015 at 12:50 PM, Matt Corallo via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> A large part of your argument is that SW will take longer to deploy than a
> hard fork, but I completely disagree. Though I do not agree with some
> people claiming we can deploy SW significantly faster than a hard fork,
> once the code is ready (probably a six month affair) we can get it deployed
> very quickly. It's true the ecosystem may take some time to upgrade, but I
> see that as a feature, not a bug - we can build up some fee pressure with
> an immediate release valve available for people to use if they want to pay
> fewer fees.
>
> On the other hand, a hard fork, while simpler for the ecosystem to upgrade
> to, is a 1-2 year affair (after the code is shipped, so at least 1.5-2.5
> from today if we all put off heads down and work). One thing that has
> concerned me greatly through this whole debate is how quickly people seem
> to think we can roll out a hard fork. Go look at the distribution of node
> versions on the network today and work backwards to get nearly every node
> upgraded... Even with a year between fork-version-release and
> fork-activation, we'd still kill a bunch of nodes and instead of reducing
> their security model, lead them to be outright robbed.
>
>
Over a year seems to be an extraordinarily long time frame is for deploying
a hard fork. It looks like <https://bitnodes.21.co/dashboard/?days=365> 75%
of reachable nodes have upgraded in the past 6 months while as much as 25%
may not have been upgraded in over a year. However, viewing historical
stats of version upgrades doesn't seem to be an appropriate comparison
because node operators have never been faced with the same incentive to
upgrade. We can point to unintentional forks in the past that have been
resolved fairly quickly by reaching out to miners, but it's also a poor
comparison. Unfortunately, we have no way of knowing what percentage of
nodes are economically important - a great deal of them may be running and
not even be used by the operators.

Perhaps it would be better if we were to formalize the expectations for
full node operators, but it seems to me that node operators have a
responsibility to keep themselves informed and decide when it is
appropriate to update their software. I'm not so sure that it's the rest of
the ecosystem's responsibility to wait around for laggards.

- Jameson

On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>
>> 1. Summary
>>
>> Segregated Witness (SegWitness, SW) is being presented in the context of
>> Scaling Bitcoin.  It has useful attributes, notably addressing a major
>> malleability vector, but is not a short term scaling solution.
>>
>>
>> 2. Definitions
>>
>> Import Fee Event, ECE, TFM, FFM from previous email.
>>
>> Older clients - Any software not upgraded to SW
>>
>> Newer clients - Upgraded, SW aware software
>>
>>
>> Block size - refers to the core block economic resource limit ed by
>> MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
>> Requires a hard fork to change.
>>
>> Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.  Not
>> changed by SW.
>>
>>
>> Extended transaction - Newer, upgraded version of transaction data format.
>>
>> Extended block - Newer, upgraded version of block data format.
>>
>>
>> EBS - Extended block size.  Block size seen by newer clients.
>>
>>
>> 3. Context of analysis
>>
>> One proposal presents SW *in lieu of* a hard fork block size increase.
>> This email focuses directly on that.
>>
>> Useful features outside block size context, such as anti-malleability or
>> fraud proof features, are not covered in depth.
>>
>>
>> 4.1.  Observations on data structure formats and views
>>
>> SW creates two *views* of each transaction and block.  SW has blocks and
>> extended blocks.  Similarly, there exists transactions and extended
>> transactions.
>>
>> This view is rendered to clients depending on compatibility level.  Newer
>> clients see extended blocks and extended transactions.  Older clients see
>> blocks (limit 1M), and do not see extended blocks.  Older clients see
>> upgraded transactions as unsigned, anyone-can-pay transactions.
>>
>> Each extended transaction exists in two states, one unsigned and one
>> signed, each of which passes validation as a valid bitcoin transaction.
>>
>>
>> 4.2.  Observations on behavior of older transaction creation
>>
>> Transactions created by older clients will not use the extended
>> transaction format.  All data is stored the standard 1M block as today.
>>
>>
>> 4.3.  Observations on new block economic model
>>
>> SW complicates block economics by creating two separate, supply limited
>> resources.
>>
>> The core block economic resource is heavily contended.  Older clients use
>> core blocks exclusively.  Newer clients use core block s more
>> conservatively, storing as much data as possible in extended blocks.
>>
>> The extended block economic resource is less heavily contended, though
>> that of course grows over time as clients upgrade.
>>
>> Because core blocks are more heavily contended, it is presumed that older
>> clients will pay a higher fee than newer clients (subject to elasticity
>> etc.).
>>
>>
>> 5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must be
>> considered.
>>
>> The current apparent proposal is to roll out Segregated Witness as a soft
>> fork, and keep block size at 1M.
>>
>> The roll-out pace cannot simply be judged by soft fork speed - which is
>> months at best.  Analysis must the layers above:  Updating bitcoin-core
>> (JS) and bitcoinj (Java), and then the timelines to roll out those updates
>> to apps, and then the timeline to update those apps to create extended
>> transactions.
>>
>> Overall, wallet software and programmer libraries must be upgraded to
>> make use of this new format, adding many more months (12+ in some stacks)
>> to the roll out timeline.  In the meantime, clients continue to contend
>> entirely for core block space.
>>
>>
>> 5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with most
>> software, unlike SW.
>>
>> A simple hard fork such as BIP 102 is automatically compatible with the
>> vast range of today's ecosystem software.
>>
>> SW requires merchants to upgrade almost immediately, requires wallet and
>> other peripheral software upgrades to make use of.  Other updates are
>> opt-in and occur more slowly.  BIP 70 processors need some updates.
>>
>> The number of LOC that must change for BIP 102 is very small, and the
>> problem domain well known, versus SW.
>>
>>
>> 5.3.  Problem:   Due to pace, Fee Event not forestalled.
>>
>> Even presuming SW is merged into Bitcoin Core tomorrow, this does not
>> address the risk of a Fee Event and associated Economic Change in the
>> coming months.
>>
>>
>> 5.4.  Problem:   More complex economic policy, new game theory, new
>> bidding structure risks.
>>
>> Splitting blocks into two pieces, each with separate and distinct
>> behaviors and resource values, creates *two fee markets.*
>>
>> Having two pricing strata within each block has certainly feasible - that
>> is the current mining policy of (1) fee/KB followed by (2) priority/age.
>>
>> Valuable or not - e.g. incentivizing older clients to upgrade - the fact
>> remains that SW creates a more-complex bidding structure by creating a
>> second economic resource.
>>
>> *This is clearly a change to a new economic policy* with standard risks
>> associated with that.  Will that induce an Economic C hange Event (see def
>> last email)?  *Unlikely*, due to slow rollout pace.
>>
>>
>> 5.5.  Problem:  Current SW mining algorithm needs improvement
>>
>> Current SW block template maker does a reasonable job, but makes some
>> naive assumptions about the fee market across an entire extended block.
>> This is a mismatch with the economic reality (just described).
>>
>> 5.6.   Problem:  New, under-analyzed attack surfaces
>>
>> Less significant and fundamental but still worth noting.
>>
>> This is not a fundamental SW problem, but simply standard complexity risk
>> factors:  splitting the signatures away from transactions, and creating a
>> new apparently-unsigned version of the transaction opens t he possibility
>> of some network attacks which cause some clients to degrade down from
>> extended block to core block mode temporarily.
>>
>> There is a chance of a failure mode that fools older clients into
>> thinking fraudulent data is valid (judgement: unlikely vis hashpower but
>> not impossible)
>>
>> 6. Conclusions and recommendations
>>
>> It seems unlikely that SW provides scaling in the short term, and SW
>> introduces new economics complexities.
>>
>> A "short term bump" hard fork block size increase addresses economic and
>> ecosystem risks that SW does not.
>>
>> Bump + SW should proce ed in parallel, independent tracks, as orthogonal
>> issues.
>>
>>
>> 7. Appendix - Other SW comments
>>
>> Hard forks provide much stronger validation, and ensure the network
>> operates at a fully trustless level.
>>
>> SW hard fork is preferred, versus soft fork.  Soft forking SW places a
>> huge amount of trust on miners to validate transaction signatures, versus
>> the rest of the network, as the network slowly upgrades to newer clients.
>>
>> An SW hard fork could also add several zero-filled placeholders in a
>> merkle tree for future use.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 16102 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Matt Corallo]

[-- Attachment #1: Type: text/plain, Size: 11350 bytes --]

We should probably start by defining "economically important". To me, it's pretty clear that every, or at least around 99% of, " economically important" node have upgraded by the time the fork kicks in, with way more than sufficient time given to everyone to upgrade (minding that this is not an emergency situation and that people have lives and many Bitcoin services are hobby projects and upgrading isn't always as easy as just restarting your node). I'd define "economically important" as any node that is used for anything more than simply "being a node" (ie people who started a node to provide resources to the network, and only using their node for that). Note, of course, that we should avoid breaking all such "non-economically important" nodes, but breaking many of them is not a big deal. Note that my proposal includes nodes such as the one doing transaction selection for the relay network. Though it is not used for payments, if it is not upgraded, things will break.

With this definition in mind, I think a year is an aggressive timeline.

On December 16, 2015 1:51:47 PM PST, Jameson Lopp <jameson.lopp@gmail.com> wrote:
>On Wed, Dec 16, 2015 at 12:50 PM, Matt Corallo via bitcoin-dev <
>bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> A large part of your argument is that SW will take longer to deploy
>than a
>> hard fork, but I completely disagree. Though I do not agree with some
>> people claiming we can deploy SW significantly faster than a hard
>fork,
>> once the code is ready (probably a six month affair) we can get it
>deployed
>> very quickly. It's true the ecosystem may take some time to upgrade,
>but I
>> see that as a feature, not a bug - we can build up some fee pressure
>with
>> an immediate release valve available for people to use if they want
>to pay
>> fewer fees.
>>
>> On the other hand, a hard fork, while simpler for the ecosystem to
>upgrade
>> to, is a 1-2 year affair (after the code is shipped, so at least
>1.5-2.5
>> from today if we all put off heads down and work). One thing that has
>> concerned me greatly through this whole debate is how quickly people
>seem
>> to think we can roll out a hard fork. Go look at the distribution of
>node
>> versions on the network today and work backwards to get nearly every
>node
>> upgraded... Even with a year between fork-version-release and
>> fork-activation, we'd still kill a bunch of nodes and instead of
>reducing
>> their security model, lead them to be outright robbed.
>>
>>
>Over a year seems to be an extraordinarily long time frame is for
>deploying
>a hard fork. It looks like <https://bitnodes.21.co/dashboard/?days=365>
>75%
>of reachable nodes have upgraded in the past 6 months while as much as
>25%
>may not have been upgraded in over a year. However, viewing historical
>stats of version upgrades doesn't seem to be an appropriate comparison
>because node operators have never been faced with the same incentive to
>upgrade. We can point to unintentional forks in the past that have been
>resolved fairly quickly by reaching out to miners, but it's also a poor
>comparison. Unfortunately, we have no way of knowing what percentage of
>nodes are economically important - a great deal of them may be running
>and
>not even be used by the operators.
>
>Perhaps it would be better if we were to formalize the expectations for
>full node operators, but it seems to me that node operators have a
>responsibility to keep themselves informed and decide when it is
>appropriate to update their software. I'm not so sure that it's the
>rest of
>the ecosystem's responsibility to wait around for laggards.
>
>- Jameson
>
>On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>>
>>>
>>> 1. Summary
>>>
>>> Segregated Witness (SegWitness, SW) is being presented in the
>context of
>>> Scaling Bitcoin.  It has useful attributes, notably addressing a
>major
>>> malleability vector, but is not a short term scaling solution.
>>>
>>>
>>> 2. Definitions
>>>
>>> Import Fee Event, ECE, TFM, FFM from previous email.
>>>
>>> Older clients - Any software not upgraded to SW
>>>
>>> Newer clients - Upgraded, SW aware software
>>>
>>>
>>> Block size - refers to the core block economic resource limit ed by
>>> MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
>>> Requires a hard fork to change.
>>>
>>> Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.
> Not
>>> changed by SW.
>>>
>>>
>>> Extended transaction - Newer, upgraded version of transaction data
>format.
>>>
>>> Extended block - Newer, upgraded version of block data format.
>>>
>>>
>>> EBS - Extended block size.  Block size seen by newer clients.
>>>
>>>
>>> 3. Context of analysis
>>>
>>> One proposal presents SW *in lieu of* a hard fork block size
>increase.
>>> This email focuses directly on that.
>>>
>>> Useful features outside block size context, such as
>anti-malleability or
>>> fraud proof features, are not covered in depth.
>>>
>>>
>>> 4.1.  Observations on data structure formats and views
>>>
>>> SW creates two *views* of each transaction and block.  SW has blocks
>and
>>> extended blocks.  Similarly, there exists transactions and extended
>>> transactions.
>>>
>>> This view is rendered to clients depending on compatibility level. 
>Newer
>>> clients see extended blocks and extended transactions.  Older
>clients see
>>> blocks (limit 1M), and do not see extended blocks.  Older clients
>see
>>> upgraded transactions as unsigned, anyone-can-pay transactions.
>>>
>>> Each extended transaction exists in two states, one unsigned and one
>>> signed, each of which passes validation as a valid bitcoin
>transaction.
>>>
>>>
>>> 4.2.  Observations on behavior of older transaction creation
>>>
>>> Transactions created by older clients will not use the extended
>>> transaction format.  All data is stored the standard 1M block as
>today.
>>>
>>>
>>> 4.3.  Observations on new block economic model
>>>
>>> SW complicates block economics by creating two separate, supply
>limited
>>> resources.
>>>
>>> The core block economic resource is heavily contended.  Older
>clients use
>>> core blocks exclusively.  Newer clients use core block s more
>>> conservatively, storing as much data as possible in extended blocks.
>>>
>>> The extended block economic resource is less heavily contended,
>though
>>> that of course grows over time as clients upgrade.
>>>
>>> Because core blocks are more heavily contended, it is presumed that
>older
>>> clients will pay a higher fee than newer clients (subject to
>elasticity
>>> etc.).
>>>
>>>
>>> 5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must
>be
>>> considered.
>>>
>>> The current apparent proposal is to roll out Segregated Witness as a
>soft
>>> fork, and keep block size at 1M.
>>>
>>> The roll-out pace cannot simply be judged by soft fork speed - which
>is
>>> months at best.  Analysis must the layers above:  Updating
>bitcoin-core
>>> (JS) and bitcoinj (Java), and then the timelines to roll out those
>updates
>>> to apps, and then the timeline to update those apps to create
>extended
>>> transactions.
>>>
>>> Overall, wallet software and programmer libraries must be upgraded
>to
>>> make use of this new format, adding many more months (12+ in some
>stacks)
>>> to the roll out timeline.  In the meantime, clients continue to
>contend
>>> entirely for core block space.
>>>
>>>
>>> 5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with
>most
>>> software, unlike SW.
>>>
>>> A simple hard fork such as BIP 102 is automatically compatible with
>the
>>> vast range of today's ecosystem software.
>>>
>>> SW requires merchants to upgrade almost immediately, requires wallet
>and
>>> other peripheral software upgrades to make use of.  Other updates
>are
>>> opt-in and occur more slowly.  BIP 70 processors need some updates.
>>>
>>> The number of LOC that must change for BIP 102 is very small, and
>the
>>> problem domain well known, versus SW.
>>>
>>>
>>> 5.3.  Problem:   Due to pace, Fee Event not forestalled.
>>>
>>> Even presuming SW is merged into Bitcoin Core tomorrow, this does
>not
>>> address the risk of a Fee Event and associated Economic Change in
>the
>>> coming months.
>>>
>>>
>>> 5.4.  Problem:   More complex economic policy, new game theory, new
>>> bidding structure risks.
>>>
>>> Splitting blocks into two pieces, each with separate and distinct
>>> behaviors and resource values, creates *two fee markets.*
>>>
>>> Having two pricing strata within each block has certainly feasible -
>that
>>> is the current mining policy of (1) fee/KB followed by (2)
>priority/age.
>>>
>>> Valuable or not - e.g. incentivizing older clients to upgrade - the
>fact
>>> remains that SW creates a more-complex bidding structure by creating
>a
>>> second economic resource.
>>>
>>> *This is clearly a change to a new economic policy* with standard
>risks
>>> associated with that.  Will that induce an Economic C hange Event
>(see def
>>> last email)?  *Unlikely*, due to slow rollout pace.
>>>
>>>
>>> 5.5.  Problem:  Current SW mining algorithm needs improvement
>>>
>>> Current SW block template maker does a reasonable job, but makes
>some
>>> naive assumptions about the fee market across an entire extended
>block.
>>> This is a mismatch with the economic reality (just described).
>>>
>>> 5.6.   Problem:  New, under-analyzed attack surfaces
>>>
>>> Less significant and fundamental but still worth noting.
>>>
>>> This is not a fundamental SW problem, but simply standard complexity
>risk
>>> factors:  splitting the signatures away from transactions, and
>creating a
>>> new apparently-unsigned version of the transaction opens t he
>possibility
>>> of some network attacks which cause some clients to degrade down
>from
>>> extended block to core block mode temporarily.
>>>
>>> There is a chance of a failure mode that fools older clients into
>>> thinking fraudulent data is valid (judgement: unlikely vis hashpower
>but
>>> not impossible)
>>>
>>> 6. Conclusions and recommendations
>>>
>>> It seems unlikely that SW provides scaling in the short term, and SW
>>> introduces new economics complexities.
>>>
>>> A "short term bump" hard fork block size increase addresses economic
>and
>>> ecosystem risks that SW does not.
>>>
>>> Bump + SW should proce ed in parallel, independent tracks, as
>orthogonal
>>> issues.
>>>
>>>
>>> 7. Appendix - Other SW comments
>>>
>>> Hard forks provide much stronger validation, and ensure the network
>>> operates at a fully trustless level.
>>>
>>> SW hard fork is preferred, versus soft fork.  Soft forking SW places
>a
>>> huge amount of trust on miners to validate transaction signatures,
>versus
>>> the rest of the network, as the network slowly upgrades to newer
>clients.
>>>
>>> An SW hard fork could also add several zero-filled placeholders in a
>>> merkle tree for future use.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>>
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>

[-- Attachment #2: Type: text/html, Size: 17715 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Matt Corallo]

[-- Attachment #1: Type: text/plain, Size: 10699 bytes --]

As for "the ecosystem waiting around for laggards", yes, it is absolutely the ecosystems y responsibility to not take actions that will result in people losing money without providing them far more than enough opportunity to fix it. One of the absolute most important features of Bitcoin is that, if you're running a full node, you are provided reasonable security against accepting invalid transactions.

On December 16, 2015 1:51:47 PM PST, Jameson Lopp <jameson.lopp@gmail.com> wrote:
>On Wed, Dec 16, 2015 at 12:50 PM, Matt Corallo via bitcoin-dev <
>bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> A large part of your argument is that SW will take longer to deploy
>than a
>> hard fork, but I completely disagree. Though I do not agree with some
>> people claiming we can deploy SW significantly faster than a hard
>fork,
>> once the code is ready (probably a six month affair) we can get it
>deployed
>> very quickly. It's true the ecosystem may take some time to upgrade,
>but I
>> see that as a feature, not a bug - we can build up some fee pressure
>with
>> an immediate release valve available for people to use if they want
>to pay
>> fewer fees.
>>
>> On the other hand, a hard fork, while simpler for the ecosystem to
>upgrade
>> to, is a 1-2 year affair (after the code is shipped, so at least
>1.5-2.5
>> from today if we all put off heads down and work). One thing that has
>> concerned me greatly through this whole debate is how quickly people
>seem
>> to think we can roll out a hard fork. Go look at the distribution of
>node
>> versions on the network today and work backwards to get nearly every
>node
>> upgraded... Even with a year between fork-version-release and
>> fork-activation, we'd still kill a bunch of nodes and instead of
>reducing
>> their security model, lead them to be outright robbed.
>>
>>
>Over a year seems to be an extraordinarily long time frame is for
>deploying
>a hard fork. It looks like <https://bitnodes.21.co/dashboard/?days=365>
>75%
>of reachable nodes have upgraded in the past 6 months while as much as
>25%
>may not have been upgraded in over a year. However, viewing historical
>stats of version upgrades doesn't seem to be an appropriate comparison
>because node operators have never been faced with the same incentive to
>upgrade. We can point to unintentional forks in the past that have been
>resolved fairly quickly by reaching out to miners, but it's also a poor
>comparison. Unfortunately, we have no way of knowing what percentage of
>nodes are economically important - a great deal of them may be running
>and
>not even be used by the operators.
>
>Perhaps it would be better if we were to formalize the expectations for
>full node operators, but it seems to me that node operators have a
>responsibility to keep themselves informed and decide when it is
>appropriate to update their software. I'm not so sure that it's the
>rest of
>the ecosystem's responsibility to wait around for laggards.
>
>- Jameson
>
>On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>>
>>>
>>> 1. Summary
>>>
>>> Segregated Witness (SegWitness, SW) is being presented in the
>context of
>>> Scaling Bitcoin.  It has useful attributes, notably addressing a
>major
>>> malleability vector, but is not a short term scaling solution.
>>>
>>>
>>> 2. Definitions
>>>
>>> Import Fee Event, ECE, TFM, FFM from previous email.
>>>
>>> Older clients - Any software not upgraded to SW
>>>
>>> Newer clients - Upgraded, SW aware software
>>>
>>>
>>> Block size - refers to the core block economic resource limit ed by
>>> MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
>>> Requires a hard fork to change.
>>>
>>> Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.
> Not
>>> changed by SW.
>>>
>>>
>>> Extended transaction - Newer, upgraded version of transaction data
>format.
>>>
>>> Extended block - Newer, upgraded version of block data format.
>>>
>>>
>>> EBS - Extended block size.  Block size seen by newer clients.
>>>
>>>
>>> 3. Context of analysis
>>>
>>> One proposal presents SW *in lieu of* a hard fork block size
>increase.
>>> This email focuses directly on that.
>>>
>>> Useful features outside block size context, such as
>anti-malleability or
>>> fraud proof features, are not covered in depth.
>>>
>>>
>>> 4.1.  Observations on data structure formats and views
>>>
>>> SW creates two *views* of each transaction and block.  SW has blocks
>and
>>> extended blocks.  Similarly, there exists transactions and extended
>>> transactions.
>>>
>>> This view is rendered to clients depending on compatibility level. 
>Newer
>>> clients see extended blocks and extended transactions.  Older
>clients see
>>> blocks (limit 1M), and do not see extended blocks.  Older clients
>see
>>> upgraded transactions as unsigned, anyone-can-pay transactions.
>>>
>>> Each extended transaction exists in two states, one unsigned and one
>>> signed, each of which passes validation as a valid bitcoin
>transaction.
>>>
>>>
>>> 4.2.  Observations on behavior of older transaction creation
>>>
>>> Transactions created by older clients will not use the extended
>>> transaction format.  All data is stored the standard 1M block as
>today.
>>>
>>>
>>> 4.3.  Observations on new block economic model
>>>
>>> SW complicates block economics by creating two separate, supply
>limited
>>> resources.
>>>
>>> The core block economic resource is heavily contended.  Older
>clients use
>>> core blocks exclusively.  Newer clients use core block s more
>>> conservatively, storing as much data as possible in extended blocks.
>>>
>>> The extended block economic resource is less heavily contended,
>though
>>> that of course grows over time as clients upgrade.
>>>
>>> Because core blocks are more heavily contended, it is presumed that
>older
>>> clients will pay a higher fee than newer clients (subject to
>elasticity
>>> etc.).
>>>
>>>
>>> 5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must
>be
>>> considered.
>>>
>>> The current apparent proposal is to roll out Segregated Witness as a
>soft
>>> fork, and keep block size at 1M.
>>>
>>> The roll-out pace cannot simply be judged by soft fork speed - which
>is
>>> months at best.  Analysis must the layers above:  Updating
>bitcoin-core
>>> (JS) and bitcoinj (Java), and then the timelines to roll out those
>updates
>>> to apps, and then the timeline to update those apps to create
>extended
>>> transactions.
>>>
>>> Overall, wallet software and programmer libraries must be upgraded
>to
>>> make use of this new format, adding many more months (12+ in some
>stacks)
>>> to the roll out timeline.  In the meantime, clients continue to
>contend
>>> entirely for core block space.
>>>
>>>
>>> 5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with
>most
>>> software, unlike SW.
>>>
>>> A simple hard fork such as BIP 102 is automatically compatible with
>the
>>> vast range of today's ecosystem software.
>>>
>>> SW requires merchants to upgrade almost immediately, requires wallet
>and
>>> other peripheral software upgrades to make use of.  Other updates
>are
>>> opt-in and occur more slowly.  BIP 70 processors need some updates.
>>>
>>> The number of LOC that must change for BIP 102 is very small, and
>the
>>> problem domain well known, versus SW.
>>>
>>>
>>> 5.3.  Problem:   Due to pace, Fee Event not forestalled.
>>>
>>> Even presuming SW is merged into Bitcoin Core tomorrow, this does
>not
>>> address the risk of a Fee Event and associated Economic Change in
>the
>>> coming months.
>>>
>>>
>>> 5.4.  Problem:   More complex economic policy, new game theory, new
>>> bidding structure risks.
>>>
>>> Splitting blocks into two pieces, each with separate and distinct
>>> behaviors and resource values, creates *two fee markets.*
>>>
>>> Having two pricing strata within each block has certainly feasible -
>that
>>> is the current mining policy of (1) fee/KB followed by (2)
>priority/age.
>>>
>>> Valuable or not - e.g. incentivizing older clients to upgrade - the
>fact
>>> remains that SW creates a more-complex bidding structure by creating
>a
>>> second economic resource.
>>>
>>> *This is clearly a change to a new economic policy* with standard
>risks
>>> associated with that.  Will that induce an Economic C hange Event
>(see def
>>> last email)?  *Unlikely*, due to slow rollout pace.
>>>
>>>
>>> 5.5.  Problem:  Current SW mining algorithm needs improvement
>>>
>>> Current SW block template maker does a reasonable job, but makes
>some
>>> naive assumptions about the fee market across an entire extended
>block.
>>> This is a mismatch with the economic reality (just described).
>>>
>>> 5.6.   Problem:  New, under-analyzed attack surfaces
>>>
>>> Less significant and fundamental but still worth noting.
>>>
>>> This is not a fundamental SW problem, but simply standard complexity
>risk
>>> factors:  splitting the signatures away from transactions, and
>creating a
>>> new apparently-unsigned version of the transaction opens t he
>possibility
>>> of some network attacks which cause some clients to degrade down
>from
>>> extended block to core block mode temporarily.
>>>
>>> There is a chance of a failure mode that fools older clients into
>>> thinking fraudulent data is valid (judgement: unlikely vis hashpower
>but
>>> not impossible)
>>>
>>> 6. Conclusions and recommendations
>>>
>>> It seems unlikely that SW provides scaling in the short term, and SW
>>> introduces new economics complexities.
>>>
>>> A "short term bump" hard fork block size increase addresses economic
>and
>>> ecosystem risks that SW does not.
>>>
>>> Bump + SW should proce ed in parallel, independent tracks, as
>orthogonal
>>> issues.
>>>
>>>
>>> 7. Appendix - Other SW comments
>>>
>>> Hard forks provide much stronger validation, and ensure the network
>>> operates at a fully trustless level.
>>>
>>> SW hard fork is preferred, versus soft fork.  Soft forking SW places
>a
>>> huge amount of trust on miners to validate transaction signatures,
>versus
>>> the rest of the network, as the network slowly upgrades to newer
>clients.
>>>
>>> An SW hard fork could also add several zero-filled placeholders in a
>>> merkle tree for future use.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>>
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>

[-- Attachment #2: Type: text/html, Size: 17006 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 2401 bytes --]

On Wed, Dec 16, 2015 at 3:50 PM, Matt Corallo <lf-lists@mattcorallo.com>
wrote:

> A large part of your argument is that SW will take longer to deploy than a
> hard fork, but I completely disagree. Though I do not agree with some
> people claiming we can deploy SW significantly faster than a hard fork,
> once the code is ready (probably a six month affair) we can get it deployed
> very quickly. It's true the ecosystem may take some time to upgrade, but I
> see that as a feature, not a bug - we can build up some fee pressure with
> an immediate release valve available for people to use if they want to pay
> fewer fees.
>

That's taking a big risk.  "Build up some fee pressure" is essentially
risking a Fee Event if uptake is slower than planned, or traffic is greater
than expected.



>
> On the other hand, a hard fork, while simpler for the ecosystem to upgrade
> to, is a 1-2 year affair (after the code is shipped, so at least 1.5-2.5
> from today if we all put off heads down and work). One thing that has
> concerned me greatly through this whole debate is how quickly people seem
> to think we can roll out a hard fork. Go look at the distribution of node
> versions on the network today and work backwards to get nearly every node
> upgraded... Even with a year between fork-version-release and
> fork-activation, we'd still kill a bunch of nodes and instead of reducing
> their security model, lead them to be outright robbed.
>

A hard fork will never achieve 100%  There are many credible folks and
estimates who feel a May hard fork is reasonable and doable.

Further, hard forks restore the full trustless nature of the post-hard-fork
nodes.  Soft forks continually erode that.  That's why SW should come via
hard fork.  The end result is more secure - 100% validation of witness
transactions.

If regular hard fork plans are proposed in public, many months in advance,
there is plenty of time for the community to react.  Hard forks create a
more predictable market and environment for Users, and a more secure
network.

Further, even if you believe SW makes hard fork unnecessary, it is the
responsible thing to code and communicate to users the plan for a Fee Event
just in case SW uptake and extension block use does not match theoretical
projections of SW proponents.

Finally, SW does not eliminate and is orthogonal to Short Term Problem #1
(orig. email - drift into ECE)

[-- Attachment #2: Type: text/html, Size: 3037 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 3732 bytes --]

There are no good short-term scaling solutions...this is a very hard problem that necessarily requires a lot of out-of-the-box thinking, something 2015 has seen a LOT of...and I'm optimistic about the ideas presented thus far.

At least SW *is* a scaling solution (albeit most of the important benefits are long term). The issue of fee events has nothing to do with scaling - it has to do with economics...specifically whether we should be subsidizing transactions, who should pay the bill for it, etc. My own personal opinion is that increasing validation costs works against adoption, not for it...even if it artificially keeps fees low - and we'll have to deal with a fee event sooner or later anyhow. You may disagree with my opinion, but please, let's stop confounding the economic issues with actual scaling.

On December 16, 2015 6:21:22 PM PST, Jeff Garzik via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>On Wed, Dec 16, 2015 at 3:50 PM, Matt Corallo
><lf-lists@mattcorallo.com>
>wrote:
>
>> A large part of your argument is that SW will take longer to deploy
>than a
>> hard fork, but I completely disagree. Though I do not agree with some
>> people claiming we can deploy SW significantly faster than a hard
>fork,
>> once the code is ready (probably a six month affair) we can get it
>deployed
>> very quickly. It's true the ecosystem may take some time to upgrade,
>but I
>> see that as a feature, not a bug - we can build up some fee pressure
>with
>> an immediate release valve available for people to use if they want
>to pay
>> fewer fees.
>>
>
>That's taking a big risk.  "Build up some fee pressure" is essentially
>risking a Fee Event if uptake is slower than planned, or traffic is
>greater
>than expected.
>
>
>
>>
>> On the other hand, a hard fork, while simpler for the ecosystem to
>upgrade
>> to, is a 1-2 year affair (after the code is shipped, so at least
>1.5-2.5
>> from today if we all put off heads down and work). One thing that has
>> concerned me greatly through this whole debate is how quickly people
>seem
>> to think we can roll out a hard fork. Go look at the distribution of
>node
>> versions on the network today and work backwards to get nearly every
>node
>> upgraded... Even with a year between fork-version-release and
>> fork-activation, we'd still kill a bunch of nodes and instead of
>reducing
>> their security model, lead them to be outright robbed.
>>
>
>A hard fork will never achieve 100%  There are many credible folks and
>estimates who feel a May hard fork is reasonable and doable.
>
>Further, hard forks restore the full trustless nature of the
>post-hard-fork
>nodes.  Soft forks continually erode that.  That's why SW should come
>via
>hard fork.  The end result is more secure - 100% validation of witness
>transactions.
>
>If regular hard fork plans are proposed in public, many months in
>advance,
>there is plenty of time for the community to react.  Hard forks create
>a
>more predictable market and environment for Users, and a more secure
>network.
>
>Further, even if you believe SW makes hard fork unnecessary, it is the
>responsible thing to code and communicate to users the plan for a Fee
>Event
>just in case SW uptake and extension block use does not match
>theoretical
>projections of SW proponents.
>
>Finally, SW does not eliminate and is orthogonal to Short Term Problem
>#1
>(orig. email - drift into ECE)
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 4657 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

On Wed, Dec 16, 2015 at 9:44 PM, Eric Lombrozo <elombrozo@gmail.com> wrote:

> At least SW *is* a scaling solution (albeit most of the important benefits
> are long term). The issue of fee events has nothing to do with scaling - it
> has to do with economics...specifically whether we should be subsidizing
> transactions, who should pay the bill for it, etc. My own personal opinion
> is that increasing validation costs works against adoption, not for
> it...even if it artificially keeps fees low - and we'll have to deal with a
> fee event sooner or later anyhow. You may disagree with my opinion, but
> please, let's stop confounding the economic issues with actual scaling.
>

At least on my part, the title of the 1st email was "It's economics & ..."
and focused on (a) economics and (b) transition issues.  There was no
confounding.  There was a list of real problems and risks taken when 1M is
not lifted in the short term.

Thus "SW is orthogonal" in these emails, because these problems remain
regardless of SW or no, as the 1st email outlined.

The 2nd email addresses the specific assertion of "no 1M hard fork needed,
because SW."

[-- Attachment #2: Type: text/html, Size: 1579 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Adam Back]

There are a range of opinions about input assumptions by different
people.  In each case, short of misunderstanding, if we have the same
input assumptions we're going to reach the same conclusions.  This is
the way of the world in a meritocracy.  The interesting point is to
compare the input assumptions and try to figure out which are more
realistic, pragmatic and achieve the best outcome.

It might be instructive to re-read Greg's roadmap and others to
re-read Jeff's original post (I will).

There is a proposed roadmap and soft-fork block-size increase and code
that Pieter is working on.  There has been rationale described for
this approach, and it achieves many useful things both short, mid and
long term for scale and other issues.

There seem to be a range of opinions on the fee market, and one
question is when do we deem it safe to aim to be prepared to support a
fee market.

How elastic is block-size demand?  (I think there is evidence of some
elasticity which indicates a partly working fee market already).  What
I mean by elasticity of block-size demand is there are off-chain
transactions and people make an economic choice of whether to go on
chain or not, and the vast majority of transactions, all told, are
off-chain.  Clearly it is ideal if they all go on chain, scale
permitting.

If we look at the roadmap at high-level:

1) bump (seg-wit or ...)
2) network improvements (IBLT/weak-block/other)
3) longer term dynamic block-size (flexcap)
4) write-cache (lightning)

It would probably be good to see some work on preparing for fee
markets.  That has happened somewhat recently in response to the
stress tests.  We do have an observed problem that if there is no
incentive to prepare, the improvements dont happen, and so we can
never be ready for a fee market.  That's kind of how we got here,
people were talking about fee-estimation and dynamic fees several
years ago before the block-size went from 250kB to 750kB, and then
lost interest as there was another 500kB to play with.  There could be
a best practice doc written asking people to prepare.  That might
help.

Presumably it's good if we do see the fee market more, for it to come
in gradually.  Flexcap probably helps there because the block-size
itself becomes elastic to demand (pay for bigger blocks).

If we want to avoid a fee market for the immediate term, are we more
worried about period 1, or period 2 or 3.  Probably 2 is more of a
worry as we're scaling in 1 where in period 2 we're preparing for
scaling and more time has passed for demand to grow.  That might for
example argue for seg-wit because it brings us closer to 4) and if we
spread things out we might delay the possibility to do lightning as
there is only so many cycles for forks (hard or soft) in testing,
deployment planning etc so it can be good to have a holistic view.

Also the question of time-frame that is safe for soft-forks or
hard-forks is another input where views seem to vary.  I think some
people are more optimistic about being able to avoid people losing
money in fast hard-forks.  One lesson on users, is users find failure
modes that testing cant, or do things you would expect them not to do.

Also we're calling hard-forks things that are really soft-forks to SPV
clients, and hard-forks only to full-nodes.  If we wanted to make a
real economic choice, we could artificially make an SPV hard-fork,
however that would make upgrade harder.

As I said in an earlier email I think everyone is empathetic to user
requirements, including economic desires - but Bitcoin has inherent
constraints that are complex to improve.  Each proposal is trying to
best meet those holistic user requirements.  There are no free lunches
and we dont want to economically hurt anyone in total or as a group or
type of use.  Not all requirements can be met, they are in a trade
off, so that calls for balance, planning and transparency.

This is also a market, we can discuss protocol tradeoffs without being
melodramatic - would be kind of undesirable if a dramatic or emotive
way to express something as easily or more clearly expressed in
technical constructive words is moving the price around.

Adam

On 17 December 2015 at 03:58, Jeff Garzik via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> On Wed, Dec 16, 2015 at 9:44 PM, Eric Lombrozo <elombrozo@gmail.com> wrote:
>>
>> At least SW *is* a scaling solution (albeit most of the important benefits
>> are long term). The issue of fee events has nothing to do with scaling - it
>> has to do with economics...specifically whether we should be subsidizing
>> transactions, who should pay the bill for it, etc. My own personal opinion
>> is that increasing validation costs works against adoption, not for
>> it...even if it artificially keeps fees low - and we'll have to deal with a
>> fee event sooner or later anyhow. You may disagree with my opinion, but
>> please, let's stop confounding the economic issues with actual scaling.
>
>
> At least on my part, the title of the 1st email was "It's economics & ..."
> and focused on (a) economics and (b) transition issues.  There was no
> confounding.  There was a list of real problems and risks taken when 1M is
> not lifted in the short term.
>
> Thus "SW is orthogonal" in these emails, because these problems remain
> regardless of SW or no, as the 1st email outlined.
>
> The 2nd email addresses the specific assertion of "no 1M hard fork needed,
> because SW."
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[jl2012]

There are at least 2 proposals on the table:

1. SWSF (segwit soft fork) with 1MB virtual block limit, approximately 
equals to 2MB actual limit

2. BIP102: 2MB actual limit

Since the actual limits for both proposals are approximately the same, 
it is not a determining factor in this discussion

The biggest advantage of SWSF is its softfork nature. However, its 
complexity is not comparable with any previous softforks we had. It is 
reasonable to doubt if it could be ready in 6 months

For BIP102, although it is a hardfork, it is a very simple one and could 
be deployed with ISM in less than a month. It is even simpler than 
BIP34, 66, and 65.

So we have a very complicated softfork vs. a very simple hardfork. The 
only reason makes BIP102 not easy is the fact that it's a hardfork.

The major criticism for a hardfork is requiring everyone to upgrade. Is 
that really a big problem?

First of all, hardfork is not a totally unknown territory. BIP50 was a 
hardfork. The accident happened on 13 March 2013. Bitcoind 0.8.1 was 
released on 18 March, which only gave 2 months of grace period for 
everyone to upgrade. The actual hardfork happened on 16 August. 
Everything completed in 5 months without any panic or chaos. This 
experience strongly suggests that 5 months is already safe for a simple 
hardfork. (in terms of simplicity, I believe BIP102 is even simpler than 
BIP50)

Another experience is from BIP66. The 0.10.0 was released on 16 Feb 
2015, exactly 10 months ago. I analyze the data on 
https://bitnodes.21.co and found that 4600 out of 5090 nodes (90.4%) 
indicate BIP66 support. Considering this is a softfork, I consider this 
as very good adoption already.

With the evidence from BIP50 and BIP66, I believe a 5 months 
pre-announcement is good enough for BIP102. As the vast majority of 
miners have declared their support for a 2MB solution, the legacy 1MB 
fork will certainly be abandoned and no one will get robbed.


My primary proposal:

Now - 15 Jan 2016: formally consult the major miners and merchants if 
they support an one-off rise to 2MB. I consider approximately 80% of 
mining power and 80% of trading volume would be good enough

16 - 31 Jan 2016: release 0.11.3 with BIP102 with ISM vote requiring 80% 
of hashing power

1 Jun 2016: the first day a 2MB block may be allowed

Before 31 Dec 2016: release SWSF



My secondary proposal:

Now: Work on SWSF in a turbo mode and have a deadline of 1 Jun 2016

1 Jun 2016: release SWSF

What if the deadline is not met? Maybe pushing an urgent BIP102 if 
things become really bad.


In any case, I hope a clear decision and road map could be made now. 
This topic has been discussed to death. We are just bringing further 
uncertainty if we keep discussing.


Matt Corallo via bitcoin-dev 於 2015-12-16 15:50 寫到:
> A large part of your argument is that SW will take longer to deploy
> than a hard fork, but I completely disagree. Though I do not agree
> with some people claiming we can deploy SW significantly faster than a
> hard fork, once the code is ready (probably a six month affair) we can
> get it deployed very quickly. It's true the ecosystem may take some
> time to upgrade, but I see that as a feature, not a bug - we can build
> up some fee pressure with an immediate release valve available for
> people to use if they want to pay fewer fees.
> 
>  On the other hand, a hard fork, while simpler for the ecosystem to
> upgrade to, is a 1-2 year affair (after the code is shipped, so at
> least 1.5-2.5 from today if we all put off heads down and work). One
> thing that has concerned me greatly through this whole debate is how
> quickly people seem to think we can roll out a hard fork. Go look at
> the distribution of node versions on the network today and work
> backwards to get nearly every node upgraded... Even with a year
> between fork-version-release and fork-activation, we'd still kill a
> bunch of nodes and instead of reducing their security model, lead them
> to be outright robbed.
> 

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Corey Haddad]

[-- Attachment #1: Type: text/plain, Size: 6936 bytes --]

A planned hardfork, similar to certain softforks, leaves users with some
reduction in security.  It does not leave them defenseless.  Consider the
following:

1: Hard to be robbed on the basis of hashpower.

In reality the old chain will see mining all but stop, and blocks would be
hours to days apart even if a couple percentage points of hashpower failed
to switch over.  Six confirmations would certainly take days.  If the fork
can be scheduled at the beginning of a difficulty period, the old chain
would almost certainly not even ever make it to the next retargeting.

2: Hard to be robber on the basis of awareness.

Expect there to be fairly widespread coverage in the Bitcoin press, and as
the fork draws near, maybe coverage in business and tech publications.
Further, the alert keys will certainly be used, so node operators will get
the message directly.

3: There still needs to be a targeted attack by a fraudster on an unaware
node operator.

To fall victim, one needs to give up something of value to an attacker in
exchange for Bitcoins (on the old chain).  The typical uninitiated
full-node user (probably a small subset anyway) is typically going to be
buying bitcoin from a trusted source, and then saving or spending them, or
perhaps gambling.  They are not, typically, going to be providing a service
or selling goods in exchange for Bitcoin unless they are at least somewhat
aware of what is going on in the Bitcoin space.  It's possible, of course,
but we are talking about small numbers here of people who fit the above.

All three parts of the above would have to go perfectly wrong for someone
to loose out.  Someone somewhere will probably get scammed as a result of a
hardfork.  That stinks, and we should make reasonable efforts to help them
avoid that fate.  But at this point in Bitcoin's development, it is still
in beta, it's still an economic experiment, and we can't allow the software
to become hamstrung out of fear that some inattentive user might bungle
their security.  If they merely waited for 6 confirmations, as is the
standard advice, they would be waiting for days.  If that along doesn't
give them a hint that something is wrong, it might still be too early days
for them to be playing with Bitcoin for anything important.

I support a hardfork deployment that takes 80% of hashpower activate + a
4-month delay.

On Wed, Dec 16, 2015 at 9:32 PM, jl2012 via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> There are at least 2 proposals on the table:
>
> 1. SWSF (segwit soft fork) with 1MB virtual block limit, approximately
> equals to 2MB actual limit
>
> 2. BIP102: 2MB actual limit
>
> Since the actual limits for both proposals are approximately the same, it
> is not a determining factor in this discussion
>
> The biggest advantage of SWSF is its softfork nature. However, its
> complexity is not comparable with any previous softforks we had. It is
> reasonable to doubt if it could be ready in 6 months
>
> For BIP102, although it is a hardfork, it is a very simple one and could
> be deployed with ISM in less than a month. It is even simpler than BIP34,
> 66, and 65.
>
> So we have a very complicated softfork vs. a very simple hardfork. The
> only reason makes BIP102 not easy is the fact that it's a hardfork.
>
> The major criticism for a hardfork is requiring everyone to upgrade. Is
> that really a big problem?
>
> First of all, hardfork is not a totally unknown territory. BIP50 was a
> hardfork. The accident happened on 13 March 2013. Bitcoind 0.8.1 was
> released on 18 March, which only gave 2 months of grace period for everyone
> to upgrade. The actual hardfork happened on 16 August. Everything completed
> in 5 months without any panic or chaos. This experience strongly suggests
> that 5 months is already safe for a simple hardfork. (in terms of
> simplicity, I believe BIP102 is even simpler than BIP50)
>
> Another experience is from BIP66. The 0.10.0 was released on 16 Feb 2015,
> exactly 10 months ago. I analyze the data on https://bitnodes.21.co and
> found that 4600 out of 5090 nodes (90.4%) indicate BIP66 support.
> Considering this is a softfork, I consider this as very good adoption
> already.
>
> With the evidence from BIP50 and BIP66, I believe a 5 months
> pre-announcement is good enough for BIP102. As the vast majority of miners
> have declared their support for a 2MB solution, the legacy 1MB fork will
> certainly be abandoned and no one will get robbed.
>
>
> My primary proposal:
>
> Now - 15 Jan 2016: formally consult the major miners and merchants if they
> support an one-off rise to 2MB. I consider approximately 80% of mining
> power and 80% of trading volume would be good enough
>
> 16 - 31 Jan 2016: release 0.11.3 with BIP102 with ISM vote requiring 80%
> of hashing power
>
> 1 Jun 2016: the first day a 2MB block may be allowed
>
> Before 31 Dec 2016: release SWSF
>
>
>
> My secondary proposal:
>
> Now: Work on SWSF in a turbo mode and have a deadline of 1 Jun 2016
>
> 1 Jun 2016: release SWSF
>
> What if the deadline is not met? Maybe pushing an urgent BIP102 if things
> become really bad.
>
>
> In any case, I hope a clear decision and road map could be made now. This
> topic has been discussed to death. We are just bringing further uncertainty
> if we keep discussing.
>
>
> Matt Corallo via bitcoin-dev 於 2015-12-16 15:50 寫到:
>
>> A large part of your argument is that SW will take longer to deploy
>> than a hard fork, but I completely disagree. Though I do not agree
>> with some people claiming we can deploy SW significantly faster than a
>> hard fork, once the code is ready (probably a six month affair) we can
>> get it deployed very quickly. It's true the ecosystem may take some
>> time to upgrade, but I see that as a feature, not a bug - we can build
>> up some fee pressure with an immediate release valve available for
>> people to use if they want to pay fewer fees.
>>
>>  On the other hand, a hard fork, while simpler for the ecosystem to
>> upgrade to, is a 1-2 year affair (after the code is shipped, so at
>> least 1.5-2.5 from today if we all put off heads down and work). One
>> thing that has concerned me greatly through this whole debate is how
>> quickly people seem to think we can roll out a hard fork. Go look at
>> the distribution of node versions on the network today and work
>> backwards to get nearly every node upgraded... Even with a year
>> between fork-version-release and fork-activation, we'd still kill a
>> bunch of nodes and instead of reducing their security model, lead them
>> to be outright robbed.
>>
>>
> _______________________________________________
>
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 7807 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Although I agree that how safe a pre-hardfork upgrade period is depends on
the complexity of the changes (we should assume everyone may need time to
reimplementat it themselves in their own implementations, not just upgrade
bitcoin core) and bip102 is indeed a very simple hardfork, I think less
than 6 months for a hardfork is starting to push it too much.
For a more complex hardfork (say, a SW hardfork or a collection of many
little fixes) I believe 1 year or more would make more sense.

BIP99 recommends a time threshold (height or median time) + 95% miner
upgrade confirmation with BIP9 (version bits).
So how about the following plan?

1) Deploy BIP102 when its ready + 6 median time months + 95% miner upgrade
confirmation

2) Deploy SW when it's ready + 95% miner upgrade confirmation via bip9.

Note that both "when it's ready" depend on something we are not paying a
lot of attention to: bip9's implementation (just like bip113, bip68-112,
bip99, the coinbase-commitments-cleanup post-SW uncontroversial hardfork,
etc).

Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is already
equivalent to the 2-4-8 "compromise" proposal (which by the way I never
liked, because I don't think anybody should be in a position to
"compromise" anything and because I don't see how "let's avoid an
unavoidable economic change for a little bit longer" arguments can
reasoably claim that "we need to kick the can down the road exactly 3 more
times" or whatever is the reasoning behind it).

[-- Attachment #2: Type: text/html, Size: 1687 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[sickpig]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

On Thu, Dec 17, 2015 at 2:09 PM, Jorge Timón <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
> Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is already
> equivalent to the 2-4-8 "compromise" proposal (which by the way I never
> liked, because I don't think anybody should be in a position to
> "compromise" anything and because I don't see how "let's avoid an
> unavoidable economic change for a little bit longer" arguments can
> reasoably claim that "we need to kick the can down the road exactly 3 more
> times" or whatever is the reasoning behind it).
>

isn't SegWit gain ~75%? hence 2mb x 1.75 = 3.5.

4x is theoric gain you get in case of 2-2 multisig txs.

am I missign something obvious?

[-- Attachment #2: Type: text/html, Size: 1176 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Anthony Towns]

On Thu, Dec 17, 2015 at 04:51:19PM +0100, sickpig--- via bitcoin-dev wrote:
> On Thu, Dec 17, 2015 at 2:09 PM, Jorge Timón wrote:
> > Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is already
> > equivalent to the 2-4-8 "compromise" proposal [...]
> isn't SegWit gain ~75%? hence 2mb x 1.75 = 3.5.

Segwit as proposed gives a 75% *discount* to witness data with the
same limit, so at a 1MB limit, that might give you (eg) 2.05MB made up
of 650kB of base block data plus 1.4MB of witness data; where 650kB +
1.4MB/4 = 1MB at the 1MB limit; or 4.1MB made up of 1.3MB of base plus
2.8MB of witness, for 1.3MB+2.8MB/4 = 2MB at a 2MB limit.

> 4x is theoric gain you get in case of 2-2 multisig txs.

With segregated witness, 2-2 multisig transactions are made up of 94B
of base data, plus about 214B of witness data; discounting the witness
data by 75% gives 94+214/4=148 bytes. That compares to about 301B for
a 2-2 multisig transaction with P2SH rather than segwit, and 301/148
gives about a 2.03x gain, not a 4x gain. A 2.05x gain is what I assumed
to get the numbers above.

You get further improvements with, eg, 3-of-3 multisig, but to get
the full, theoretical 4x gain you'd need a fairly degenerate looking
transaction.

Pay to public key hash with segwit lets you move about half the
transaction data into the witness, giving about a 1.6x improvement by
my count (eg 1.6MB = 800kB of base data plus 800kB of witness data,
where 800kB+800kB/4=1MB), so I think a gain of between 1.6 and 2.0 is
a reasonable expectation to have for the proposed segwit scheme overall.

Cheers,
aj

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[sickpig]

[-- Attachment #1: Type: text/plain, Size: 2122 bytes --]

Anthony,


On Thu, Dec 17, 2015 at 6:55 PM, Anthony Towns via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Thu, Dec 17, 2015 at 04:51:19PM +0100, sickpig--- via bitcoin-dev wrote:
> > On Thu, Dec 17, 2015 at 2:09 PM, Jorge Timón wrote:
> > > Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is already
> > > equivalent to the 2-4-8 "compromise" proposal [...]
> > isn't SegWit gain ~75%? hence 2mb x 1.75 = 3.5.
>
> Segwit as proposed gives a 75% *discount* to witness data with the
> same limit, so at a 1MB limit, that might give you (eg) 2.05MB made up
> of 650kB of base block data plus 1.4MB of witness data; where 650kB +
> 1.4MB/4 = 1MB at the 1MB limit; or 4.1MB made up of 1.3MB of base plus
> 2.8MB of witness, for 1.3MB+2.8MB/4 = 2MB at a 2MB limit.
>
> > 4x is theoric gain you get in case of 2-2 multisig txs.
>
> With segregated witness, 2-2 multisig transactions are made up of 94B
> of base data, plus about 214B of witness data; discounting the witness
> data by 75% gives 94+214/4=148 bytes. That compares to about 301B for
> a 2-2 multisig transaction with P2SH rather than segwit, and 301/148
> gives about a 2.03x gain, not a 4x gain. A 2.05x gain is what I assumed
> to get the numbers above.
>
> You get further improvements with, eg, 3-of-3 multisig, but to get
> the full, theoretical 4x gain you'd need a fairly degenerate looking
> transaction.
>
> Pay to public key hash with segwit lets you move about half the
> transaction data into the witness, giving about a 1.6x improvement by
> my count (eg 1.6MB = 800kB of base data plus 800kB of witness data,
> where 800kB+800kB/4=1MB), so I think a gain of between 1.6 and 2.0 is
> a reasonable expectation to have for the proposed segwit scheme overall.
>
>
many thanks for the explanation.

so it should be fair to say that BIP 102 + SW would bring a gain between
2*1.6 and 2*2.

Just for the sake of simplicity if we take the middle of the interval we
could say
that BIP102 + SW will bring us a max block (virtual) size equal to 1MB * 2
* 1.8 = 3.6

Is it right?

[-- Attachment #2: Type: text/html, Size: 2822 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 2863 bytes --]

Not entirely correct, no. Edge cases also matter. Segwit is described as
4MB because that is the largest possible combined block size that can be
constructed. BIP 102 + segwit would allow a maximum relay of 8MB. So you
have to be confident that an 8MB relay size would be acceptable, even if a
block full of actual transactions would be closer to 3.5MB.

On Fri, Dec 18, 2015 at 6:01 PM, sickpig--- via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Anthony,
>
>
> On Thu, Dec 17, 2015 at 6:55 PM, Anthony Towns via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> On Thu, Dec 17, 2015 at 04:51:19PM +0100, sickpig--- via bitcoin-dev
>> wrote:
>> > On Thu, Dec 17, 2015 at 2:09 PM, Jorge Timón wrote:
>> > > Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is already
>> > > equivalent to the 2-4-8 "compromise" proposal [...]
>> > isn't SegWit gain ~75%? hence 2mb x 1.75 = 3.5.
>>
>> Segwit as proposed gives a 75% *discount* to witness data with the
>> same limit, so at a 1MB limit, that might give you (eg) 2.05MB made up
>> of 650kB of base block data plus 1.4MB of witness data; where 650kB +
>> 1.4MB/4 = 1MB at the 1MB limit; or 4.1MB made up of 1.3MB of base plus
>> 2.8MB of witness, for 1.3MB+2.8MB/4 = 2MB at a 2MB limit.
>>
>> > 4x is theoric gain you get in case of 2-2 multisig txs.
>>
>> With segregated witness, 2-2 multisig transactions are made up of 94B
>> of base data, plus about 214B of witness data; discounting the witness
>> data by 75% gives 94+214/4=148 bytes. That compares to about 301B for
>> a 2-2 multisig transaction with P2SH rather than segwit, and 301/148
>> gives about a 2.03x gain, not a 4x gain. A 2.05x gain is what I assumed
>> to get the numbers above.
>>
>> You get further improvements with, eg, 3-of-3 multisig, but to get
>> the full, theoretical 4x gain you'd need a fairly degenerate looking
>> transaction.
>>
>> Pay to public key hash with segwit lets you move about half the
>> transaction data into the witness, giving about a 1.6x improvement by
>> my count (eg 1.6MB = 800kB of base data plus 800kB of witness data,
>> where 800kB+800kB/4=1MB), so I think a gain of between 1.6 and 2.0 is
>> a reasonable expectation to have for the proposed segwit scheme overall.
>>
>>
> many thanks for the explanation.
>
> so it should be fair to say that BIP 102 + SW would bring a gain between
> 2*1.6 and 2*2.
>
> Just for the sake of simplicity if we take the middle of the interval we
> could say
> that BIP102 + SW will bring us a max block (virtual) size equal to 1MB * 2
> * 1.8 = 3.6
>
> Is it right?
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 3978 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Dave Scotese]

[-- Attachment #1: Type: text/plain, Size: 4935 bytes --]

A couple observations:

   - The consensus block limit is different than the disk space required to
   do validation.  Some participants are worried about one and some about the
   other, and sometimes they feel what amounts to an imaginary contention
   because they perceive these two different things as the same.  They are
   both addressed by scaling solutions, but to different degrees.  This is the
   most concrete I can get about my impression whenever someone writes "not
   correct."  Less concrete is my usual impression, "you're both right."

   - "Kicking the can" has value, but no one has connected the value to the
   phrase, so here it is: The more time we have to make changes, the better
   the changes will be.  Of course it's a trade-off (because we suffer through
   that extra time with the unsolved problem), but using (or thinking of)
   "kicking the can" as bad is a mistake.

   - Whether or not there is a massive campaign targeting *current
   bitcoiners* has a very strong effect on upgrade rates.

It seems that a hardfork to a 2MB limit on 5/5/16 is a tad more than one
LOC, since we want an if-then around it so it doesn't happen til the agreed
date.  But I still support it.

On Fri, Dec 18, 2015 at 11:50 PM, Mark Friedenbach via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Not entirely correct, no. Edge cases also matter. Segwit is described as
> 4MB because that is the largest possible combined block size that can be
> constructed. BIP 102 + segwit would allow a maximum relay of 8MB. So you
> have to be confident that an 8MB relay size would be acceptable, even if a
> block full of actual transactions would be closer to 3.5MB.
>
> On Fri, Dec 18, 2015 at 6:01 PM, sickpig--- via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Anthony,
>>
>>
>> On Thu, Dec 17, 2015 at 6:55 PM, Anthony Towns via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> On Thu, Dec 17, 2015 at 04:51:19PM +0100, sickpig--- via bitcoin-dev
>>> wrote:
>>> > On Thu, Dec 17, 2015 at 2:09 PM, Jorge Timón wrote:
>>> > > Unless I'm missing something, 2 mb x4 = 8mb, so bip102 + SW is
>>> already
>>> > > equivalent to the 2-4-8 "compromise" proposal [...]
>>> > isn't SegWit gain ~75%? hence 2mb x 1.75 = 3.5.
>>>
>>> Segwit as proposed gives a 75% *discount* to witness data with the
>>> same limit, so at a 1MB limit, that might give you (eg) 2.05MB made up
>>> of 650kB of base block data plus 1.4MB of witness data; where 650kB +
>>> 1.4MB/4 = 1MB at the 1MB limit; or 4.1MB made up of 1.3MB of base plus
>>> 2.8MB of witness, for 1.3MB+2.8MB/4 = 2MB at a 2MB limit.
>>>
>>> > 4x is theoric gain you get in case of 2-2 multisig txs.
>>>
>>> With segregated witness, 2-2 multisig transactions are made up of 94B
>>> of base data, plus about 214B of witness data; discounting the witness
>>> data by 75% gives 94+214/4=148 bytes. That compares to about 301B for
>>> a 2-2 multisig transaction with P2SH rather than segwit, and 301/148
>>> gives about a 2.03x gain, not a 4x gain. A 2.05x gain is what I assumed
>>> to get the numbers above.
>>>
>>> You get further improvements with, eg, 3-of-3 multisig, but to get
>>> the full, theoretical 4x gain you'd need a fairly degenerate looking
>>> transaction.
>>>
>>> Pay to public key hash with segwit lets you move about half the
>>> transaction data into the witness, giving about a 1.6x improvement by
>>> my count (eg 1.6MB = 800kB of base data plus 800kB of witness data,
>>> where 800kB+800kB/4=1MB), so I think a gain of between 1.6 and 2.0 is
>>> a reasonable expectation to have for the proposed segwit scheme overall.
>>>
>>>
>> many thanks for the explanation.
>>
>> so it should be fair to say that BIP 102 + SW would bring a gain between
>> 2*1.6 and 2*2.
>>
>> Just for the sake of simplicity if we take the middle of the interval we
>> could say
>> that BIP102 + SW will bring us a max block (virtual) size equal to 1MB *
>> 2 * 1.8 = 3.6
>>
>> Is it right?
>>
>>
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>


-- 
I like to provide some work at no charge to prove my value. Do you need a
techie?
I own Litmocracy <http://www.litmocracy.com> and Meme Racing
<http://www.memeracing.net> (in alpha).
I'm the webmaster for The Voluntaryist <http://www.voluntaryist.com> which
now accepts Bitcoin.
I also code for The Dollar Vigilante <http://dollarvigilante.com/>.
"He ought to find it more profitable to play by the rules" - Satoshi
Nakamoto

[-- Attachment #2: Type: text/html, Size: 6805 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 5252 bytes --]

There are many reasons to support segwit beyond it being a soft-fork. For
example:

* the limitation of non-witness data to no more than 1MB makes the
quadratic scaling costs in large transaction validation no worse than they
currently are;
* redeem scripts in witness use a more accurate cost accounting than
non-witness data (further improvements to this beyond what Pieter has
implemented are possible); and
* segwit provides features (e.g. opt-in malleability protection) which are
required by higher-level scaling solutions.

With that in mind I really don't understand the viewpoint that it would be
better to engage a strictly inferior proposal such as a simple adjustment
of the block size to 2MB.

On Thu, Dec 17, 2015 at 1:32 PM, jl2012 via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> There are at least 2 proposals on the table:
>
> 1. SWSF (segwit soft fork) with 1MB virtual block limit, approximately
> equals to 2MB actual limit
>
> 2. BIP102: 2MB actual limit
>
> Since the actual limits for both proposals are approximately the same, it
> is not a determining factor in this discussion
>
> The biggest advantage of SWSF is its softfork nature. However, its
> complexity is not comparable with any previous softforks we had. It is
> reasonable to doubt if it could be ready in 6 months
>
> For BIP102, although it is a hardfork, it is a very simple one and could
> be deployed with ISM in less than a month. It is even simpler than BIP34,
> 66, and 65.
>
> So we have a very complicated softfork vs. a very simple hardfork. The
> only reason makes BIP102 not easy is the fact that it's a hardfork.
>
> The major criticism for a hardfork is requiring everyone to upgrade. Is
> that really a big problem?
>
> First of all, hardfork is not a totally unknown territory. BIP50 was a
> hardfork. The accident happened on 13 March 2013. Bitcoind 0.8.1 was
> released on 18 March, which only gave 2 months of grace period for everyone
> to upgrade. The actual hardfork happened on 16 August. Everything completed
> in 5 months without any panic or chaos. This experience strongly suggests
> that 5 months is already safe for a simple hardfork. (in terms of
> simplicity, I believe BIP102 is even simpler than BIP50)
>
> Another experience is from BIP66. The 0.10.0 was released on 16 Feb 2015,
> exactly 10 months ago. I analyze the data on https://bitnodes.21.co and
> found that 4600 out of 5090 nodes (90.4%) indicate BIP66 support.
> Considering this is a softfork, I consider this as very good adoption
> already.
>
> With the evidence from BIP50 and BIP66, I believe a 5 months
> pre-announcement is good enough for BIP102. As the vast majority of miners
> have declared their support for a 2MB solution, the legacy 1MB fork will
> certainly be abandoned and no one will get robbed.
>
>
> My primary proposal:
>
> Now - 15 Jan 2016: formally consult the major miners and merchants if they
> support an one-off rise to 2MB. I consider approximately 80% of mining
> power and 80% of trading volume would be good enough
>
> 16 - 31 Jan 2016: release 0.11.3 with BIP102 with ISM vote requiring 80%
> of hashing power
>
> 1 Jun 2016: the first day a 2MB block may be allowed
>
> Before 31 Dec 2016: release SWSF
>
>
>
> My secondary proposal:
>
> Now: Work on SWSF in a turbo mode and have a deadline of 1 Jun 2016
>
> 1 Jun 2016: release SWSF
>
> What if the deadline is not met? Maybe pushing an urgent BIP102 if things
> become really bad.
>
>
> In any case, I hope a clear decision and road map could be made now. This
> topic has been discussed to death. We are just bringing further uncertainty
> if we keep discussing.
>
>
> Matt Corallo via bitcoin-dev 於 2015-12-16 15:50 寫到:
>
>> A large part of your argument is that SW will take longer to deploy
>> than a hard fork, but I completely disagree. Though I do not agree
>> with some people claiming we can deploy SW significantly faster than a
>> hard fork, once the code is ready (probably a six month affair) we can
>> get it deployed very quickly. It's true the ecosystem may take some
>> time to upgrade, but I see that as a feature, not a bug - we can build
>> up some fee pressure with an immediate release valve available for
>> people to use if they want to pay fewer fees.
>>
>>  On the other hand, a hard fork, while simpler for the ecosystem to
>> upgrade to, is a 1-2 year affair (after the code is shipped, so at
>> least 1.5-2.5 from today if we all put off heads down and work). One
>> thing that has concerned me greatly through this whole debate is how
>> quickly people seem to think we can roll out a hard fork. Go look at
>> the distribution of node versions on the network today and work
>> backwards to get nearly every node upgraded... Even with a year
>> between fork-version-release and fork-activation, we'd still kill a
>> bunch of nodes and instead of reducing their security model, lead them
>> to be outright robbed.
>>
>>
> _______________________________________________
>
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 6168 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[jl2012]

I know my reply is a long one but please read before you hit send. I 
have 2 proposals: fast BIP102 + slow SWSF and fast SWSF only. I guess no 
one here is arguing for not doing segwit; and it is on the top of my 
wish list. My main argument (maybe also Jeff's) is that segwit is too 
complicated and may not be a viable short term solution (with the 
reasons I listed that I don't want to repeat)

And also I don't agree with you that BIP102 is *strictly* inferior than 
segwit. We never had a complex softfork like segwit, but we did have a 
successful simple hardfork (BIP50), and BIP102 is very simple. (Details 
in my last post. I'm not going to repeat)

Mark Friedenbach 於 2015-12-17 04:33 寫到:
> There are many reasons to support segwit beyond it being a soft-fork.
> For example:
> 
> * the limitation of non-witness data to no more than 1MB makes the
> quadratic scaling costs in large transaction validation no worse than
> they currently are;
> * redeem scripts in witness use a more accurate cost accounting than
> non-witness data (further improvements to this beyond what Pieter has
> implemented are possible); and
> * segwit provides features (e.g. opt-in malleability protection) which
> are required by higher-level scaling solutions.
> 
> With that in mind I really don't understand the viewpoint that it
> would be better to engage a strictly inferior proposal such as a
> simple adjustment of the block size to 2MB.

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Anthony Towns]

On Thu, Dec 17, 2015 at 12:32:11AM -0500, jl2012 via bitcoin-dev wrote:
> There are at least 2 proposals on the table:
> 1. SWSF (segwit soft fork) with 1MB virtual block limit, approximately
>    equals to 2MB actual limit
> 2. BIP102: 2MB actual limit

I think there's a few variants of (2) -- there's "just 2MB", "2MB now,
4MB in a while, 8MB after that", "1MB for a while longer, then 2MB,
then 4MB" (halved from 2/4/8 since segwit gives 1.6x-2x benefit), and
variations of those with different dates, whether or not to smooth out
the increases to avoid economic shocks, and how to determine activation
(flag day? miner consensus? combination?).

> Since the actual limits for both proposals are approximately the same, it is
> not a determining factor in this discussion

That's true on the benefit side (both give about double the number of
ordinary transactions per block; though segregated witness has other
benefits). On the cost side, the limits are different:

 * worst case block data size is 2x for BIP102, 4x for segwit (affecting
   bandwidth, latency and storage costs for nodes)

 * worst case sigops is 2x for BIP102, but the same as today for segwit
   (affecting block validation time)

 * worst case bytes to hash a block is 4x for BIP102 (doubling block
   size and sigops), but the same as today for segwit (again affecting
   block validation time)

 * worst case UTXO bloat is 2x for BIP102, but the same as today for
   segwit (affecting memory usage, and validation time)

In the "expected" case (where people aren't attacking the blockchain)
I think they're the same on all these metrics. But increasing the
limits could easily make attacks more common, especially if it makes
them more effective.

I think the main attack vector of these is that increasing block
validation time via too many (active) sigops or too many bytes hashed
allows a selfish mining attack, but I'm not clear enough on how that
would work exactly to estimate where the boundary between acceptable and
unacceptable risk is (and how feasible non-consensus-level countermeasures
might be).

But at 1x sigops, you can already (accidently!) construct blocks that
take minutes to verify; and at 4x you can probably already construct a
block that takes 10 minutes to verify, which would probably be pretty
bad... But I'm not sure this isn't already exploitable, so maybe we're
already assuming a certain level of altruism and making things worse
doesn't actually make them worse?

I think it would be good for BIP102 or similar to include an evaluation
of that risk before being deployed [0].

> The major criticism for a hardfork is requiring everyone to upgrade. Is that
> really a big problem?

Yes. That doesn't mean it's not worth it, though.

(The 2-month timeline for the BIP50 accidental hardfork to be accepted
on the network seems persuasive to me that it's possible to roll out a
deliberate, SPV-compatible, hardfork on today's network in 3-6 months)

> My primary proposal:
> 1 Jun 2016: the first day a 2MB block may be allowed
> 
> My secondary proposal:
> 1 Jun 2016: release SWSF

I think it makes sense to just do both of these independently; ie:

 * release segwit via softfork ASAP (perhaps targetting March or April
   to get it included in bitcoin, activation a month or three
   afterwards?), with virtual block size calculated as proposed and
   capped by MAX_BLOCK_SIZE [1]

 * increase MAX_BLOCK_SIZE via hardfork to 2MB after block 420,000
   (phased in gradually? with future scheduled increases to 4MB or 8MB?)

If segwit gets delayed because it's complicated, that's okay; if
it comes out earlier, that's okay too. If the hardfork gets delayed
because miners aren't ready or because it's better to introduce it in a
staggered fashion, or because there's no clear evaluation of the risks,
that's okay too.

But more importantly, it allows evaluate the pros and cons of each
implementation separately and on its own merits, rather than arguing
against working on one just because you're in favour of doing the
other ASAP.


They have benefits if you combine them too; for instance, if the
MAX_BLOCK_SIZE increase is phased in rather than done as a step increase
(ie block x's limit is 1MB, block x+1's limit is 1.005MB or similar,
and block x+2's limit is 1.01MB, etc) having segwit available in parallel
could provide a helpful escape valve: if an individual bitcoin user has
been dying for more capacity, they can spend the time/effort to update
their software for segwit and get it immediately without having to wait
as the consensus limits rise.

Conversely, having both segwit and a phased increase to MAX_BLOCK_SIZE
means that miners generally won't be immediately mining 2MB (or 4MB)
blocks halfway through the year, which should avoid both technological
shocks (bandwidth just doubled!) or economic shocks (supply has increased
so fees have plummeted), which could be good.


FWIW, the worst case scenarios are:

 * block data size:
     BIP102:  2x   (worst/avg)
     segwit:  4x   (worst, ~2x avg)
     both:    8x   (worst, ~4x avg)
     BIP101:  8x   (worst/avg)

 * sigops per block:
     BIP102:  2x
     segwit:  1x
     both:    2x
     BIP101:  8x

 * bytes hashed per block:
     BIP102:  4x
     segwit:  1x
     both:    4x
     BIP101:  64x

 * UTXO rate of increase:
     BIP102:  2x
     segwit:  1x
     both:    2x
     BIP101:  8x

Compared to the (expected, eventual, near-term) benefits:

 * transactions per block:
     BIP102:  2x
     segwit:  1.6x-2x
     both:    3.2x-4x
     BIP101:  8x

 * misc:
     BIP101/2: planned hardforks are possible, bitcoin community governance
       is demonstrably working, etc
     segwit: malleability fixes, script improvements, lightning
       enablement, etc

The block data is the only case where the average case is already just
about the worst case; for the others, as long as the worst case doesn't
inspire new attacks, the future average case should just increase in
proportion to the additional transactions.

Cheers,
aj

[0] (and segwit should actually account for sigops in witness data before
     being deployed...)

[1] If segwit warrants a hardfork to clean up data structures, I think
    that should be deferred until well after the MAX_BLOCK_SIZE hardfork,
    rather than trying to do it at the same time. As such, doing segwit by
    soft fork in the short term seems to make sense, since it also helps
    with transaction malleability and further improvements to script.

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Marcel Jamin]

[-- Attachment #1: Type: text/plain, Size: 9696 bytes --]

Maybe we should first gather concrete estimates about and roughly agree on

- how long SW (SF) development will probably take

- how long the ecosystem needs to prepare for a hardfork (SW (HF) or a
simple can kicking block size increase)

Opinions differ wildly from what it looks like, but maybe we can get to
estimates that the majority here can accept.

---

Personally, I think that the disclaimer "Bitcoin is an experiment" is
pervasive. It's still a pre-release, even with a $6bn vote of confidence.
If you don't follow developments in this phase, don't upgrade and then have
an elevated risk of losing money by getting scammed, then that's a little
bit your fault. I'd absolutely support a change in mentality on that once
1.0.0 arrives, but until then is bitcoin a work-in-progress experiment and
a high risk investment.

A planned hard-fork is an experiment that needs to be run anyway. When do
we want to do that, if not now?


2015-12-16 21:50 GMT+01:00 Matt Corallo via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org>:

> A large part of your argument is that SW will take longer to deploy than a
> hard fork, but I completely disagree. Though I do not agree with some
> people claiming we can deploy SW significantly faster than a hard fork,
> once the code is ready (probably a six month affair) we can get it deployed
> very quickly. It's true the ecosystem may take some time to upgrade, but I
> see that as a feature, not a bug - we can build up some fee pressure with
> an immediate release valve available for people to use if they want to pay
> fewer fees.
>
> On the other hand, a hard fork, while simpler for the ecosystem to upgrade
> to, is a 1-2 year affair (after the code is shipped, so at least 1.5-2.5
> from today if we all put off heads down and work). One thing that has
> concerned me greatly through this whole debate is how quickly people seem
> to think we can roll out a hard fork. Go look at the distribution of node
> versions on the network today and work backwards to get nearly every node
> upgraded... Even with a year between fork-version-release and
> fork-activation, we'd still kill a bunch of nodes and instead of reducing
> their security model, lead them to be outright robbed.
>
> On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>
>> 1. Summary
>>
>> Segregated Witness (SegWitness, SW) is being presented in the context of
>> Scaling Bitcoin.  It has useful attributes, notably addressing a major
>> malleability vector, but is not a short term scaling solution.
>>
>>
>> 2. Definitions
>>
>> Import Fee Event, ECE, TFM, FFM from previous email.
>>
>> Older clients - Any software not upgraded to SW
>>
>> Newer clients - Upgraded, SW aware software
>>
>>
>> Block size - refers to the core block economic resource limit ed by
>> MAX_BLOCK_SIZE.  Witness data (or extension block data) is excluded.
>> Requires a hard fork to change.
>>
>> Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.  Not
>> changed by SW.
>>
>>
>> Extended transaction - Newer, upgraded version of transaction data format.
>>
>> Extended block - Newer, upgraded version of block data format.
>>
>>
>> EBS - Extended block size.  Block size seen by newer clients.
>>
>>
>> 3. Context of analysis
>>
>> One proposal presents SW *in lieu of* a hard fork block size increase.
>> This email focuses directly on that.
>>
>> Useful features outside block size context, such as anti-malleability or
>> fraud proof features, are not covered in depth.
>>
>>
>> 4.1.  Observations on data structure formats and views
>>
>> SW creates two *views* of each transaction and block.  SW has blocks and
>> extended blocks.  Similarly, there exists transactions and extended
>> transactions.
>>
>> This view is rendered to clients depending on compatibility level.  Newer
>> clients see extended blocks and extended transactions.  Older clients see
>> blocks (limit 1M), and do not see extended blocks.  Older clients see
>> upgraded transactions as unsigned, anyone-can-pay transactions.
>>
>> Each extended transaction exists in two states, one unsigned and one
>> signed, each of which passes validation as a valid bitcoin transaction.
>>
>>
>> 4.2.  Observations on behavior of older transaction creation
>>
>> Transactions created by older clients will not use the extended
>> transaction format.  All data is stored the standard 1M block as today.
>>
>>
>> 4.3.  Observations on new block economic model
>>
>> SW complicates block economics by creating two separate, supply limited
>> resources.
>>
>> The core block economic resource is heavily contended.  Older clients use
>> core blocks exclusively.  Newer clients use core block s more
>> conservatively, storing as much data as possible in extended blocks.
>>
>> The extended block economic resource is less heavily contended, though
>> that of course grows over time as clients upgrade.
>>
>> Because core blocks are more heavily contended, it is presumed that older
>> clients will pay a higher fee than newer clients (subject to elasticity
>> etc.).
>>
>>
>> 5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must be
>> considered.
>>
>> The current apparent proposal is to roll out Segregated Witness as a soft
>> fork, and keep block size at 1M.
>>
>> The roll-out pace cannot simply be judged by soft fork speed - which is
>> months at best.  Analysis must the layers above:  Updating bitcoin-core
>> (JS) and bitcoinj (Java), and then the timelines to roll out those updates
>> to apps, and then the timeline to update those apps to create extended
>> transactions.
>>
>> Overall, wallet software and programmer libraries must be upgraded to
>> make use of this new format, adding many more months (12+ in some stacks)
>> to the roll out timeline.  In the meantime, clients continue to contend
>> entirely for core block space.
>>
>>
>> 5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with most
>> software, unlike SW.
>>
>> A simple hard fork such as BIP 102 is automatically compatible with the
>> vast range of today's ecosystem software.
>>
>> SW requires merchants to upgrade almost immediately, requires wallet and
>> other peripheral software upgrades to make use of.  Other updates are
>> opt-in and occur more slowly.  BIP 70 processors need some updates.
>>
>> The number of LOC that must change for BIP 102 is very small, and the
>> problem domain well known, versus SW.
>>
>>
>> 5.3.  Problem:   Due to pace, Fee Event not forestalled.
>>
>> Even presuming SW is merged into Bitcoin Core tomorrow, this does not
>> address the risk of a Fee Event and associated Economic Change in the
>> coming months.
>>
>>
>> 5.4.  Problem:   More complex economic policy, new game theory, new
>> bidding structure risks.
>>
>> Splitting blocks into two pieces, each with separate and distinct
>> behaviors and resource values, creates *two fee markets.*
>>
>> Having two pricing strata within each block has certainly feasible - that
>> is the current mining policy of (1) fee/KB followed by (2) priority/age.
>>
>> Valuable or not - e.g. incentivizing older clients to upgrade - the fact
>> remains that SW creates a more-complex bidding structure by creating a
>> second economic resource.
>>
>> *This is clearly a change to a new economic policy* with standard risks
>> associated with that.  Will that induce an Economic C hange Event (see def
>> last email)?  *Unlikely*, due to slow rollout pace.
>>
>>
>> 5.5.  Problem:  Current SW mining algorithm needs improvement
>>
>> Current SW block template maker does a reasonable job, but makes some
>> naive assumptions about the fee market across an entire extended block.
>> This is a mismatch with the economic reality (just described).
>>
>> 5.6.   Problem:  New, under-analyzed attack surfaces
>>
>> Less significant and fundamental but still worth noting.
>>
>> This is not a fundamental SW problem, but simply standard complexity risk
>> factors:  splitting the signatures away from transactions, and creating a
>> new apparently-unsigned version of the transaction opens t he possibility
>> of some network attacks which cause some clients to degrade down from
>> extended block to core block mode temporarily.
>>
>> There is a chance of a failure mode that fools older clients into
>> thinking fraudulent data is valid (judgement: unlikely vis hashpower but
>> not impossible)
>>
>> 6. Conclusions and recommendations
>>
>> It seems unlikely that SW provides scaling in the short term, and SW
>> introduces new economics complexities.
>>
>> A "short term bump" hard fork block size increase addresses economic and
>> ecosystem risks that SW does not.
>>
>> Bump + SW should proce ed in parallel, independent tracks, as orthogonal
>> issues.
>>
>>
>> 7. Appendix - Other SW comments
>>
>> Hard forks provide much stronger validation, and ensure the network
>> operates at a fully trustless level.
>>
>> SW hard fork is preferred, versus soft fork.  Soft forking SW places a
>> huge amount of trust on miners to validate transaction signatures, versus
>> the rest of the network, as the network slowly upgrades to newer clients.
>>
>> An SW hard fork could also add several zero-filled placeholders in a
>> merkle tree for future use.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 15168 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Pieter Wuille]

On Wed, Dec 16, 2015 at 9:38 PM, Jeff Garzik via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> 4.3.  Observations on new block economic model
>
> SW complicates block economics by creating two separate, supply limited
> resources.

Not correct. I propose defining the virtual_block_size as base_size +
witness_size * 0.25, and limiting virtual_block_size to 1M. This
creates a single variable to optimize for. If accepted, miners are
incentived to maximize fee per virtual_block_size instead of per size.

Wallet software can individually choose whether to upgrade or not.
Once they upgrade, they get to perform 1.75x as many transactions for
the same fee (assuming non-complex transactions), and this is
independent of whether anyone else upgrades.

> 5.1.  Problem:  Pace of roll-out will be slow - Whole Ecosystem must be
> considered.
>
> The current apparent proposal is to roll out Segregated Witness as a soft
> fork, and keep block size at 1M.
>
> The roll-out pace cannot simply be judged by soft fork speed - which is
> months at best.  Analysis must the layers above:  Updating bitcoin-core (JS)
> and bitcoinj (Java), and then the timelines to roll out those updates to
> apps, and then the timeline to update those apps to create extended
> transactions.

Agree, however everyone can upgrade whenever they want, and get the
reduced fees as soon as they do. This is contrary to a hard fork,
which forces every full node to upgrade at once (though indeed, light
clients are not necessarily forced to upgrade).

> 5.2.  Problem:   Hard fork to bigger block size Just Works(tm) with most
> software, unlike SW.
>
> A simple hard fork such as BIP 102 is automatically compatible with the vast
> range of today's ecosystem software.
>
> SW requires merchants to upgrade almost immediately, requires wallet and
> other peripheral software upgrades to make use of.  Other updates are opt-in
> and occur more slowly.  BIP 70 processors need some updates.
>
> The number of LOC that must change for BIP 102 is very small, and the
> problem domain well known, versus SW.

It multiplies all current DoS vectors by a factor equal to the
capacity increase factor. SW increases capacity while leaving several
worst-case effects constant.

> 5.4.  Problem:   More complex economic policy, new game theory, new bidding
> structure risks.
>
> Splitting blocks into two pieces, each with separate and distinct behaviors
> and resource values, creates two fee markets.

I believe you have misunderstood the proposal in that case.

> 5.5.  Problem:  Current SW mining algorithm needs improvement
>
> Current SW block template maker does a reasonable job, but makes some naive
> assumptions about the fee market across an entire extended block.  This is a
> mismatch with the economic reality (just described).

Again, I think you misunderstood. The proposal includes a single new
formula for block size, and optimizes for it. In case the proposal is
accepted, the mining code is automatically as optimal as it was
before.

> 6. Conclusions and recommendations
>
> A "short term bump" hard fork block size increase addresses economic and
> ecosystem risks that SW does not.
>
> Bump + SW should proceed in parallel, independent tracks, as orthogonal
> issues.

I agree here. SW is not a replacement for a scale increase. However, I
think it can be adopted much more easily, as it doesn't require the
massively pervasive consensus that a hardfork requires to perform
safely.

> 7. Appendix - Other SW comments
>
> Hard forks provide much stronger validation, and ensure the network operates
> at a fully trustless level.
>
> SW hard fork is preferred, versus soft fork.  Soft forking SW places a huge
> amount of trust on miners to validate transaction signatures, versus the
> rest of the network, as the network slowly upgrades to newer clients.

But old clients may not care about the new rules, and they still
validate the old ones they chose to enforce.

Furthermore, soft forks cannot be prevented: miners can always choose
to enforce stronger rules than the network demands from them.

-- 
Pieter

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1297 bytes --]

On Wed, Dec 16, 2015 at 3:59 PM, Pieter Wuille <pieter.wuille@gmail.com>
wrote:

> On Wed, Dec 16, 2015 at 9:38 PM, Jeff Garzik via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
> > 4.3.  Observations on new block economic model
> >
> > SW complicates block economics by creating two separate, supply limited
> > resources.
>
> Not correct. I propose defining the virtual_block_size as base_size +
> witness_size * 0.25, and limiting virtual_block_size to 1M. This
> creates a single variable to optimize for. If accepted, miners are
> incentived to maximize fee per virtual_block_size instead of per size.
>

It is correct.  There are two separate sets of economic actors and levels
of contention for each set of space.

That is true regardless of the proposed miner selection algorithm.



> > 5.4.  Problem:   More complex economic policy, new game theory, new
> bidding
> > structure risks.
> >
> > Splitting blocks into two pieces, each with separate and distinct
> behaviors
> > and resource values, creates two fee markets.
>
> I believe you have misunderstood the proposal in that case.
>

See above.  There are two separate and distinct resource velocities and
demand levels in reality.  That creates two markets regardless of miner
selection algorithm in the block maker.

[-- Attachment #2: Type: text/html, Size: 2060 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Pieter Wuille]

On Wed, Dec 16, 2015 at 10:27 PM, Jeff Garzik <jgarzik@gmail.com> wrote:
>> Not correct. I propose defining the virtual_block_size as base_size +
>> witness_size * 0.25, and limiting virtual_block_size to 1M. This
>> creates a single variable to optimize for. If accepted, miners are
>> incentived to maximize fee per virtual_block_size instead of per size.
>
>
> It is correct.  There are two separate sets of economic actors and levels of
> contention for each set of space.
>
> That is true regardless of the proposed miner selection algorithm.

Maybe I haven't explained this properly, so consider this example:

A miner receives sets of 200 byte transactions with all identical
fees. Non-witness ones (whose virtual size is thus 200 bytes) and a
witness one (where 120 of the 200 bytes are witness data, so its
virtual size is 80 + 120*0.25 = 110 bytes).

The consensus rules would limit 1) the base size to 1000000 bytes 2)
the virtual size to 1000000 bytes. However, as the virtual size is
defined as the sum of the base size plus a non-negative number,
satisfying (2) always implies satisfying (1).

Thus, the miners' best strategy is to accept the witness transactions,
as it allows 1000000/110=9090 transactions rather than
1000000/200=5000.

In fact, the optimal fee maximizing strategy is always to maximize fee
per virtual size.

-- 
Pieter

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

Maybe a new analogy helps.

SW presents a blended price and blended basket of two goods.  You can
interact with the Service through the blended price, but that does not
erase the fact that the basket contains two separate from similar resources.

A different set of economic actors uses one resource, and/or both.  There
are explicit incentives to shift actors from solely using one resource to
using both.

The fact that separate sets of economic actors and incentives exist is
sufficient to prove it is indeed a basket of goods, not a single good.


On Wed, Dec 16, 2015 at 4:36 PM, Pieter Wuille <pieter.wuille@gmail.com>
wrote:

> Thus, the miners' best strategy is to accept the witness transactions,
> as it allows 1000000/110=9090 transactions rather than
> 1000000/200=5000.
>

Under your blended algorithm, this seems reasonable as a first pass.



> In fact, the optimal fee maximizing strategy is always to maximize fee
> per virtual size.
>

This is a microscopic, not macroscopic analysis.  Externalities and long
term incentives can severely perturb or invalidate that line of thinking.

Typical counter-example:  Many miners are perfectly happy with very low
fees to encourage long term growth of their bitcoin income through network
effect growth -- rendering fee micro-optimizations largely in the realm of
DoS prevention rather than miner incentive.

[-- Attachment #2: Type: text/html, Size: 2175 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

On Wed, Dec 16, 2015 at 5:09 PM, Jeff Garzik <jgarzik@gmail.com> wrote:

> SW presents a blended price and blended basket of two goods.  You can
> interact with the Service through the blended price, but that does not
> erase the fact that the basket contains two separate from similar resources.
>

separate-but-similar

[-- Attachment #2: Type: text/html, Size: 683 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

On Wed, Dec 16, 2015 at 5:09 PM, Jeff Garzik <jgarzik@gmail.com> wrote:

> SW presents a blended price and blended basket of two goods.  You can
> interact with the Service through the blended price, but that does not
> erase the fact that the basket contains two separate from similar resources.
>
> A different set of economic actors uses one resource, and/or both.  There
> are explicit incentives to shift actors from solely using one resource to
> using both.
>

Illustration:  If SW is deployed via soft fork, the count of nodes that
validate witness data is significantly lower than the count of nodes that
validate non-witness data.  Soft forks are not trustless operation, they
depend on miner trust, slowly eroding the trustless validation of older
nodes over time.

Higher security in one data area versus another produces another economic
value distinction between the two goods in the basket, and creates a "pay
more for higher security in core block, pay less for lower security in
witness" dynamic.

This economic distinction is not present if SW is deployed via hard fork.

[-- Attachment #2: Type: text/html, Size: 1776 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[jl2012]

This is not correct.

As only about 1/3 of nodes support BIP65 now, would you consider CLTV tx 
are less secure than others? I don't think so. Since one invalid CLTV tx 
will make the whole block invalid. Having more nodes to fully validate 
non-CLTV txs won't make them any safer. The same logic also applies to 
SW softfork.

You may argue that a softfork would make the network as a whole less 
secure, as old nodes have to trust new nodes. However, the security of 
all content in the same block must be the same, by definition.

Anyway, I support SW softfork at the beginning, and eventually (~2 
years) moving to a hardfork with higher block size limit and better 
commitment structure.

Jeff Garzik via bitcoin-dev 於 2015-12-17 13:27 寫到:

> 
> Illustration:  If SW is deployed via soft fork, the count of nodes
> that validate witness data is significantly lower than the count of
> nodes that validate non-witness data.  Soft forks are not trustless
> operation, they depend on miner trust, slowly eroding the trustless
> validation of older nodes over time.
> 
> Higher security in one data area versus another produces another
> economic value distinction between the two goods in the basket, and
> creates a "pay more for higher security in core block, pay less for
> lower security in witness" dynamic.
> 
> This economic distinction is not present if SW is deployed via hard
> fork.
> 
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

On Thu, Dec 17, 2015 at 1:46 PM, jl2012 <jl2012@xbt.hk> wrote:

> This is not correct.
>
> As only about 1/3 of nodes support BIP65 now, would you consider CLTV tx
> are less secure than others? I don't think so. Since one invalid CLTV tx
> will make the whole block invalid. Having more nodes to fully validate
> non-CLTV txs won't make them any safer. The same logic also applies to SW
> softfork.
>


Yes - the logic applies to all soft forks.  Each soft fork degrades the
security of non-upgraded nodes.

The core design of bitcoin is that trustless nodes validate the work of
miners, not trust them.

Soft forks move in the opposite direction.  Each new soft-forked feature
leans very heavily on miner trust rather than P2P network validation.

[-- Attachment #2: Type: text/html, Size: 1268 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 1378 bytes --]

Doesn't a good soft fork signaling mechanism along with an activation warning system for non-upgraded nodes (i.e. BIP9, or even block version ISM for that matter) essentially fix this? I don't quite get why this should be an issue.

On December 17, 2015 10:52:39 AM PST, Jeff Garzik via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>On Thu, Dec 17, 2015 at 1:46 PM, jl2012 <jl2012@xbt.hk> wrote:
>
>> This is not correct.
>>
>> As only about 1/3 of nodes support BIP65 now, would you consider CLTV
>tx
>> are less secure than others? I don't think so. Since one invalid CLTV
>tx
>> will make the whole block invalid. Having more nodes to fully
>validate
>> non-CLTV txs won't make them any safer. The same logic also applies
>to SW
>> softfork.
>>
>
>
>Yes - the logic applies to all soft forks.  Each soft fork degrades the
>security of non-upgraded nodes.
>
>The core design of bitcoin is that trustless nodes validate the work of
>miners, not trust them.
>
>Soft forks move in the opposite direction.  Each new soft-forked
>feature
>leans very heavily on miner trust rather than P2P network validation.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 2205 bytes --]

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Adam Back]

While it is interesting to contemplate moving to a world with
hard-fork only upgrades (deprecate soft-forks), now is possibly not
the time to consider that.  Someone can take that topic and make a
what-if sketch for how it could work and put it on the wishlist wiki
if its not already there.

We want to be pragmatic and constructive to reach consensus and that
takes not mixing in what-ifs or orthogonal long standing problems into
the mix, as needing to be fixed now.

Adam


On 17 December 2015 at 19:52, Jeff Garzik via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>
> On Thu, Dec 17, 2015 at 1:46 PM, jl2012 <jl2012@xbt.hk> wrote:
>>
>> This is not correct.
>>
>> As only about 1/3 of nodes support BIP65 now, would you consider CLTV tx
>> are less secure than others? I don't think so. Since one invalid CLTV tx
>> will make the whole block invalid. Having more nodes to fully validate
>> non-CLTV txs won't make them any safer. The same logic also applies to SW
>> softfork.
>
>
>
> Yes - the logic applies to all soft forks.  Each soft fork degrades the
> security of non-upgraded nodes.
>
> The core design of bitcoin is that trustless nodes validate the work of
> miners, not trust them.
>
> Soft forks move in the opposite direction.  Each new soft-forked feature
> leans very heavily on miner trust rather than P2P network validation.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] Segregated Witness in the context of Scaling Bitcoin

[Anthony Towns]

On Wed, Dec 16, 2015 at 10:36:09PM +0100, Pieter Wuille via bitcoin-dev wrote:
> On Wed, Dec 16, 2015 at 10:27 PM, Jeff Garzik <jgarzik@gmail.com> wrote:
> >> Not correct. I propose defining the virtual_block_size as base_size +
> >> witness_size * 0.25, and limiting virtual_block_size to 1M. This
> >> creates a single variable to optimize for. If accepted, miners are
> >> incentived to maximize fee per virtual_block_size instead of per size.
> > It is correct.  There are two separate sets of economic actors and levels of
> > contention for each set of space.
> > That is true regardless of the proposed miner selection algorithm.

You're right that the miner selection algorithm doesn't force it to be
the way Pieter describe. But your claim is still incorrect. :)

> Maybe I haven't explained this properly, so consider this example:

Alternatively:

With Pieter's segwit proposal (as it stands), there are two
consensus-limited resources: number of signature ops in the base block
must be no more than 20k, and the virtual block size must be no more
than 1MB (where virtual block size = base block size plus a quarter of
the witness data size).

Nodes and miners have other constraints -- bandwidth, storage, CPU, etc,
such that they might not want to max out these limits for whatever reason,
but those limits aren't enforced by consensus, so can be adjusted as
technology improves just by individual miner policy.

> In fact, the optimal fee maximizing strategy is always to maximize fee
> per virtual size.

(modulo sigop constraints, same as today for fee per base block size)

That's on the "supply" side (ie, miners are forced to be a single group
of economic actors with alighned constraints due to consensus rules).

On the demand side, there might be people who are able to trade off
witness data for base data at different ratios. For most, it's just 1:1
up to a limit as they move scriptsig to witness data, and obviously if
you have to trade 1B of base data for more than 4B of witness data it's
uneconomic. But since the ratio is fixed, there's no bartering to be
done, it's just the same simple calculation for everyone -- does 1B of
base convert to <4B of witness? then do it; otherwise, don't. But once
they've selected a tradeoff, all they can do is choose an absolute fee
value for their transaction, and then you're just back to having some
people who are willing to pay higher fees per virtual block size, and
others who are only willing to pay lower fees.

Cheers,
aj