[bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Sancho Panza]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=UTF-8, Size: 2625 bytes --]

¡Hola!

Please find below a proposal [resubmission] for a new informational BIP
provisionally named 'bip-genvbvoting'.

I present it here in rough draft for your esteemed consideration and as
a basis for discussion.

Best regards,
Sancho

--- begin draft of bip-genvbvoting ---

==Preamble==

BIP: ?
Title: Generalized version bits voting
Author: Sancho Panza <sanch0panza@protonmail.com>
Status: Draft
Type: Informational
Created: 2017-03-29
Replaces: 9
License: CC0-1.0
GNU-All-Permissive

==Abstract==

This document describes a generalized version bits voting scheme based
on and intended to replace BIP9.

The generalization consists of allowing each version bit to be treated
individually using a configurable percentage threshold and window size,
instead of the fixed 95% (mainnet) and 2016 block window specified in
BIP9.

The state machine and governing parameters (name, bit, starttime,
timeout) remain as is, but additional parameters called 'threshold' and
'windowsize' are added to the per-bit set.

As before, a set of per-chain parameters will exist for the version bits
governed by BIP9.

==Motivation==

The Bitcoin protocol requires a flexible consensus-finding scheme
to ensure that it can adapt to the needs of the market (its users) and
remain competitive as an electronic payment system.

While BIP9 has served the community reasonably well until now, the
author remarks several shortcomings in its approach:

- it limits itself to backward-compatible changes, precluding its
applicability to hard forks

- a fixed 95% threshold is not flexible enough to allow for a 'spectrum
of contentiousness' to be represented

- the 95% threshold allows small minorities to veto proposed changes,
lead to stagnation (viz. current standoffs)

A more generalized implementation of voting on changes using version bits
can address these issues in a way that can satisfy the needs of both soft
and hard forks, as well as represent varying degrees of contentiousness.

==Specification==

To be elaborated.

It is thought that only cosmetic changes are needed to generalize from
only soft forks to 'soft or hard forks', and to add the additional
per-bit parameters 'threshold' and 'windowsize'

References to fixed values will need to be eliminated and replaced
by respective parameters.

The design of the state machine is envisioned to remain unchanged.

==Implementation==

A reference implementation can be constructed after elaboration of
the specification.

==Copyright==

This BIP is dual-licensed under the Creative Commons CC0 1.0 Universal
and GNU All-Permissive licenses.

--- end draft of bip-genvbvoting ---

[-- Attachment #2: Type: text/html, Size: 4072 bytes --]

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Tom Zander]

On Monday, 3 April 2017 11:06:02 CEST Sancho Panza wrote:
> ==Specification==
> 
> To be elaborated.

Please do elaborate :)

The meat of the proposal is missing.
 
> It is thought that only cosmetic changes are needed to generalize from
> only soft forks to 'soft or hard forks', and to add the additional
> per-bit parameters 'threshold' and 'windowsize'

I agree that the type of forks are rather irrelevant to the voting 
mechanism. As we remember that BIP109 used a voting bit too.

The per-bit (lets call that per-proposal) parameter threshold and windowsize 
are a different matter though, based on the next paragraph you wrote;

> The design of the state machine is envisioned to remain unchanged.

The entire point of BIP9 is to allow nodes that do not know about an upgrade 
to still have a functional state machine. But I don’t see how you can have a 
state machine if the two basic variables that drive it are not specified.

Now, to be clear, I am a big fan of making the window size and the threshold 
more flexible.
But in my opinion we would not be able to have a state machine without those 
variables in the actual BIP because old nodes would miss the data to 
transition to certain states.

Maybe an idea; we have 30 bits. 2 currently in use (although we could reuse 
the CSV one). Maybe we can come up with 3 default sets of properties and 
when a proposal starts to use bit 11 it behaves differently than if it uses 
22.
-- 
Tom Zander
Blog: https://zander.github.io
Vlog: https://vimeo.com/channels/tomscryptochannel

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Sancho Panza]

[-- Attachment #1: Type: text/plain, Size: 2783 bytes --]

Thanks for the feedback.
I'll post a link to more refined proposal on github once that elaboration is more complete.
For now I think more discussion will be very helpful.
I think the flexibility around the tallying window size will take the most careful consideration, so that a user of this proposal can retain full compatibility with BIP9 for a certain versionbit if (s)he wishes.

The entire point of BIP9 is to allow nodes that do not know about an upgrade
to still have a functional state machine. But I don’t see how you can have a
state machine if the two basic variables that drive it are not specified.

What I mean by the state machine remaining essentially unchanged is that its basic design (states and transitions) would remain the same.
But the parameters that decide those transitions would be unique per bit.
I think you misunderstood me if you think there will be strictly one singular state machine.

Instead nodes would effectively be running a state machine instance for each signaling bit - with each state machine possibly (but not necessarily!) configured differently.

An initial implementation might provide this all in compiled code.
A slightly more sophisticated implementation would push the signaling configuration mostly into an external configuration file which could adhere to a fixed format and could easily be adapted and shared between implementations.

But in my opinion we would not be able to have a state machine without those
variables in the actual BIP because old nodes would miss the data to
transition to certain states.

As I see it, this is the same situation we are in now with old nodes - they see that there is some action on unknown bits, but they can do nothing more than warn their operators about this.
This proposal does not fundamentally change that situation.

Maybe an idea; we have 30 bits. 2 currently in use (although we could reuse
the CSV one). Maybe we can come up with 3 default sets of properties and
when a proposal starts to use bit 11 it behaves differently than if it uses
22.

One could place conventions on how certain bit ranges are used, but I don't much see the point of the BIP doing this, although it could suggest examples.

I would prefer if the BIP's reference implementation provides strict BIP9 compatibility in that how it configures the bits (i.e. all with 2016 block windows evaluated in strict synchronicity with BIP9, and default 95% thresholds).
Of course in reality most bits are unused today.
Someone wishing to use a bit for a feature deployment would announce so publicly (e.g. in a BIP) and release an implementation which is suitably configured.
Others wishing to provide compatibility with that feature would adjust their code and bip-genvbvoting configuration files accordingly.

Sancho

[-- Attachment #2: Type: text/html, Size: 3604 bytes --]

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Sancho Panza]

[-- Attachment #1: Type: text/plain, Size: 2967 bytes --]

[Apologies, reposting this in an attempt to improve on the botched formatting of previous reply. I am still getting used to the limitations of this mail service.]

Thanks for the feedback.
I'll post a link to more refined proposal on github once that elaboration is more complete.
For now I think more discussion will be very helpful.
I think the flexibility around the tallying window size will take the most careful consideration, so that a user of this proposal can retain full compatibility with BIP9 for a certain versionbit if (s)he wishes.

> The entire point of BIP9 is to allow nodes that do not know about an upgrade
> to still have a functional state machine. But I don’t see how you can have a
> state machine if the two basic variables that drive it are not specified.

What I mean by the state machine remaining essentially unchanged is that its basic design (states and transitions) would remain the same.
But the parameters that decide those transitions would be unique per bit.
I think you misunderstood me if you think there will be strictly one singular state machine.

Instead nodes would effectively be running a state machine instance for each signaling bit - with each state machine possibly (but not necessarily!) configured differently.

An initial implementation might provide this all in compiled code.
A slightly more sophisticated implementation would push the signaling configuration mostly into an external configuration file which could adhere to a fixed format and could easily be adapted and shared between implementations.

> But in my opinion we would not be able to have a state machine without those
> variables in the actual BIP because old nodes would miss the data to
> transition to certain states.

As I see it, this is the same situation we are in now with old nodes - they see that there is some action on unknown bits, but they can do nothing more than warn their operators about this.
This proposal does not fundamentally change that situation.

> Maybe an idea; we have 30 bits. 2 currently in use (although we could reuse
> the CSV one). Maybe we can come up with 3 default sets of properties and
> when a proposal starts to use bit 11 it behaves differently than if it uses
> 22.

One could place conventions on how certain bit ranges are used, but I don't much see the point of the BIP doing this, although it could suggest examples.

I would prefer if the BIP's reference implementation provides strict BIP9 compatibility in that how it configures the bits (i.e. all with 2016 block windows evaluated in strict synchronicity with BIP9, and default 95% thresholds).
Of course in reality most bits are unused today.
Someone wishing to use a bit for a feature deployment would announce so publicly (e.g. in a BIP) and release an implementation which is suitably configured.
Others wishing to provide compatibility with that feature would adjust their code and bip-genvbvoting configuration files accordingly.

Sancho

[-- Attachment #2: Type: text/html, Size: 3558 bytes --]

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Luke Dashjr]

On Monday, April 03, 2017 9:06:02 AM Sancho Panza via bitcoin-dev wrote:
> While BIP9 has served the community reasonably well until now, the
> author remarks several shortcomings in its approach:
> 
> - it limits itself to backward-compatible changes, precluding its
> applicability to hard forks

BIP 9 doesn't limit itself, merely acknowledges the *inherent* nature of it 
not being applicable to hardforks. BIP 9 provides a mechanism for having 
miners coordinate softforks because they can make the upgrade process smoother 
this way. But the same is not true of hardforks: miners are essentially 
irrelevant to them, and cannot make the process any smoother. Therefore, BIP 9 
and any miner signalling in general is not very useful for deploying these.

> - a fixed 95% threshold is not flexible enough to allow for a 'spectrum
> of contentiousness' to be represented
> 
> - the 95% threshold allows small minorities to veto proposed changes,
> lead to stagnation (viz. current standoffs)

Softforks are not required to use BIP 9, and even if they do, they are not 
required to use the recommended thresholds.

Basically, the problems you're trying to solve don't exist...

Luke

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Sancho Panza]

[-- Attachment #1: Type: text/plain, Size: 3148 bytes --]

> BIP 9 doesn't limit itself, merely acknowledges the *inherent* nature of it
> not being applicable to hardforks. BIP 9 provides a mechanism for having
> miners coordinate softforks because they can make the upgrade process smoother
> this way. But the same is not true of hardforks: miners are essentially
> irrelevant to them, and cannot make the process any smoother.

I accept that BIP9 is inherently concerned only with softforks, as it is explicit about this in every instance.
However, I see no fundamental distinction between the 'royal privilege' assigned to miners w.r.t. softfork activation and the role they would play in properly coordinated hardforks.
In either case, the majority of miners would hopefully want to wait for the right conditions to create the fork block, whether that block be the first one to contain SegWit transactions or the first one to be larger than 1MB (to give two current examples).
The advance coordination with the rest of the users in the system seems important in either case.

This is a big motivator for this BIP: the versionbits can be used as a coordination mechanism for hardforks just as much as softforks.
With the added flexibility offered by this BIP, miners could use these bits to make the process smoother for softforks as well as hardforks.

For example (this is an idea I did not write in the initial BIP draft), the period for which a fork on a particular bit remains LOCKED_IN could be made customizable too, instead of one single retargeting period.
This would allow fork implementors to specify a longer adaptation period suitable to the impacts of the feature they are planning to deploy.

> Therefore, BIP 9 and any miner signalling in general is not very useful
> for deploying these.

I think BIP9 is a very useful tool that allows a decent determination of how much of the hashing power supports a particular fork proposal.

My view is that both soft and hard forks without support from the majority of miners place themselves at high risk.
In general every soft fork can result in a counter hardfork by those who are not aligned with its objectives, just like every hardfork can result in a counter softfork for the same reason by those opposed to it.

It seems to me that this somewhat balances out the (dis)advantages and effectively puts these fork types on a similar footing.
This is a rationale for generalizing the signaling mechanism introduced by BIP9.

In practice, developers will still need to choose whether their feature is best deployed by softfork or hardfork. This proposal affords them that choice, and does not propose any arbitrary conditions (e.g. a predefined split of the versionbits range into particular categories of forks or activation levels).

> Softforks are not required to use BIP 9, and even if they do, they are not
> required to use the recommended thresholds.

This is true, but introducing more flexibility into the signaling framework of BIP9 means it will be more useful for further developments - including a potential hardfork which was on the Core roadmap to accomodate certain wishlist items that cannot easily be addressed by softforks.

[-- Attachment #2: Type: text/html, Size: 3652 bytes --]

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Tom Zander]

On Tuesday, 4 April 2017 20:01:51 CEST Luke Dashjr via bitcoin-dev wrote:
> BIP 9 provides a mechanism for having
> miners coordinate softforks because they can make the upgrade process
> smoother this way. But the same is not true of hardforks: miners are
> essentially irrelevant to them, and cannot make the process any smoother.

Can you explain how miners are irrelevant if the upgrade is not a soft fork?

-- 
Tom Zander
Blog: https://zander.github.io
Vlog: https://vimeo.com/channels/tomscryptochannel

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Thomas Kerin]

[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]

A schism is just that: miners can't ameliorate a HF transition in the way they can censor transactions without permission. This is how miners became a convenient way to activate soft-forks. 

So while BIP9 can indicate the later censorship (a soft fork) in a way that nodes can follow (or not) a hardfork always requires nodes to upgrade to the version increasing the degrees of freedom of the system. 

Signaling is less useful here: the change is not opt-in and will require coordination; and the continuation of the chain thereafter depends on people actually running the hard-fork code, not just being aware there is something happening.


On 04/05/2017 12:08 PM, Tom Zander via bitcoin-dev wrote:

On Tuesday, 4 April 2017 20:01:51 CEST Luke Dashjr via bitcoin-dev wrote: 

BIP 9 provides a mechanism for having miners coordinate softforks because they can make the upgrade process smoother this way. But the same is not true of hardforks: miners are essentially irrelevant to them, and cannot make the process any smoother. 

Can you explain how miners are irrelevant if the upgrade is not a soft fork? 



-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1279 bytes --]

Re: [bitcoin-dev] BIP proposal: Generalized version bits voting (bip-genvbvoting)

[Sancho Panza]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

Thomas,

> the change is not opt-in and will require coordination; and the continuation of the chain thereafter depends on people actually running the hard-fork code, not just being aware there is something happening.

This situation applies to soft forks as well.

- if you wish your software to validate correctly, it is not opt-in
- it requires coordination to activate without much orphan risk to miners (hence BIP9). Witness the long preparation time ahead of SegWit deployment for wallet providers, miners etc. to coordinate to support it on their systems
- after activation, it depends on people running it (most notably miners, otherwise the soft-fork is no longer enforced leading to a hard fork)
- awareness alone does not ensure full validation capability is retained during a soft fork

Therefore, these differences seem insignificant enough to merit treating soft and hard forks equal in terms of the coordination features afforded through the versionbits.

Sancho

[-- Attachment #2: Type: text/html, Size: 1189 bytes --]