[bitcoin-dev] Modern Soft Fork Activation

[bitcoin-dev] Modern Soft Fork Activation

[Yosef]

tl;dr How about 80% ?

The fallback to BIP-8 makes sense, but it's not a graceful one and we absolutely prefer BIP-9 to succeed. A failure to reach 95% readiness signalling means 2.5 years delay, 3.5 years in total, not yet counting.

95% can prove difficult to achieve. Some % of negligent miners that forget to upgrade is expected. Completing that to 5% is not too difficult for a small malicious minority trying to delay the activation. This is the issue Matt's goal #5 aims to prevent, and while the fallback to BIP-8 helps, BIP-9’s 95% requirement makes it worse by allowing quite a neglected minority to force a dramatic delay. Also note how in such case it would have been better to skip BIP-9 altogether and maybe save 1.5 years.

Matt mentions the 95% requirement under goal #3 "Don't (needlessly) lose hashpower to un-upgraded miners". If that is the trade-off, I'd say 2.5 years delay is worse than a momentary loss of hashrate. The protocol is quiet resilient to hashrate fluctuations and, as others mentioned, at that point miners don’t just signal, but lose coins if they don't upgrade, so the hashpower loss is expected to shortly correct. This also means goal #4 is not really effected.

On Sat Jan 11 14:42:07 UTC 2020, Anthony Towns wrote:

> For me, the focus there is on Matt's first point: "avoid activating [or merging, or even proposing] in the face of significant, reasonable, and directed objection"

I agree, and believe the outreach and review process around taproot are maybe the best we had so far. Notably goal #1 should be mostly satisfied already at merge time, so risking 3.5 years delay after that, seems excessive.

BIP-91 “Reduced threshold Segwit MASF” was deployed by miners specifically to reduce the 95% requirement down to 80% during the segwit drama. While hopefully taproot won’t produce any such excitements, the events around segwit activation and the weird “hash wars” meme that later followed – might encourage some to try similar games again.

The difference between `5% minus apathetic-miners` and `20% minus apathetic-miners` is dramatic and can make such attempts an order of magnitude more difficult.

The taproot process is looking great so far, I feel it will be a mistake to put it on a route that can easily extend to so many years. I suggest keeping Matt’s proposal as is but decrease BIP-9’s 95% threshold down to around 80%.

Yosef

Re: [bitcoin-dev] Modern Soft Fork Activation

[Anthony Towns]

On Mon, Jan 13, 2020 at 08:34:24AM +0000, Yosef via bitcoin-dev wrote:
> tl;dr How about 80% ?

The point of having hashpower upgraded is that it means that there's low
liklihood of long chains of blocks that are invalid per the new rules, so
that if you haven't upgraded your node but wait for a few confirmations,
you'll still (with very high liklihood) only see blocks valid per the
new rules.

If you have 80% of miners enforcing the rules, then if someone produces
a block that violates the new rules (but is valid for the old ones),
then you've got a 20% chance of one of the non-enforcing miners getting
the next block, and a 4% chance of non-enforcing miners getting both
the next blocks, giving 3 confirmations to invalid transactions. That
seems a bit high.

3 confirmations isn't unrealistic, eg Coinbase apparently recently
dropped its requirement to that apparently:

https://blog.coinbase.com/announcing-new-confirmation-requirements-4a5504ba8d81

I could maybe see a 90% threshold though?

> 95% can prove difficult to achieve. Some % of negligent miners that forget to upgrade is expected.

Is it? We went from 59% to 54% to 28% to 0% (!!) of blocks not signalling
for segwit during consecutive two-week blocks in the BIP-91/148
period; and from 100% of blocks not signalling for BIP-91 to 99.4%,
48%, 15%, and 11% during consecutive 2.3 day periods targeting an 80%
threshold. Certainly that was a particularly high-stakes period, but
they were both pretty short. For comparison, for CSV, we went from 100%
not signalling to 61%, to 54% to 3.4% in consecutive two-week periods.

> Completing that to 5% is not too difficult for a small malicious minority trying to delay the activation. This is the issue Matt's goal #5 aims to prevent, and while the fallback to BIP-8 helps, BIP-9’s 95% requirement makes it worse by allowing quite a neglected minority to force a dramatic delay. Also note how in such case it would have been better to skip BIP-9 altogether and maybe save 1.5 years.

I don't think you can really skip steps if you need a flag day:

 - the first 12 months is for *really seriously* making sure there's no
   problems with the proposed upgrade; you can't that because people
   might not look for problems until the code's out there and ready for
   actual use

 - the next 6 months is for updating the software to lock in the flag
   day; you can't skip that because it takes time to get new releases out

 - the next 24 months is to ensure everyone's upgraded their nodes so
   that they won't be at risk of thinking they've received bitcoins when
   those coins aren't in compliance with the new rules; and you can't
   skip that because if we don't have hashpower reliably enforcing the
   rules, *everybody* needs to upgrade, which can take a lot of time.

Times could be tweaked, but the "everyone has to upgrade their node
software" is almost the same constraint that hard forks have, so I think
you want to end up with a long overall lead time any which way. For
comparison, 0.12.1 came out about 45 months ago and 0.13.2 came out
about 36 months ago -- about 0.5% of nodes are running 0.12 or earler,
and about 4.9% of nodes are running 0.13 or earlier, at least per [0],
so the overall timeline of 42 months seems plausible to me...

[0] https://luke.dashjr.org/programs/bitcoin/files/charts/software.html

I think (especially if we attempt BIP-91/BIP-148-style compulsory
signalling again) it's worth also considering the failure case if miners
false-signal: that is they signal support of the new soft-fork rules,
but then don't actually enforce them. If you end up with, say, 15% of
hashpower not upgraded or signalling, 25% of hashpower not upgraded but
signalling so their blocks don't get orphaned, and only 65% of hashpower
upgraded, you have a 1% chance of 5 blocks built on top of a block
that's invalid according to the new rules, giving those transactions 6
confirmations as far as non-upgraded nodes are concerned.

Cheers,
aj

Re: [bitcoin-dev] Modern Soft Fork Activation

[Anthony Towns]

On Tue, Jan 14, 2020 at 07:42:07PM +0000, Matt Corallo wrote:
> Thus, part of the goal here is that we ensure we have that "out", and
> can observe the response of the ecosystem once the change is "staring
> them in the face", as it were.

> A BIP 9 process is here not only to offer
> a compelling activation path, but *also* to allow for observation and
> discussion time for any lingering minor objections prior to a BIP 8/flag
> day activation.

One thing that I wonder is if your proposal (and BIP9) has enough of
time for this sort of observation?

If something looks good to devs and miners, but still has some
underlying problem, it seems like it would be pretty easy to for it
to activate quickly just because miners happen to upgrade quickly and
don't see a need to tweak the default signalling parameters. I think
the BIP 68/112/113 bundle starttime was met at block 409643 (May 1,
2016), so it entered STARTED at 411264 (May 11), was LOCKED_IN at 417312
(June 21), and active at 481824 (July 5). If we're worried people will
only seriously look at things once activation is possible, having just
a month or two to find new problems isn't very long.

One approach to improve that might be to have the first point release that
includes the soft-fork activation parameters _not_ update getblocktemplate
to signal the version bit by default, but only do that in a second point
release later. That way miners could manually enable signalling if there's
some reason to rush things (which still means there's pressure to actually
look at the changes), but by default there's a bit of extra time.

(This might just be a reason why people should look at proposals before
they're ready to activate, though; or why users of bitcoin should also
be miners)

> On the other hand, in practice, we've seen that version bits are set on
> the pool side, and not on the node side, meaning the goal of ensuring
> miners have upgraded isn't really accomplished in practice, you just end
> up forking the chain for no gain.

ITYM version bits are set via mining software rather than the node
software the constructs blocks (when validation happens), so that
there's no strong link between signalling and having actually updated
your software to properly enforce the new rules? I think people have
suggested in the past moving signalling into the coinbase or similar
rather than the version field of the header to make that link a bit
tighter. Maybe this is worth doing at the same time? (For pools that
want to let their users choose whether to signal or not, that'd mean
offering two different templates for them to mine, I guess) That would
mean miners using the version field as extra nonce space wouldn't be
confused with upgrade signalling at least...

(I don't have an opinion on whether either of these is worth worrying
about)

Cheers,
aj

Re: [bitcoin-dev] Modern Soft Fork Activation

[Matt Corallo]

In general, your thoughts on the theory of how consensus changes should
work I strongly agree with. However, my one significant disagreement is
how practical it is for things to *actually* work that way. While I wish
ecosystem players (both businesses and users) spent their time
interacting with the Bitcoin development community enough that they had
a deep understanding of upcoming protocol change designs, it just isn't
realistic to expect that. Thus, having an "out" to avoid activation
after a release has been cut with fork activation logic is quite a
compelling requirement.

Thus, part of the goal here is that we ensure we have that "out", and
can observe the response of the ecosystem once the change is "staring
them in the face", as it were. A BIP 9 process is here not only to offer
a compelling activation path, but *also* to allow for observation and
discussion time for any lingering minor objections prior to a BIP 8/flag
day activation.

As for a "mandatory signaling period" as a part of BIP 8, I find this
idea strange both in that it flies in the face of all recent soft fork
design work, and because it doesn't actually accomplish its stated goal.

Recent soft-fork design has all been about how to design something with
minimal ecosystem impact. Certainly in the 95% activation case I can't
say I feel strongly, but if you actually *hit* the BIP 8 flag day,
deliberately causing significant network forks for old clients has the
potential to cause real ecosystem risk. While part of the reason for a
24-month time horizon between BIP 8 decision and flag-day activation
endeavors to de-risk the chance that major players are running on
un-upgraded nodes, you cannot ignore the reality of them, both full-,
and SPV-clients.

On the other hand, in practice, we've seen that version bits are set on
the pool side, and not on the node side, meaning the goal of ensuring
miners have upgraded isn't really accomplished in practice, you just end
up forking the chain for no gain.

Matt

On 1/11/20 2:42 PM, Anthony Towns wrote:
> On Fri, Jan 10, 2020 at 09:30:09PM +0000, Matt Corallo via bitcoin-dev wrote:
>> 1) a standard BIP 9 deployment with a one-year time horizon for
>> activation with 95% miner readiness,
>> 2) in the case that no activation occurs within a year, a six month
>> quieting period during which the community can analyze and discussion
>> the reasons for no activation and,
>> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
>> parameter which was supported since the original deployment release
>> would enable users to opt into a BIP 8 deployment with a 24-month
>> time-horizon for flag-day activation (as well as a new Bitcoin Core
>> release enabling the flag universally).
> 
> FWIW etc, but my perspective on this is that the way we want consensus
> changes in Bitcoin to work is:
> 
>  - decentralised: we want everyone to be able to participate, in
>    designing/promoting/reviewing changes, without decision making
>    power getting centralised amongst one group or another
> 
>  - technical: we want changes to be judged on their objective technical
>    merits; politics and animal spirits and the like are fine, especially
>    for working out what to prioritise, but they shouldn't be part of the
>    final yes/no decision on consensus changes
> 
>  - improvements: changes might not make everyone better off, but we
>    don't want changes to screw anyone over either -- pareto
>    improvements in economics, "first, do no harm", etc. (if we get this
>    right, there's no need to make compromises and bundle multiple
>    flawed proposals so that everyone's an equal mix of happy and
>    miserable)
> 
> In particular, we don't want to misalign skills and responsibilities: it's
> fine for developers to judge if a proposal has bugs or technical problems,
> but we don't want want developers to have to decide if a proposal is
> "sufficiently popular" or "economically sound" and the like, for instance.
> Likewise we don't want to have miners or pool operators have to take
> responsibility for managing the whole economy, rather than just keeping
> their systems running.
> 
> So the way I hope this will work out is:
> 
>  - investors, industry, people in general work out priorities for what's
>    valuable to work on; this is an economic/policy/subjective question,
>    that everyone can participate in, and everyone can act on --
>    either directly if they're developers who can work on proposals and
>    implementations directly, or indirectly by persuading or paying other
>    people to work on whatever's important
> 
>  - developers work on proposals, designing and implementing them to make
>    (some subset of) bitcoin users better off, and to not make anyone worse
>    off.
> 
>  - if someone discovers a good technical reason why a proposal does make
>    people worse off, we don't try to keep pushing the proposal over the
>    top of objections, but go back to the drawing board and try to fix
>    the problems
> 
>  - once we've done as much development as we can, including setting up
>    experimental testnet/signet style deployments for testing, we setup a
>    deployment. the idea at this point is to make sure the live network
>    upgrade works, and to retain the ability to abort if last minute
>    problems come up. no doubt some review and testing will be left until
>    the last minute and only done here, but *ideally* the focus should be
>    on catching errors *well before* this point.
> 
>  - as a result, the activation strategy mostly needs to be about ensuring
>    that the Bitcoin network stays in consensus, rather than checking
>    popularity or voting -- the yes/no decisions should have mostly been
>    made earlier already. so we have two strategies for locking in the
>    upgrade: either 95%+ of hashpower signals that they've upgraded to
>    software that will enforce the changes forever more, _or_ after a
>    year of trying to deploy, we fail to find any technical problems,
>    and then allow an additional 2.5 years to ensure all node software is
>    upgraded to enforce the new rules before locking them in.
> 
> The strategy behind the last point is that we need to establish that
> there's consensus amongst all of Bitcoin before we commit to a flag day,
> and if we've found some method to establish consensus on that, then we're
> done -- we've already got consensus, we don't need to put a blockchain
> protocol on top of that and signal that we've got consensus. (Activating
> via hashpower still needs signalling, because we need to coordinate on
> *when* sufficient hashpower has upgraded)
> 
> This approach is obviously compatible with BIP-148 or BIP-91 style
> forced-signalling UASFs if some upgrade does need to be done urgently
> despite miner opposition; the forced signalling just needs to occur during
> the BIP-9 or BIP-8 phases, and no during the "quiet period". Probably the
> first period of BIP-8 after the quiet period would make the most sense.
> 
> But without that, this approach seems very friendly for miners: even
> if they don't upgrade, they won't mine invalid blocks (unless the rules
> activate and someone else deliberately mines an invalid block and they
> build on top of it), and if a change is incompatible with, say 10%
> of hashpower, it won't be forced on them for 3.5 years, by which point
> it's probably a good bet that everyone's upgrading to a new generation
> of mining hardware anyway. But even that's a backstop, because if a
> change *is* incompatible with existing mining hardware, that's an easily
> describable technical problem that should mean we go back to the drawing
> board and fix it, not deploy the change despite the problems. [0]
> 
> On Fri, Jan 10, 2020 at 11:21:51PM +0100, Jorge Timón via bitcoin-dev wrote:
>> Regarding bip8-like activation, luke-jr suggested that [..] a
>> consensus rule could require the blocks to signal for activation in
>> the last activation window.
> 
> FWIW, that had been my (strong) preference too, but I think I'm now
> convinced it's not needed/useful.
> 
>> I see 2 main advantages for this:
>> 1) Outdated nodes can implement warnings (like in bip9) and they can
>> see those warnings even if it's activated in the last activation
>> window.
> 
> The 3.5 year window from BIP-9-starttime to BIP-8-flagday means you'd
> have to be using *very* out of date software to need to autodetect
> unknown upgrades. If an upgrade starts on 2021-01-01 say, it'd be
> supported by 0.21.x, 0.22.0, and 0.23.0 (with bip8 as an opt-in) and
> 0.24.0, 0.25.0, 0.26.0, 0.27.0, and 0.28.0 (with bip8 as always on)
> before flag day activation on 2024-06-01.
> 
> 0.21.x to 0.23.x could warn if they see potential early BIP-8 activation
> via versionbits, and also warn if the flag day date is seen saying "flag
> day activation may have happened, please check external sources and
> consider upgrading your node software".
> 
> So you'd need to be running 0.20.x, released 4 years prior to the
> activation to be outdated enough to not get warnings, I think.
> 
>> 2) It is easier for users to actively resist a given change they
>> oppose. Instead of requiring signaling, their nodes can be set to
>> ignore chains that activate it. This will result in a fork, but if
>> different groups of users want different things, this is arguably the
>> best behaviour: a "clean" split.
> 
> If you're knowingly doing a deliberate minority chain split, you'll
> almost certainly change the PoW function, and trivially get a clean
> split as a result of doing that.
> 
> But I think what we want is to move away from consensus decisions being a
> "who has the most money/power/hashpower/nodes/reddit-accounts" contest
> to being a question of "have we dealt with every reasonable technical
> objection?" -- I think that's better for decentralisation in that anyone
> can stop a bad proposal without having to be rich/powerful/persuasive,
> and better for encouraging constructive contributions. 
> 
> The other side to this is that if it's just a matter of resolving
> technical problems, then it's also straightforward for a small but skilled
> group to get a consensus change through even if the vast majority doesn't
> think it's a priority -- they just need to make a good proposal, make
> sure it doesn't make people worse off, work through all the objections
> people find, and be willing to wait for it to go through reviews and
> upgrade steps which may take extra time if other people don't think it's
> a high priority. But those are all just technical challenges, that can
> be resolved with skill and patience, whoever you might be. So to me,
> that's a win for decentralisation as well.
> 
>> I assume many people won't like this, but I really think we should
>> consider how users should ideally resist an unwanted change, even if
>> the proponents had the best intentions in mind, there may be
>> legitimate reasons to resist it that they may not have considered.
> 
> For me, the focus there is on Matt's first point: "avoid activating
> [or merging, or even proposing] in the face of significant, reasonable,
> and directed objection". If you want to stop a change you should have to
> do nothing more than describe the problems with it; and if there aren't
> problems with it, you shouldn't be trying to stop the change.
> 
> (A benefit to having the BIP-8 settings defined simultaneously with
> the initial activation attempt is that it means that if the core
> devs/maintainers go crazy with power and try to force/prevent the BIP-8
> activation despite clear community consensus going the other way, then
> it will be easy to for the client, and set the parameter correctly --
> literally just a matter of changing a value in chainparams.cpp, unlike the
> difficulties of changing the blocksize from 1MB to 2MB. Other variations
> of this overall approach have the same benefit)
> 
> Cheers,
> aj (very grateful to Greg and Matt for explaining a lot of thing
>     about this approach and helping resolve my concerns with it)
> 
> [0] Trigger warning, PTSD over the 2015-2017 blocksize wars...
> 
>     The segwit timeline was something like this:
> 
>      2015-05 - blocksize debate begins on bitcoin-dev
>      2015-08 - bitcoin xt with bip101 hardfork released
>      2015-09 - scaling bitcoin phase 1
>      2015-12 - segwit proposal at scaling bitcoin phase 2
>      2016-01 - segwit testnet launched
>      2016-02 - bitcoin classic with bip109 hardfork released
>      2016-04 - first release (0.12.1) with a bip9 deployment (csv)
>      2016-06 - segwit merged
>      2016-07 - csv activated
>      2016-10 - first release (0.13.1) with segwit activation params
>      2016-11 - segwit activation starttime
>      2017-02 - UASF first proposed
>      2017-03 - antpool to swith to bitcoin unlimited
>      2017-04 - covert ASICBoost vs segwit conflict described
>      2017-05 - NY segwit2x agreement, btc1 with bip102 hardfork started
>      2017-05 - BIP-91 proposed
>      2017-06 - UAHF proposal from bitmain that became BCH
>      2017-07 - BIP-91 lockin
>      2017-08 - BIP-148 activation
>      2017-08 - BCH chainsplit
>      2017-08 - segwit lockin and activation
>      2017-11 - 2x fork called off; btc1 nodes stall; 2x chain stillborn
>      2018-02 - first release (0.16.0) with segwit wallet support
> 
>     (That's about 33 months in total, compared to the 24 months we've
>     already spent since taproot was first described in Jan 2018, or the
>     42 months before flag-day activation in Matt's proposal)
> 
>     I don't think that timeline is a good example of how things should
>     work, and would call out a few mistakes in particular:
> 
>      * too much urgency to increase the blocksize resulting in rushed
>        decision making, especially for the hardfork implementations, but
>        also for segwit
> 
>      * alternative clients attempted to activate forks without
>        resolving technical problems (eventually resulting in the btc1
>        client stalling prior to the expected hard fork block, eg)
> 
>      * a lot of emphasis was on numbers (market share, hashpower, etc)
>        rather than technical merits, resulting in a lot of false
>        signalling an political meaneuvering
> 
>      * the incompatibility between ASICBoost and segwit wasn't noticed
>        prior to activation, and wasn't fixed when it was noticed
>        (certainly you can justify this as a tit-for-tat response to the
>        other errors having been made in bad faith, or as not being a real
>        problem because everyone claimed that they weren't doing covert
>        ASICBoost, but considered on its own I think the incompatibility
>        should have been resolved)
> 
>      * the UASF approach had significant potential technical problems
>        (potential for long reorgs, p2p network splits) that weren't
>        resolved by the time it became active. happily, this was mitigated
>        by hashpower enforcement of BIP-148 rules via BIP-91. neither
>        BIP-148 or BIP-91 gained enough consensus to be supported in
>        bitcoin core though
> 
>     I don't personally think we need to fix every problem we had with
>     segwit's process -- it eventually mostly worked out okay, after all --
>     but I think Matt's approach has a good chance of fixing a lot of
>     them, while still leaving us flexibility to deal with whatever new
>     problems we come up with in their place.
> 

Re: [bitcoin-dev] Modern Soft Fork Activation

[Matt Corallo]

Good thing no one is proposing a naive BIP 9 approach :). I'll note that
BIP 9 has been fairly robust (spy-mining issues notwithstanding, which
we believe are at least largely solved in the wild) in terms of safety,
though I noted extensively in the first mail that it failed in terms of
misunderstanding the activation parameters. I think the above proposal
largely solves that, and I don't see much in the way of arguing that
point from you, here.

As an aside, BIP 9 is also the Devil We Know, which carries a lot of
value, since we've found (and addressed) direct issues with it, whereas
all other activation methods we have ~0 experience with in the modern
Bitcoin network.

On 1/10/20 11:37 PM, Luke Dashjr wrote:
> I think BIP 9 is a proven failure, and flag day softforks have their own 
> problems:
> 
> A) There is no way to unambiguously say "the rules for this chain are 
> <x,y,z>". It leaves the chain in a kind of "quantum state" where the rules 
> could be one thing, or could be another. Until the new rules are violated, we 
> do not know if the softfork was a success or not. Because of this, people 
> will rightly shy away from relying on the new rules. This problem is made 
> worse by the fact that common node policies might not produce blocks which 
> violate the rules. If we had gone with BIP149 for Segwit, it is IMO probable 
> we would still not have a clear answer today to "Is Segwit active or not?"
> 
> B) Because of (A), there is also no clear way to intentionally reject the 
> softfork. Those who do not consent to it are effectively compelled to accept 
> it anyway. While it is usually possible to craft an opposing softfork, this 
> should IMO be well-defined and simple to do (including a plan to do so in any 
> BIP9-alike spec).
> 
> For these reasons, in 2017, I proposed revising BIP 8 with a mandatory signal, 
> similar to how BIP148 worked: https://github.com/bitcoin/bips/pull/550
> However, the author of BIP 8 has since vanished, and because we had no 
> immediate softfork plans, efforts to move this forward were abandoned 
> temporarily. It seems like a good time to resume this work.
> 
> In regard to your goal #3, I would like to note that after the mandatory 
> signal period, old miners could resume mining unchanged. This means there is 
> a temporary loss of hashrate to the network, but I think it is overall better 
> than the alternatives. The temporary loss of income from invalid blocks will 
> also give affected miners a last push to upgrade, hopefully improving the 
> long run security of the network hashrate.
> 
> Luke
> 
> (P.S. As for your #1, I do think it is oversimplified in some cases, but we 
> should leave that for later discussion when it actually becomes relevant.)

Re: [bitcoin-dev] Modern Soft Fork Activation

[Luke Dashjr]

On Saturday 11 January 2020 14:42:07 Anthony Towns wrote:
>  the UASF approach had significant potential technical problems
>        (potential for long reorgs, p2p network splits) that weren't
>        resolved by the time it became active.

Long reorgs, only for old nodes, were a possibility, but not a problem.

The p2p network split issues WERE resolved well before activation.
(In fact, Bitcoin Knots still ships with the general p2p fixes.)

>        neither 
>        BIP-148 or BIP-91 gained enough consensus to be supported in
>        bitcoin core though

There was no measurable difference in community support between BIP148 and 
Segwit itself, months before BIP148's activation. (There was about 20% that 
indicated they would support BIP148 "only if Bitcoin Core releases it", which 
IMO "counts" in this context.)

The only difference was in the opinions of developers. Basing the decision to 
exclude BIP148 as even an *option* on this basis was IMO improper and 
shouldn't be repeated. The community's readiness to switch to another 
fork/build for UASFs is also valuable, but shouldn't be necessary.

Luke

Re: [bitcoin-dev] Modern Soft Fork Activation

[Anthony Towns]

On Fri, Jan 10, 2020 at 09:30:09PM +0000, Matt Corallo via bitcoin-dev wrote:
> 1) a standard BIP 9 deployment with a one-year time horizon for
> activation with 95% miner readiness,
> 2) in the case that no activation occurs within a year, a six month
> quieting period during which the community can analyze and discussion
> the reasons for no activation and,
> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
> parameter which was supported since the original deployment release
> would enable users to opt into a BIP 8 deployment with a 24-month
> time-horizon for flag-day activation (as well as a new Bitcoin Core
> release enabling the flag universally).

FWIW etc, but my perspective on this is that the way we want consensus
changes in Bitcoin to work is:

 - decentralised: we want everyone to be able to participate, in
   designing/promoting/reviewing changes, without decision making
   power getting centralised amongst one group or another

 - technical: we want changes to be judged on their objective technical
   merits; politics and animal spirits and the like are fine, especially
   for working out what to prioritise, but they shouldn't be part of the
   final yes/no decision on consensus changes

 - improvements: changes might not make everyone better off, but we
   don't want changes to screw anyone over either -- pareto
   improvements in economics, "first, do no harm", etc. (if we get this
   right, there's no need to make compromises and bundle multiple
   flawed proposals so that everyone's an equal mix of happy and
   miserable)

In particular, we don't want to misalign skills and responsibilities: it's
fine for developers to judge if a proposal has bugs or technical problems,
but we don't want want developers to have to decide if a proposal is
"sufficiently popular" or "economically sound" and the like, for instance.
Likewise we don't want to have miners or pool operators have to take
responsibility for managing the whole economy, rather than just keeping
their systems running.

So the way I hope this will work out is:

 - investors, industry, people in general work out priorities for what's
   valuable to work on; this is an economic/policy/subjective question,
   that everyone can participate in, and everyone can act on --
   either directly if they're developers who can work on proposals and
   implementations directly, or indirectly by persuading or paying other
   people to work on whatever's important

 - developers work on proposals, designing and implementing them to make
   (some subset of) bitcoin users better off, and to not make anyone worse
   off.

 - if someone discovers a good technical reason why a proposal does make
   people worse off, we don't try to keep pushing the proposal over the
   top of objections, but go back to the drawing board and try to fix
   the problems

 - once we've done as much development as we can, including setting up
   experimental testnet/signet style deployments for testing, we setup a
   deployment. the idea at this point is to make sure the live network
   upgrade works, and to retain the ability to abort if last minute
   problems come up. no doubt some review and testing will be left until
   the last minute and only done here, but *ideally* the focus should be
   on catching errors *well before* this point.

 - as a result, the activation strategy mostly needs to be about ensuring
   that the Bitcoin network stays in consensus, rather than checking
   popularity or voting -- the yes/no decisions should have mostly been
   made earlier already. so we have two strategies for locking in the
   upgrade: either 95%+ of hashpower signals that they've upgraded to
   software that will enforce the changes forever more, _or_ after a
   year of trying to deploy, we fail to find any technical problems,
   and then allow an additional 2.5 years to ensure all node software is
   upgraded to enforce the new rules before locking them in.

The strategy behind the last point is that we need to establish that
there's consensus amongst all of Bitcoin before we commit to a flag day,
and if we've found some method to establish consensus on that, then we're
done -- we've already got consensus, we don't need to put a blockchain
protocol on top of that and signal that we've got consensus. (Activating
via hashpower still needs signalling, because we need to coordinate on
*when* sufficient hashpower has upgraded)

This approach is obviously compatible with BIP-148 or BIP-91 style
forced-signalling UASFs if some upgrade does need to be done urgently
despite miner opposition; the forced signalling just needs to occur during
the BIP-9 or BIP-8 phases, and no during the "quiet period". Probably the
first period of BIP-8 after the quiet period would make the most sense.

But without that, this approach seems very friendly for miners: even
if they don't upgrade, they won't mine invalid blocks (unless the rules
activate and someone else deliberately mines an invalid block and they
build on top of it), and if a change is incompatible with, say 10%
of hashpower, it won't be forced on them for 3.5 years, by which point
it's probably a good bet that everyone's upgrading to a new generation
of mining hardware anyway. But even that's a backstop, because if a
change *is* incompatible with existing mining hardware, that's an easily
describable technical problem that should mean we go back to the drawing
board and fix it, not deploy the change despite the problems. [0]

On Fri, Jan 10, 2020 at 11:21:51PM +0100, Jorge Timón via bitcoin-dev wrote:
> Regarding bip8-like activation, luke-jr suggested that [..] a
> consensus rule could require the blocks to signal for activation in
> the last activation window.

FWIW, that had been my (strong) preference too, but I think I'm now
convinced it's not needed/useful.

> I see 2 main advantages for this:
> 1) Outdated nodes can implement warnings (like in bip9) and they can
> see those warnings even if it's activated in the last activation
> window.

The 3.5 year window from BIP-9-starttime to BIP-8-flagday means you'd
have to be using *very* out of date software to need to autodetect
unknown upgrades. If an upgrade starts on 2021-01-01 say, it'd be
supported by 0.21.x, 0.22.0, and 0.23.0 (with bip8 as an opt-in) and
0.24.0, 0.25.0, 0.26.0, 0.27.0, and 0.28.0 (with bip8 as always on)
before flag day activation on 2024-06-01.

0.21.x to 0.23.x could warn if they see potential early BIP-8 activation
via versionbits, and also warn if the flag day date is seen saying "flag
day activation may have happened, please check external sources and
consider upgrading your node software".

So you'd need to be running 0.20.x, released 4 years prior to the
activation to be outdated enough to not get warnings, I think.

> 2) It is easier for users to actively resist a given change they
> oppose. Instead of requiring signaling, their nodes can be set to
> ignore chains that activate it. This will result in a fork, but if
> different groups of users want different things, this is arguably the
> best behaviour: a "clean" split.

If you're knowingly doing a deliberate minority chain split, you'll
almost certainly change the PoW function, and trivially get a clean
split as a result of doing that.

But I think what we want is to move away from consensus decisions being a
"who has the most money/power/hashpower/nodes/reddit-accounts" contest
to being a question of "have we dealt with every reasonable technical
objection?" -- I think that's better for decentralisation in that anyone
can stop a bad proposal without having to be rich/powerful/persuasive,
and better for encouraging constructive contributions. 

The other side to this is that if it's just a matter of resolving
technical problems, then it's also straightforward for a small but skilled
group to get a consensus change through even if the vast majority doesn't
think it's a priority -- they just need to make a good proposal, make
sure it doesn't make people worse off, work through all the objections
people find, and be willing to wait for it to go through reviews and
upgrade steps which may take extra time if other people don't think it's
a high priority. But those are all just technical challenges, that can
be resolved with skill and patience, whoever you might be. So to me,
that's a win for decentralisation as well.

> I assume many people won't like this, but I really think we should
> consider how users should ideally resist an unwanted change, even if
> the proponents had the best intentions in mind, there may be
> legitimate reasons to resist it that they may not have considered.

For me, the focus there is on Matt's first point: "avoid activating
[or merging, or even proposing] in the face of significant, reasonable,
and directed objection". If you want to stop a change you should have to
do nothing more than describe the problems with it; and if there aren't
problems with it, you shouldn't be trying to stop the change.

(A benefit to having the BIP-8 settings defined simultaneously with
the initial activation attempt is that it means that if the core
devs/maintainers go crazy with power and try to force/prevent the BIP-8
activation despite clear community consensus going the other way, then
it will be easy to for the client, and set the parameter correctly --
literally just a matter of changing a value in chainparams.cpp, unlike the
difficulties of changing the blocksize from 1MB to 2MB. Other variations
of this overall approach have the same benefit)

Cheers,
aj (very grateful to Greg and Matt for explaining a lot of thing
    about this approach and helping resolve my concerns with it)

[0] Trigger warning, PTSD over the 2015-2017 blocksize wars...

    The segwit timeline was something like this:

     2015-05 - blocksize debate begins on bitcoin-dev
     2015-08 - bitcoin xt with bip101 hardfork released
     2015-09 - scaling bitcoin phase 1
     2015-12 - segwit proposal at scaling bitcoin phase 2
     2016-01 - segwit testnet launched
     2016-02 - bitcoin classic with bip109 hardfork released
     2016-04 - first release (0.12.1) with a bip9 deployment (csv)
     2016-06 - segwit merged
     2016-07 - csv activated
     2016-10 - first release (0.13.1) with segwit activation params
     2016-11 - segwit activation starttime
     2017-02 - UASF first proposed
     2017-03 - antpool to swith to bitcoin unlimited
     2017-04 - covert ASICBoost vs segwit conflict described
     2017-05 - NY segwit2x agreement, btc1 with bip102 hardfork started
     2017-05 - BIP-91 proposed
     2017-06 - UAHF proposal from bitmain that became BCH
     2017-07 - BIP-91 lockin
     2017-08 - BIP-148 activation
     2017-08 - BCH chainsplit
     2017-08 - segwit lockin and activation
     2017-11 - 2x fork called off; btc1 nodes stall; 2x chain stillborn
     2018-02 - first release (0.16.0) with segwit wallet support

    (That's about 33 months in total, compared to the 24 months we've
    already spent since taproot was first described in Jan 2018, or the
    42 months before flag-day activation in Matt's proposal)

    I don't think that timeline is a good example of how things should
    work, and would call out a few mistakes in particular:

     * too much urgency to increase the blocksize resulting in rushed
       decision making, especially for the hardfork implementations, but
       also for segwit

     * alternative clients attempted to activate forks without
       resolving technical problems (eventually resulting in the btc1
       client stalling prior to the expected hard fork block, eg)

     * a lot of emphasis was on numbers (market share, hashpower, etc)
       rather than technical merits, resulting in a lot of false
       signalling an political meaneuvering

     * the incompatibility between ASICBoost and segwit wasn't noticed
       prior to activation, and wasn't fixed when it was noticed
       (certainly you can justify this as a tit-for-tat response to the
       other errors having been made in bad faith, or as not being a real
       problem because everyone claimed that they weren't doing covert
       ASICBoost, but considered on its own I think the incompatibility
       should have been resolved)

     * the UASF approach had significant potential technical problems
       (potential for long reorgs, p2p network splits) that weren't
       resolved by the time it became active. happily, this was mitigated
       by hashpower enforcement of BIP-148 rules via BIP-91. neither
       BIP-148 or BIP-91 gained enough consensus to be supported in
       bitcoin core though

    I don't personally think we need to fix every problem we had with
    segwit's process -- it eventually mostly worked out okay, after all --
    but I think Matt's approach has a good chance of fixing a lot of
    them, while still leaving us flexibility to deal with whatever new
    problems we come up with in their place.

Re: [bitcoin-dev] Modern Soft Fork Activation

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 1363 bytes --]

It's not at a "directly implementable policy state", but I think you might
be interested in checking out the spork protocol upgrade model I proposed a
while back. https://www.youtube.com/watch?v=J1CP7qbnpqA&feature=youtu.be

It has some interesting properties around the 5 properties you've mentioned.

1) Avoid activating in the face of significant, reasonable, and directed
objection. Period.

Up to miners to orphan spork-activating blocks.

2) Avoid activating within a timeframe which does not make high
node-level-adoption likely.

Mandatory minimum flag day for Spork initiation, statistically
improbable/impossible for even earlier adoption.

3) Don't (needlessly) lose hashpower to un-upgraded miners.

Difficulty adjustments make the missing spork'd block "go away" over time,
the additional difficulty of *not activating* a rejected spork fills in as
an additional PoW.


4) Use hashpower enforcement to de-risk the upgrade process, wherever
possible.

Miners choose to activate or build on activating blocks.

5) Follow the will of the community, irrespective of individuals or
unreasoned objection, but without ever overruling any reasonable
objection.

Honest signalling makes people be forced to "put their money where there
mouth is" on what the community wants.
--
@JeremyRubin <https://twitter.com/JeremyRubin>
<https://twitter.com/JeremyRubin>

[-- Attachment #2: Type: text/html, Size: 4099 bytes --]

Re: [bitcoin-dev] Modern Soft Fork Activation

[Luke Dashjr]

I think BIP 9 is a proven failure, and flag day softforks have their own 
problems:

A) There is no way to unambiguously say "the rules for this chain are 
<x,y,z>". It leaves the chain in a kind of "quantum state" where the rules 
could be one thing, or could be another. Until the new rules are violated, we 
do not know if the softfork was a success or not. Because of this, people 
will rightly shy away from relying on the new rules. This problem is made 
worse by the fact that common node policies might not produce blocks which 
violate the rules. If we had gone with BIP149 for Segwit, it is IMO probable 
we would still not have a clear answer today to "Is Segwit active or not?"

B) Because of (A), there is also no clear way to intentionally reject the 
softfork. Those who do not consent to it are effectively compelled to accept 
it anyway. While it is usually possible to craft an opposing softfork, this 
should IMO be well-defined and simple to do (including a plan to do so in any 
BIP9-alike spec).

For these reasons, in 2017, I proposed revising BIP 8 with a mandatory signal, 
similar to how BIP148 worked: https://github.com/bitcoin/bips/pull/550
However, the author of BIP 8 has since vanished, and because we had no 
immediate softfork plans, efforts to move this forward were abandoned 
temporarily. It seems like a good time to resume this work.

In regard to your goal #3, I would like to note that after the mandatory 
signal period, old miners could resume mining unchanged. This means there is 
a temporary loss of hashrate to the network, but I think it is overall better 
than the alternatives. The temporary loss of income from invalid blocks will 
also give affected miners a last push to upgrade, hopefully improving the 
long run security of the network hashrate.

Luke

(P.S. As for your #1, I do think it is oversimplified in some cases, but we 
should leave that for later discussion when it actually becomes relevant.)



On Friday 10 January 2020 21:30:09 Matt Corallo via bitcoin-dev wrote:
> There are a series of soft-fork designs which have recently been making
> good progress towards implementation and future adoption. However, for
> various reasons, activation methods therefor have gotten limited
> discussion. I'd like to reopen that discussion here.
>
> It is likely worth revisiting the goals both for soft forks and their
> activation methods to start. I'm probably missing some, but some basic
> requirements:
>
> 1) Avoid activating in the face of significant, reasonable, and directed
> objection. Period. If someone has a well-accepted, reasonable use of
> Bitcoin that is working today, have no reason to believe wouldn't work
> long into the future without a change, and which would be made
> impossible or significantly more difficult by a change, that change must
> not happen. I certainly hope there is no objection on this point (see
> the last point for an important caveat that I'm sure everyone will jump
> to point out).
>
> 2) Avoid activating within a timeframe which does not make high
> node-level-adoption likely. As with all "node" arguments, I'll note that
> I mean "economically-used" nodes, not the thousand or so spy nodes on
> Google Cloud and AWS. Rule changes don't make sense without nodes
> enforcing them, whether they happen to be a soft fork, hard fork, or a
> blue fork, so activating in a reduced timeframe that doesn't allow for
> large-scale node adoption doesn't have any value, and may cause other
> unintended side effects.
>
> 3) Don't (needlessly) lose hashpower to un-upgraded miners. As a part of
> Bitcoin's security comes from miners, reducing the hashpower of the
> network as a side effect of a rule change is a needless reduction in a
> key security parameter of the network. This is why, in recent history,
> soft forks required 95% of hashpower to indicate that they have upgraded
> and are capable of enforcing the new rules. Further, this is why recent
> soft forks have not included changes which would result in a standard
> Bitcoin Core instance mining invalid-by-new-rules changes (by relying on
> the standardness behavior of Bitcoin Core).
>
> 4) Use hashpower enforcement to de-risk the upgrade process, wherever
> possible. As a corollary of the above, one of the primary reasons we use
> soft forks is that hashpower-based enforcement of rules is an elegant
> way to prevent network splits during the node upgrade process. While it
> does not make sense to invest material value in systems protected by new
> rules until a significant majority of "economic nodes" is enforcing said
> rules, hashpower lets us neatly bridge the gap in time between
> activation and then. By having a supermajority of miners enforce the new
> rules, attempts at violating the new rules does not result in a
> significant network split, disrupting existing users of the system. If
> we aren't going to take advantage of this, we should do a hard fork
> instead, with the necessarily slow timescale that entails.
>
> 5) Follow the will of the community, irrespective of individuals or
> unreasoned objection, but without ever overruling any reasonable
> objection. Recent history also includes "objection" to soft forks in the
> form of "this is bad because it doesn't fix a different problem I want
> fixed ASAP". I don't think anyone would argue this qualifies as a
> reasonable objection to a change, and we should be in a place, as a
> community (never as developers or purely one group), to ignore such
> objections and make forward progress in spite of them. We don't make
> good engineering decisions by "bundling" unrelated features together to
> enable political football and compromise.
>
> I think BIP 9 (plus a well-crafted softfork) pretty effectively checks
> the boxes for #2-4 here, and when done carefully with lots of community
> engagement and measurement, can effectively fulfill #1 as well. #5 is,
> as I'm sure everyone is aware, where it starts to fall down pretty hard.
>
> BIP 8 has been proposed as an alternative, largely in response to issues
> with #5. However, a naive deployment of it, rather obviously, completely
> fails #1, #3, and #4, and, in my view, fails #5 as well by both giving
> an impression of, setting a precedent of, and possibly even in practice
> increasing the ability of developers to decide the consensus rules of
> the system. A BIP 8 deployment that more accurately measures community
> support as a prerequisite could arguably fulfill #1 and #5, though I'm
> unaware of any concrete proposals on how to accomplish that. Arguably, a
> significantly longer activation window could also allow BIP 8 to fulfill
> #3 and #4, but only by exploiting the "needlessly" and "wherever
> possible" loopholes.
>
> You may note that, from the point of view of achieving the critical
> goals here, BIP 8 is only different from a flag-day activation in that,
> if it takes the "happy-path" of activating before the flag day, it looks
> like BIP 9, but isn't guaranteed to. It additionally has the
> "nice-to-have" property that activation can occur before the flag-day in
> the case of faster miner adoption, though there is a limit of how fast
> is useful due to node adoption.
>
> Thus, to strike a balance between the drawbacks of BIP 8 and BIP 9, the
> Great Consensus Cleanup softfork proposal included this text in the
>
> discussion section (with the spec describing a BIP 9 deployment):
> > In spite of some suggestion that other activation methods be used, BIP
> > 9 is proposed as ensuring miners have upgraded to enforce new rules is
> > an important part of minimizing disruption. While previous BIP 9 soft-
> > forks have resulted in political contention, this comparatively-
> > unimportant soft-fork provides a good opportunity to attempt to return
> > to utilizing BIP 9 to ensure miner upgrade prior to activation, which
> > the authors believe is a critical goal. However, if there is broad
> > agreement to activate these rules when the BIP 9 expiry time is
> > reached, and miners have not yet signaled sufficient level of
> > readiness, a later flag-day activation may be merited. For this
> > reason, implementations may wish to provide a compatibility option
> > which allows flag-day enforcement of these rules without an update.
>
> Ultimately, through admittedly rather limited discussion, I still like
> this model (though I cannot claim it as my own, the original proposal
> came from Greg Maxwell). BIP 9 only falls apart in case of unreasonable
> objection, which, naturally, should carry a high bar to ignore, given we
> have to have some level of agreement that it is, in fact, unreasonable
> (or untargeted). While I admit this is a possibility, I both find it
> less likely than in previous soft-forks, and even if it is the case, it
> only slows down the process, it doesn't necessarily stop it. In the case
> that it does fail, BIP 9 process, in fact, provides a good learning
> opportunity as to the level of community readiness and desire for a
> given change. While we can (and should, and are) learning a lot about
> community readiness for, and acceptability of a change through outreach
> and discussion, there is something about a change with a timeframe that
> forces people to more carefully consider it.
>
> Thus, as something a bit more concrete, I think an activation method
> which sets the right precedent and appropriately considers the above
> goals, would be:
>
> 1) a standard BIP 9 deployment with a one-year time horizon for
> activation with 95% miner readiness,
> 2) in the case that no activation occurs within a year, a six month
> quieting period during which the community can analyze and discussion
> the reasons for no activation and,
> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
> parameter which was supported since the original deployment release
> would enable users to opt into a BIP 8 deployment with a 24-month
> time-horizon for flag-day activation (as well as a new Bitcoin Core
> release enabling the flag universally).
>
> This provides a very long time horizon for more standard activation,
> while still ensuring the goals in #5 are met, even if, in those cases,
> the time horizon needs to be significantly extended to meet the goals of
> #3. Developing Bitcoin is not a race. If we have to, waiting 42 months
> ensures we're not setting a negative precedent that we'll come to regret
> as Bitcoin continues to grow.
>
> Matt
>
> Thanks also to AJ for feedback on an earlier version of this rant.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Modern Soft Fork Activation

[Jorge Timón]

I see how your approach doesn't lose goal 3 while "mine" does.
Regarding goal 4, I don't think any of the approaches loses it. "Use
hashpower enforcement to de-risk the upgrade process, wherever
possible".
Well, in the case of activation while there's "many" non upgrade
miners, they simply can't help to reduce upgrade risks unless they
upgrade. It doesn't matter if the activation is silent or with
mandatory signaling. Am I missing something?

>  in order to nudge miners

That's not the goal at all. All my arguments have been focused on
users, not miners.

> If you want to fork yourself off the network, you can do it in easier ways,

Well, it's not that you want to fork yourself off the network, is that
you don't want change X. Ideally change X wouldn't be activated, but
if it is, you prefer to be in a chain without change X.
Let's say we're using your system to deploy change X you oppose for
legitimate reasons.
What easier thing would you do as a user to resist change X with all
other users who also oppose it?

If there are simpler and better ways to do this, great. It's just
something to think about.





On Fri, Jan 10, 2020 at 11:43 PM Matt Corallo <lf-lists@mattcorallo.com> wrote:
>
> I went back and forth with a few folks on this one. I think the fact that we lose goals 3/4 very explicitly in order to nudge miners seems like a poor trade off. I’ll note that your point 2 here seems a bit disconnected to me. If you want to fork yourself off the network, you can do it in easier ways, and if miners want to maliciously censors transactions to the detriment of users, rejecting a version bit doesn’t really help avoid that.
>
> Your point about upgrade warnings is well-made, but I’m dubious of it’s value over the network chaos many large forks might otherwise cause.
>
> Matt
>
> > On Jan 10, 2020, at 17:22, Jorge Timón <jtimon@jtimon.cc> wrote:
> >
> > Well, bip9 doesn't only fall apart in case of unreasonable objection,
> > it also fails simply with miners' apathy.
> > Anyway, your proposed plan should take care of that case too, I think.
> > Overall sounds good to me.
> >
> > Regarding bip8-like activation, luke-jr suggested that instead of
> > simply activating on date x if failed to do so by miners' signaling, a
> > consensus rule could require the blocks to signal for activation in
> > the last activation window.
> > I see 2 main advantages for this:
> >
> > 1) Outdated nodes can implement warnings (like in bip9) and they can
> > see those warnings even if it's activated in the last activation
> > window. Of course this can become counterproductive if miners' squat
> > signaling bits for asicboost again.
> >
> > 2) It is easier for users to actively resist a given change they
> > oppose. Instead of requiring signaling, their nodes can be set to
> > ignore chains that activate it. This will result in a fork, but if
> > different groups of users want different things, this is arguably the
> > best behaviour: a "clean" split.
> >
> > I assume many people won't like this, but I really think we should
> > consider how users should ideally resist an unwanted change, even if
> > the proponents had the best intentions in mind, there may be
> > legitimate reasons to resist it that they may not have considered.
> >
> >> On Fri, Jan 10, 2020 at 10:30 PM Matt Corallo via bitcoin-dev
> >> <bitcoin-dev@lists.linuxfoundation.org> wrote:
> >>
> >> There are a series of soft-fork designs which have recently been making
> >> good progress towards implementation and future adoption. However, for
> >> various reasons, activation methods therefor have gotten limited
> >> discussion. I'd like to reopen that discussion here.
> >>
> >> It is likely worth revisiting the goals both for soft forks and their
> >> activation methods to start. I'm probably missing some, but some basic
> >> requirements:
> >>
> >> 1) Avoid activating in the face of significant, reasonable, and directed
> >> objection. Period. If someone has a well-accepted, reasonable use of
> >> Bitcoin that is working today, have no reason to believe wouldn't work
> >> long into the future without a change, and which would be made
> >> impossible or significantly more difficult by a change, that change must
> >> not happen. I certainly hope there is no objection on this point (see
> >> the last point for an important caveat that I'm sure everyone will jump
> >> to point out).
> >>
> >> 2) Avoid activating within a timeframe which does not make high
> >> node-level-adoption likely. As with all "node" arguments, I'll note that
> >> I mean "economically-used" nodes, not the thousand or so spy nodes on
> >> Google Cloud and AWS. Rule changes don't make sense without nodes
> >> enforcing them, whether they happen to be a soft fork, hard fork, or a
> >> blue fork, so activating in a reduced timeframe that doesn't allow for
> >> large-scale node adoption doesn't have any value, and may cause other
> >> unintended side effects.
> >>
> >> 3) Don't (needlessly) lose hashpower to un-upgraded miners. As a part of
> >> Bitcoin's security comes from miners, reducing the hashpower of the
> >> network as a side effect of a rule change is a needless reduction in a
> >> key security parameter of the network. This is why, in recent history,
> >> soft forks required 95% of hashpower to indicate that they have upgraded
> >> and are capable of enforcing the new rules. Further, this is why recent
> >> soft forks have not included changes which would result in a standard
> >> Bitcoin Core instance mining invalid-by-new-rules changes (by relying on
> >> the standardness behavior of Bitcoin Core).
> >>
> >> 4) Use hashpower enforcement to de-risk the upgrade process, wherever
> >> possible. As a corollary of the above, one of the primary reasons we use
> >> soft forks is that hashpower-based enforcement of rules is an elegant
> >> way to prevent network splits during the node upgrade process. While it
> >> does not make sense to invest material value in systems protected by new
> >> rules until a significant majority of "economic nodes" is enforcing said
> >> rules, hashpower lets us neatly bridge the gap in time between
> >> activation and then. By having a supermajority of miners enforce the new
> >> rules, attempts at violating the new rules does not result in a
> >> significant network split, disrupting existing users of the system. If
> >> we aren't going to take advantage of this, we should do a hard fork
> >> instead, with the necessarily slow timescale that entails.
> >>
> >> 5) Follow the will of the community, irrespective of individuals or
> >> unreasoned objection, but without ever overruling any reasonable
> >> objection. Recent history also includes "objection" to soft forks in the
> >> form of "this is bad because it doesn't fix a different problem I want
> >> fixed ASAP". I don't think anyone would argue this qualifies as a
> >> reasonable objection to a change, and we should be in a place, as a
> >> community (never as developers or purely one group), to ignore such
> >> objections and make forward progress in spite of them. We don't make
> >> good engineering decisions by "bundling" unrelated features together to
> >> enable political football and compromise.
> >>
> >> I think BIP 9 (plus a well-crafted softfork) pretty effectively checks
> >> the boxes for #2-4 here, and when done carefully with lots of community
> >> engagement and measurement, can effectively fulfill #1 as well. #5 is,
> >> as I'm sure everyone is aware, where it starts to fall down pretty hard.
> >>
> >> BIP 8 has been proposed as an alternative, largely in response to issues
> >> with #5. However, a naive deployment of it, rather obviously, completely
> >> fails #1, #3, and #4, and, in my view, fails #5 as well by both giving
> >> an impression of, setting a precedent of, and possibly even in practice
> >> increasing the ability of developers to decide the consensus rules of
> >> the system. A BIP 8 deployment that more accurately measures community
> >> support as a prerequisite could arguably fulfill #1 and #5, though I'm
> >> unaware of any concrete proposals on how to accomplish that. Arguably, a
> >> significantly longer activation window could also allow BIP 8 to fulfill
> >> #3 and #4, but only by exploiting the "needlessly" and "wherever
> >> possible" loopholes.
> >>
> >> You may note that, from the point of view of achieving the critical
> >> goals here, BIP 8 is only different from a flag-day activation in that,
> >> if it takes the "happy-path" of activating before the flag day, it looks
> >> like BIP 9, but isn't guaranteed to. It additionally has the
> >> "nice-to-have" property that activation can occur before the flag-day in
> >> the case of faster miner adoption, though there is a limit of how fast
> >> is useful due to node adoption.
> >>
> >> Thus, to strike a balance between the drawbacks of BIP 8 and BIP 9, the
> >> Great Consensus Cleanup softfork proposal included this text in the
> >> discussion section (with the spec describing a BIP 9 deployment):
> >>
> >>> In spite of some suggestion that other activation methods be used, BIP
> >>> 9 is proposed as ensuring miners have upgraded to enforce new rules is
> >>> an important part of minimizing disruption. While previous BIP 9 soft-
> >>> forks have resulted in political contention, this comparatively-
> >>> unimportant soft-fork provides a good opportunity to attempt to return
> >>> to utilizing BIP 9 to ensure miner upgrade prior to activation, which
> >>> the authors believe is a critical goal. However, if there is broad
> >>> agreement to activate these rules when the BIP 9 expiry time is
> >>> reached, and miners have not yet signaled sufficient level of
> >>> readiness, a later flag-day activation may be merited. For this
> >>> reason, implementations may wish to provide a compatibility option
> >>> which allows flag-day enforcement of these rules without an update.
> >>
> >> Ultimately, through admittedly rather limited discussion, I still like
> >> this model (though I cannot claim it as my own, the original proposal
> >> came from Greg Maxwell). BIP 9 only falls apart in case of unreasonable
> >> objection, which, naturally, should carry a high bar to ignore, given we
> >> have to have some level of agreement that it is, in fact, unreasonable
> >> (or untargeted). While I admit this is a possibility, I both find it
> >> less likely than in previous soft-forks, and even if it is the case, it
> >> only slows down the process, it doesn't necessarily stop it. In the case
> >> that it does fail, BIP 9 process, in fact, provides a good learning
> >> opportunity as to the level of community readiness and desire for a
> >> given change. While we can (and should, and are) learning a lot about
> >> community readiness for, and acceptability of a change through outreach
> >> and discussion, there is something about a change with a timeframe that
> >> forces people to more carefully consider it.
> >>
> >> Thus, as something a bit more concrete, I think an activation method
> >> which sets the right precedent and appropriately considers the above
> >> goals, would be:
> >>
> >> 1) a standard BIP 9 deployment with a one-year time horizon for
> >> activation with 95% miner readiness,
> >> 2) in the case that no activation occurs within a year, a six month
> >> quieting period during which the community can analyze and discussion
> >> the reasons for no activation and,
> >> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
> >> parameter which was supported since the original deployment release
> >> would enable users to opt into a BIP 8 deployment with a 24-month
> >> time-horizon for flag-day activation (as well as a new Bitcoin Core
> >> release enabling the flag universally).
> >>
> >> This provides a very long time horizon for more standard activation,
> >> while still ensuring the goals in #5 are met, even if, in those cases,
> >> the time horizon needs to be significantly extended to meet the goals of
> >> #3. Developing Bitcoin is not a race. If we have to, waiting 42 months
> >> ensures we're not setting a negative precedent that we'll come to regret
> >> as Bitcoin continues to grow.
> >>
> >> Matt
> >>
> >> Thanks also to AJ for feedback on an earlier version of this rant.
> >> _______________________________________________
> >> bitcoin-dev mailing list
> >> bitcoin-dev@lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] Modern Soft Fork Activation

[Matt Corallo]

I went back and forth with a few folks on this one. I think the fact that we lose goals 3/4 very explicitly in order to nudge miners seems like a poor trade off. I’ll note that your point 2 here seems a bit disconnected to me. If you want to fork yourself off the network, you can do it in easier ways, and if miners want to maliciously censors transactions to the detriment of users, rejecting a version bit doesn’t really help avoid that.

Your point about upgrade warnings is well-made, but I’m dubious of it’s value over the network chaos many large forks might otherwise cause.

Matt

> On Jan 10, 2020, at 17:22, Jorge Timón <jtimon@jtimon.cc> wrote:
> 
> Well, bip9 doesn't only fall apart in case of unreasonable objection,
> it also fails simply with miners' apathy.
> Anyway, your proposed plan should take care of that case too, I think.
> Overall sounds good to me.
> 
> Regarding bip8-like activation, luke-jr suggested that instead of
> simply activating on date x if failed to do so by miners' signaling, a
> consensus rule could require the blocks to signal for activation in
> the last activation window.
> I see 2 main advantages for this:
> 
> 1) Outdated nodes can implement warnings (like in bip9) and they can
> see those warnings even if it's activated in the last activation
> window. Of course this can become counterproductive if miners' squat
> signaling bits for asicboost again.
> 
> 2) It is easier for users to actively resist a given change they
> oppose. Instead of requiring signaling, their nodes can be set to
> ignore chains that activate it. This will result in a fork, but if
> different groups of users want different things, this is arguably the
> best behaviour: a "clean" split.
> 
> I assume many people won't like this, but I really think we should
> consider how users should ideally resist an unwanted change, even if
> the proponents had the best intentions in mind, there may be
> legitimate reasons to resist it that they may not have considered.
> 
>> On Fri, Jan 10, 2020 at 10:30 PM Matt Corallo via bitcoin-dev
>> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>> 
>> There are a series of soft-fork designs which have recently been making
>> good progress towards implementation and future adoption. However, for
>> various reasons, activation methods therefor have gotten limited
>> discussion. I'd like to reopen that discussion here.
>> 
>> It is likely worth revisiting the goals both for soft forks and their
>> activation methods to start. I'm probably missing some, but some basic
>> requirements:
>> 
>> 1) Avoid activating in the face of significant, reasonable, and directed
>> objection. Period. If someone has a well-accepted, reasonable use of
>> Bitcoin that is working today, have no reason to believe wouldn't work
>> long into the future without a change, and which would be made
>> impossible or significantly more difficult by a change, that change must
>> not happen. I certainly hope there is no objection on this point (see
>> the last point for an important caveat that I'm sure everyone will jump
>> to point out).
>> 
>> 2) Avoid activating within a timeframe which does not make high
>> node-level-adoption likely. As with all "node" arguments, I'll note that
>> I mean "economically-used" nodes, not the thousand or so spy nodes on
>> Google Cloud and AWS. Rule changes don't make sense without nodes
>> enforcing them, whether they happen to be a soft fork, hard fork, or a
>> blue fork, so activating in a reduced timeframe that doesn't allow for
>> large-scale node adoption doesn't have any value, and may cause other
>> unintended side effects.
>> 
>> 3) Don't (needlessly) lose hashpower to un-upgraded miners. As a part of
>> Bitcoin's security comes from miners, reducing the hashpower of the
>> network as a side effect of a rule change is a needless reduction in a
>> key security parameter of the network. This is why, in recent history,
>> soft forks required 95% of hashpower to indicate that they have upgraded
>> and are capable of enforcing the new rules. Further, this is why recent
>> soft forks have not included changes which would result in a standard
>> Bitcoin Core instance mining invalid-by-new-rules changes (by relying on
>> the standardness behavior of Bitcoin Core).
>> 
>> 4) Use hashpower enforcement to de-risk the upgrade process, wherever
>> possible. As a corollary of the above, one of the primary reasons we use
>> soft forks is that hashpower-based enforcement of rules is an elegant
>> way to prevent network splits during the node upgrade process. While it
>> does not make sense to invest material value in systems protected by new
>> rules until a significant majority of "economic nodes" is enforcing said
>> rules, hashpower lets us neatly bridge the gap in time between
>> activation and then. By having a supermajority of miners enforce the new
>> rules, attempts at violating the new rules does not result in a
>> significant network split, disrupting existing users of the system. If
>> we aren't going to take advantage of this, we should do a hard fork
>> instead, with the necessarily slow timescale that entails.
>> 
>> 5) Follow the will of the community, irrespective of individuals or
>> unreasoned objection, but without ever overruling any reasonable
>> objection. Recent history also includes "objection" to soft forks in the
>> form of "this is bad because it doesn't fix a different problem I want
>> fixed ASAP". I don't think anyone would argue this qualifies as a
>> reasonable objection to a change, and we should be in a place, as a
>> community (never as developers or purely one group), to ignore such
>> objections and make forward progress in spite of them. We don't make
>> good engineering decisions by "bundling" unrelated features together to
>> enable political football and compromise.
>> 
>> I think BIP 9 (plus a well-crafted softfork) pretty effectively checks
>> the boxes for #2-4 here, and when done carefully with lots of community
>> engagement and measurement, can effectively fulfill #1 as well. #5 is,
>> as I'm sure everyone is aware, where it starts to fall down pretty hard.
>> 
>> BIP 8 has been proposed as an alternative, largely in response to issues
>> with #5. However, a naive deployment of it, rather obviously, completely
>> fails #1, #3, and #4, and, in my view, fails #5 as well by both giving
>> an impression of, setting a precedent of, and possibly even in practice
>> increasing the ability of developers to decide the consensus rules of
>> the system. A BIP 8 deployment that more accurately measures community
>> support as a prerequisite could arguably fulfill #1 and #5, though I'm
>> unaware of any concrete proposals on how to accomplish that. Arguably, a
>> significantly longer activation window could also allow BIP 8 to fulfill
>> #3 and #4, but only by exploiting the "needlessly" and "wherever
>> possible" loopholes.
>> 
>> You may note that, from the point of view of achieving the critical
>> goals here, BIP 8 is only different from a flag-day activation in that,
>> if it takes the "happy-path" of activating before the flag day, it looks
>> like BIP 9, but isn't guaranteed to. It additionally has the
>> "nice-to-have" property that activation can occur before the flag-day in
>> the case of faster miner adoption, though there is a limit of how fast
>> is useful due to node adoption.
>> 
>> Thus, to strike a balance between the drawbacks of BIP 8 and BIP 9, the
>> Great Consensus Cleanup softfork proposal included this text in the
>> discussion section (with the spec describing a BIP 9 deployment):
>> 
>>> In spite of some suggestion that other activation methods be used, BIP
>>> 9 is proposed as ensuring miners have upgraded to enforce new rules is
>>> an important part of minimizing disruption. While previous BIP 9 soft-
>>> forks have resulted in political contention, this comparatively-
>>> unimportant soft-fork provides a good opportunity to attempt to return
>>> to utilizing BIP 9 to ensure miner upgrade prior to activation, which
>>> the authors believe is a critical goal. However, if there is broad
>>> agreement to activate these rules when the BIP 9 expiry time is
>>> reached, and miners have not yet signaled sufficient level of
>>> readiness, a later flag-day activation may be merited. For this
>>> reason, implementations may wish to provide a compatibility option
>>> which allows flag-day enforcement of these rules without an update.
>> 
>> Ultimately, through admittedly rather limited discussion, I still like
>> this model (though I cannot claim it as my own, the original proposal
>> came from Greg Maxwell). BIP 9 only falls apart in case of unreasonable
>> objection, which, naturally, should carry a high bar to ignore, given we
>> have to have some level of agreement that it is, in fact, unreasonable
>> (or untargeted). While I admit this is a possibility, I both find it
>> less likely than in previous soft-forks, and even if it is the case, it
>> only slows down the process, it doesn't necessarily stop it. In the case
>> that it does fail, BIP 9 process, in fact, provides a good learning
>> opportunity as to the level of community readiness and desire for a
>> given change. While we can (and should, and are) learning a lot about
>> community readiness for, and acceptability of a change through outreach
>> and discussion, there is something about a change with a timeframe that
>> forces people to more carefully consider it.
>> 
>> Thus, as something a bit more concrete, I think an activation method
>> which sets the right precedent and appropriately considers the above
>> goals, would be:
>> 
>> 1) a standard BIP 9 deployment with a one-year time horizon for
>> activation with 95% miner readiness,
>> 2) in the case that no activation occurs within a year, a six month
>> quieting period during which the community can analyze and discussion
>> the reasons for no activation and,
>> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
>> parameter which was supported since the original deployment release
>> would enable users to opt into a BIP 8 deployment with a 24-month
>> time-horizon for flag-day activation (as well as a new Bitcoin Core
>> release enabling the flag universally).
>> 
>> This provides a very long time horizon for more standard activation,
>> while still ensuring the goals in #5 are met, even if, in those cases,
>> the time horizon needs to be significantly extended to meet the goals of
>> #3. Developing Bitcoin is not a race. If we have to, waiting 42 months
>> ensures we're not setting a negative precedent that we'll come to regret
>> as Bitcoin continues to grow.
>> 
>> Matt
>> 
>> Thanks also to AJ for feedback on an earlier version of this rant.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Modern Soft Fork Activation

[Jorge Timón]

Well, bip9 doesn't only fall apart in case of unreasonable objection,
it also fails simply with miners' apathy.
Anyway, your proposed plan should take care of that case too, I think.
Overall sounds good to me.

Regarding bip8-like activation, luke-jr suggested that instead of
simply activating on date x if failed to do so by miners' signaling, a
consensus rule could require the blocks to signal for activation in
the last activation window.
I see 2 main advantages for this:

1) Outdated nodes can implement warnings (like in bip9) and they can
see those warnings even if it's activated in the last activation
window. Of course this can become counterproductive if miners' squat
signaling bits for asicboost again.

2) It is easier for users to actively resist a given change they
oppose. Instead of requiring signaling, their nodes can be set to
ignore chains that activate it. This will result in a fork, but if
different groups of users want different things, this is arguably the
best behaviour: a "clean" split.

I assume many people won't like this, but I really think we should
consider how users should ideally resist an unwanted change, even if
the proponents had the best intentions in mind, there may be
legitimate reasons to resist it that they may not have considered.

On Fri, Jan 10, 2020 at 10:30 PM Matt Corallo via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> There are a series of soft-fork designs which have recently been making
> good progress towards implementation and future adoption. However, for
> various reasons, activation methods therefor have gotten limited
> discussion. I'd like to reopen that discussion here.
>
> It is likely worth revisiting the goals both for soft forks and their
> activation methods to start. I'm probably missing some, but some basic
> requirements:
>
> 1) Avoid activating in the face of significant, reasonable, and directed
> objection. Period. If someone has a well-accepted, reasonable use of
> Bitcoin that is working today, have no reason to believe wouldn't work
> long into the future without a change, and which would be made
> impossible or significantly more difficult by a change, that change must
> not happen. I certainly hope there is no objection on this point (see
> the last point for an important caveat that I'm sure everyone will jump
> to point out).
>
> 2) Avoid activating within a timeframe which does not make high
> node-level-adoption likely. As with all "node" arguments, I'll note that
> I mean "economically-used" nodes, not the thousand or so spy nodes on
> Google Cloud and AWS. Rule changes don't make sense without nodes
> enforcing them, whether they happen to be a soft fork, hard fork, or a
> blue fork, so activating in a reduced timeframe that doesn't allow for
> large-scale node adoption doesn't have any value, and may cause other
> unintended side effects.
>
> 3) Don't (needlessly) lose hashpower to un-upgraded miners. As a part of
> Bitcoin's security comes from miners, reducing the hashpower of the
> network as a side effect of a rule change is a needless reduction in a
> key security parameter of the network. This is why, in recent history,
> soft forks required 95% of hashpower to indicate that they have upgraded
> and are capable of enforcing the new rules. Further, this is why recent
> soft forks have not included changes which would result in a standard
> Bitcoin Core instance mining invalid-by-new-rules changes (by relying on
> the standardness behavior of Bitcoin Core).
>
> 4) Use hashpower enforcement to de-risk the upgrade process, wherever
> possible. As a corollary of the above, one of the primary reasons we use
> soft forks is that hashpower-based enforcement of rules is an elegant
> way to prevent network splits during the node upgrade process. While it
> does not make sense to invest material value in systems protected by new
> rules until a significant majority of "economic nodes" is enforcing said
> rules, hashpower lets us neatly bridge the gap in time between
> activation and then. By having a supermajority of miners enforce the new
> rules, attempts at violating the new rules does not result in a
> significant network split, disrupting existing users of the system. If
> we aren't going to take advantage of this, we should do a hard fork
> instead, with the necessarily slow timescale that entails.
>
> 5) Follow the will of the community, irrespective of individuals or
> unreasoned objection, but without ever overruling any reasonable
> objection. Recent history also includes "objection" to soft forks in the
> form of "this is bad because it doesn't fix a different problem I want
> fixed ASAP". I don't think anyone would argue this qualifies as a
> reasonable objection to a change, and we should be in a place, as a
> community (never as developers or purely one group), to ignore such
> objections and make forward progress in spite of them. We don't make
> good engineering decisions by "bundling" unrelated features together to
> enable political football and compromise.
>
> I think BIP 9 (plus a well-crafted softfork) pretty effectively checks
> the boxes for #2-4 here, and when done carefully with lots of community
> engagement and measurement, can effectively fulfill #1 as well. #5 is,
> as I'm sure everyone is aware, where it starts to fall down pretty hard.
>
> BIP 8 has been proposed as an alternative, largely in response to issues
> with #5. However, a naive deployment of it, rather obviously, completely
> fails #1, #3, and #4, and, in my view, fails #5 as well by both giving
> an impression of, setting a precedent of, and possibly even in practice
> increasing the ability of developers to decide the consensus rules of
> the system. A BIP 8 deployment that more accurately measures community
> support as a prerequisite could arguably fulfill #1 and #5, though I'm
> unaware of any concrete proposals on how to accomplish that. Arguably, a
> significantly longer activation window could also allow BIP 8 to fulfill
> #3 and #4, but only by exploiting the "needlessly" and "wherever
> possible" loopholes.
>
> You may note that, from the point of view of achieving the critical
> goals here, BIP 8 is only different from a flag-day activation in that,
> if it takes the "happy-path" of activating before the flag day, it looks
> like BIP 9, but isn't guaranteed to. It additionally has the
> "nice-to-have" property that activation can occur before the flag-day in
> the case of faster miner adoption, though there is a limit of how fast
> is useful due to node adoption.
>
> Thus, to strike a balance between the drawbacks of BIP 8 and BIP 9, the
> Great Consensus Cleanup softfork proposal included this text in the
> discussion section (with the spec describing a BIP 9 deployment):
>
> > In spite of some suggestion that other activation methods be used, BIP
> > 9 is proposed as ensuring miners have upgraded to enforce new rules is
> > an important part of minimizing disruption. While previous BIP 9 soft-
> > forks have resulted in political contention, this comparatively-
> > unimportant soft-fork provides a good opportunity to attempt to return
> > to utilizing BIP 9 to ensure miner upgrade prior to activation, which
> > the authors believe is a critical goal. However, if there is broad
> > agreement to activate these rules when the BIP 9 expiry time is
> > reached, and miners have not yet signaled sufficient level of
> > readiness, a later flag-day activation may be merited. For this
> > reason, implementations may wish to provide a compatibility option
> > which allows flag-day enforcement of these rules without an update.
>
> Ultimately, through admittedly rather limited discussion, I still like
> this model (though I cannot claim it as my own, the original proposal
> came from Greg Maxwell). BIP 9 only falls apart in case of unreasonable
> objection, which, naturally, should carry a high bar to ignore, given we
> have to have some level of agreement that it is, in fact, unreasonable
> (or untargeted). While I admit this is a possibility, I both find it
> less likely than in previous soft-forks, and even if it is the case, it
> only slows down the process, it doesn't necessarily stop it. In the case
> that it does fail, BIP 9 process, in fact, provides a good learning
> opportunity as to the level of community readiness and desire for a
> given change. While we can (and should, and are) learning a lot about
> community readiness for, and acceptability of a change through outreach
> and discussion, there is something about a change with a timeframe that
> forces people to more carefully consider it.
>
> Thus, as something a bit more concrete, I think an activation method
> which sets the right precedent and appropriately considers the above
> goals, would be:
>
> 1) a standard BIP 9 deployment with a one-year time horizon for
> activation with 95% miner readiness,
> 2) in the case that no activation occurs within a year, a six month
> quieting period during which the community can analyze and discussion
> the reasons for no activation and,
> 3) in the case that it makes sense, a simple command-line/bitcoin.conf
> parameter which was supported since the original deployment release
> would enable users to opt into a BIP 8 deployment with a 24-month
> time-horizon for flag-day activation (as well as a new Bitcoin Core
> release enabling the flag universally).
>
> This provides a very long time horizon for more standard activation,
> while still ensuring the goals in #5 are met, even if, in those cases,
> the time horizon needs to be significantly extended to meet the goals of
> #3. Developing Bitcoin is not a race. If we have to, waiting 42 months
> ensures we're not setting a negative precedent that we'll come to regret
> as Bitcoin continues to grow.
>
> Matt
>
> Thanks also to AJ for feedback on an earlier version of this rant.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Modern Soft Fork Activation

[Matt Corallo]

There are a series of soft-fork designs which have recently been making
good progress towards implementation and future adoption. However, for
various reasons, activation methods therefor have gotten limited
discussion. I'd like to reopen that discussion here.

It is likely worth revisiting the goals both for soft forks and their
activation methods to start. I'm probably missing some, but some basic
requirements:

1) Avoid activating in the face of significant, reasonable, and directed
objection. Period. If someone has a well-accepted, reasonable use of
Bitcoin that is working today, have no reason to believe wouldn't work
long into the future without a change, and which would be made
impossible or significantly more difficult by a change, that change must
not happen. I certainly hope there is no objection on this point (see
the last point for an important caveat that I'm sure everyone will jump
to point out).

2) Avoid activating within a timeframe which does not make high
node-level-adoption likely. As with all "node" arguments, I'll note that
I mean "economically-used" nodes, not the thousand or so spy nodes on
Google Cloud and AWS. Rule changes don't make sense without nodes
enforcing them, whether they happen to be a soft fork, hard fork, or a
blue fork, so activating in a reduced timeframe that doesn't allow for
large-scale node adoption doesn't have any value, and may cause other
unintended side effects.

3) Don't (needlessly) lose hashpower to un-upgraded miners. As a part of
Bitcoin's security comes from miners, reducing the hashpower of the
network as a side effect of a rule change is a needless reduction in a
key security parameter of the network. This is why, in recent history,
soft forks required 95% of hashpower to indicate that they have upgraded
and are capable of enforcing the new rules. Further, this is why recent
soft forks have not included changes which would result in a standard
Bitcoin Core instance mining invalid-by-new-rules changes (by relying on
the standardness behavior of Bitcoin Core).

4) Use hashpower enforcement to de-risk the upgrade process, wherever
possible. As a corollary of the above, one of the primary reasons we use
soft forks is that hashpower-based enforcement of rules is an elegant
way to prevent network splits during the node upgrade process. While it
does not make sense to invest material value in systems protected by new
rules until a significant majority of "economic nodes" is enforcing said
rules, hashpower lets us neatly bridge the gap in time between
activation and then. By having a supermajority of miners enforce the new
rules, attempts at violating the new rules does not result in a
significant network split, disrupting existing users of the system. If
we aren't going to take advantage of this, we should do a hard fork
instead, with the necessarily slow timescale that entails.

5) Follow the will of the community, irrespective of individuals or
unreasoned objection, but without ever overruling any reasonable
objection. Recent history also includes "objection" to soft forks in the
form of "this is bad because it doesn't fix a different problem I want
fixed ASAP". I don't think anyone would argue this qualifies as a
reasonable objection to a change, and we should be in a place, as a
community (never as developers or purely one group), to ignore such
objections and make forward progress in spite of them. We don't make
good engineering decisions by "bundling" unrelated features together to
enable political football and compromise.

I think BIP 9 (plus a well-crafted softfork) pretty effectively checks
the boxes for #2-4 here, and when done carefully with lots of community
engagement and measurement, can effectively fulfill #1 as well. #5 is,
as I'm sure everyone is aware, where it starts to fall down pretty hard.

BIP 8 has been proposed as an alternative, largely in response to issues
with #5. However, a naive deployment of it, rather obviously, completely
fails #1, #3, and #4, and, in my view, fails #5 as well by both giving
an impression of, setting a precedent of, and possibly even in practice
increasing the ability of developers to decide the consensus rules of
the system. A BIP 8 deployment that more accurately measures community
support as a prerequisite could arguably fulfill #1 and #5, though I'm
unaware of any concrete proposals on how to accomplish that. Arguably, a
significantly longer activation window could also allow BIP 8 to fulfill
#3 and #4, but only by exploiting the "needlessly" and "wherever
possible" loopholes.

You may note that, from the point of view of achieving the critical
goals here, BIP 8 is only different from a flag-day activation in that,
if it takes the "happy-path" of activating before the flag day, it looks
like BIP 9, but isn't guaranteed to. It additionally has the
"nice-to-have" property that activation can occur before the flag-day in
the case of faster miner adoption, though there is a limit of how fast
is useful due to node adoption.

Thus, to strike a balance between the drawbacks of BIP 8 and BIP 9, the
Great Consensus Cleanup softfork proposal included this text in the
discussion section (with the spec describing a BIP 9 deployment):

> In spite of some suggestion that other activation methods be used, BIP
> 9 is proposed as ensuring miners have upgraded to enforce new rules is
> an important part of minimizing disruption. While previous BIP 9 soft-
> forks have resulted in political contention, this comparatively-
> unimportant soft-fork provides a good opportunity to attempt to return
> to utilizing BIP 9 to ensure miner upgrade prior to activation, which
> the authors believe is a critical goal. However, if there is broad
> agreement to activate these rules when the BIP 9 expiry time is
> reached, and miners have not yet signaled sufficient level of
> readiness, a later flag-day activation may be merited. For this
> reason, implementations may wish to provide a compatibility option
> which allows flag-day enforcement of these rules without an update.

Ultimately, through admittedly rather limited discussion, I still like
this model (though I cannot claim it as my own, the original proposal
came from Greg Maxwell). BIP 9 only falls apart in case of unreasonable
objection, which, naturally, should carry a high bar to ignore, given we
have to have some level of agreement that it is, in fact, unreasonable
(or untargeted). While I admit this is a possibility, I both find it
less likely than in previous soft-forks, and even if it is the case, it
only slows down the process, it doesn't necessarily stop it. In the case
that it does fail, BIP 9 process, in fact, provides a good learning
opportunity as to the level of community readiness and desire for a
given change. While we can (and should, and are) learning a lot about
community readiness for, and acceptability of a change through outreach
and discussion, there is something about a change with a timeframe that
forces people to more carefully consider it.

Thus, as something a bit more concrete, I think an activation method
which sets the right precedent and appropriately considers the above
goals, would be:

1) a standard BIP 9 deployment with a one-year time horizon for
activation with 95% miner readiness,
2) in the case that no activation occurs within a year, a six month
quieting period during which the community can analyze and discussion
the reasons for no activation and,
3) in the case that it makes sense, a simple command-line/bitcoin.conf
parameter which was supported since the original deployment release
would enable users to opt into a BIP 8 deployment with a 24-month
time-horizon for flag-day activation (as well as a new Bitcoin Core
release enabling the flag universally).

This provides a very long time horizon for more standard activation,
while still ensuring the goals in #5 are met, even if, in those cases,
the time horizon needs to be significantly extended to meet the goals of
#3. Developing Bitcoin is not a race. If we have to, waiting 42 months
ensures we're not setting a negative precedent that we'll come to regret
as Bitcoin continues to grow.

Matt

Thanks also to AJ for feedback on an earlier version of this rant.