[bitcoin-dev] And Then What? Defining a Complete Process for Upgrades

[bitcoin-dev] And Then What? Defining a Complete Process for Upgrades

[Jeremy]

[-- Attachment #1.1: Type: text/plain, Size: 6830 bytes --]

This letter is particularly aimed at addressing Rusty Russell's quest for a
development process that respects all groups in a balance of powers.
However, in the spirit of open discussion, I'm sending it directly to the
list.

This proposal is aimed to be compatible with Taproot's ST, and I hope will
help us form some rough consensus around what we try next. Some of the
concepts here are synthesized from what I've seen discussed, but I haven't
included citations of anyone's specific ideas as I'm not sure of the exact
provenance -- I won't claim to have invented this per se, I'm trying to
capture the zeitgeist of what anyone might think to be the process if
pressed to draw it out. Lemme know how I did.

The specific parameters are up for debate, but I'm trying to make sure I've
captured the relevant state transitions.

In this diagram time flows left-to-right, and transitions happen at the
beginning, end, or middle of a block of time. It should be relatively clear
when things happen, but if not, please ask to clarify.

[image: Activation.png]

Clarifications:
- ST: Speedy trial, whereby > T% signals on a block activate the rule
- neg-ST: Speedy Trial, whereby >X% signals on a block Reject the rule
- neg-ST and ST at the same time on different bits: 11 or 00 are "abstain"
votes and are discarded. only 10 or 01 are counted. The purpose of
simultaneous bits is to allow both earlier lock in and to permit early
failure, rather than just one or the other.
- PoW Fork: If a new rule is active, but there is insufficient hashrate,
the rule must be abandoned or PoW must be changed to minimize disruption.
In order to minimize disruption, a node will consider an alternative PoW
chain if < 1/4 of the typical hash rate is seen for a day. Alternative PoW
is defined as SHA-256 10,000 layers, and starts at low difficulty. This is
selected to be maximally similar to Bitcoin's existing PoW, but
sufficiently different to obviate extant ASICS. A node will consider the
new PoW to be equal in value to the old PoW, and will select between the
two based on most-work. Work can be either within a single chain. The new
PoW should have a difficulty adjustment every day for the first month, at
which point, it will relax to every 2 weeks. The details of this should be
described in a separate BIP.
- PoW Fork Lockin: PoW fork is only *required* once the new rule is active.
Thus it's not required in the case of mandatory signalling to force the
signalling contemporaneously, but it can be used to commit to forking the
PoW at some time in the future. It may make sense to not activate the new
rule till the new PoW is active. The game theory of this should be studied
carefully, it is my opinion that the safest option is to PoW fork during
signalling as otherwise miners may protest progress at all.
- Changes: Any time the underlying activation proposal is changed, the
process is restarted. E.g., suppose taproot is rejected because Quantum
Scaries, and we hash the key. The process restarts from the beginning.
Restarts can only occur during quieting periods.
- Quieting Period 1: In the first quieting period, if reached, the "Bitcoin
Core Community" can release the next step, or change the BIP. I left out
failing in this period as a change or a redeployment should always be
attempted.
- Quieting Period 2: In the second quieting period, the outcome is either
to reject the change entirely or to agree to force it. The "Bitcoin Core
Community" may also prepare the release at this stage, and sign, but should
re-label the client as "Bitcoin Community's <Feature> Forcing Client".  A
release labeled "Bitcoin Core" may also be made without mandatory
signalling and without forced activation can also be made, such a client
should have (depending on if the flag day is to use signalling) either
ability to activate in response to signalling or a hidden
<feature>activeathash parameter to allow clients to enable the feature
post-hoc of the activating block.
- Forced Signalling: It's unclear to me the merit of forced signalling
being 90% of 2016 blocks v.s. 90% of 100 blocks. A shorter forced signaling
assuages certain concerns around lost hashrate -- 1 day of disruption is a
lot better than 2 weeks.
- Timeline: As spec'd above, this whole process takes about 2 years worst
case. ST1 0 months; Quieting 1 at 3 months; ST2 at 6 months; Quieting 2 at
9 months, final attempt at 1 year. The Forcing client period should
probably be 1 yr till active. This is *a bit* slower than the "BIP8
LOT=True UASF Client", but I think not so much slower that it's unworkable.

The most contentious part of this I intuit to be the PoW Fork -- please,
let's avoid discussing the mechanic of how to most safely accomplish this.
The main point of including it in this diagram is to emphasize that if you
commit to being on a minority chain with because of a rule activation with,
say, 5% hashrate, you would experience very tangible disruption. In theory,
every fork upgrade (even signalled) entails such a risk, but we assume some
level of miner honesty (unfortunately!) that they never signal falsely.
This may be a bad assumption with mandatory signalling. The alternative is
to permit hard forks in our diagram, and allow users to downgrade their
client and deactivate this rule. Since this can lead to loss of funds, we
do not consider this a safe option, and it is a hardfork as well so is
technically compatible with the "PoW fork" branch.


Questions I have:

1) What state transitions are missing from this diagram? Are there other
paths that should be included?
2) Is it ever feasible to make a change to the upgrade and not restart the
whole process?
3) How long should all of the periods be? Can the 1st 2 ST's be 3 month?
Should we make the second one 6 month? Does it depend on previous signalled
hashrate?
4) Do we ever adjust signalling thresholds?
5) Does forced signalling need to be 2016 blocks?
6) In the second ST should the min active height be allowed to be within
signalling time if > 3 mo?
7) Under what circumstances would we *want* to skip the second ST period
and directly signal? What would we lose by committing to not skipping it,
ever (6 months?).
8) I purposefully left the purple edge from ST2 bit 2 to Quieting 2: in
theory, this edge is not there because it is overruled by the neg-ST
failing to fail. Under what circumstances might we give this precedence
over neg-ST? E.g., signalling activate < 50%?
9) How much parameter flexibility do we have during Quieting periods?
Should we be fixed beforehand?
10) who wants to write the software for any of this... *noses*
11) do we need to hard-code the PoW fork ahead of time? Or can it just be
"prepared" as an alt binary in case of emergency?



Best,

Jeremy



--
@JeremyRubin <https://twitter.com/JeremyRubin>
<https://twitter.com/JeremyRubin>

[-- Attachment #1.2: Type: text/html, Size: 12497 bytes --]

[-- Attachment #2: Activation.png --]
[-- Type: image/png, Size: 81125 bytes --]

Re: [bitcoin-dev] And Then What? Defining a Complete Process for Upgrades

[Keagan McClelland]

[-- Attachment #1.1: Type: text/plain, Size: 15001 bytes --]

Hi Jeremy,

First off, thank you for putting this together and trying to organize
discussion around a repeatable process for soft-fork upgrades. As I read
this I am struggling to identify the complete set of problems that this
process is attempting to solve for. My best understanding of the gridlock
that happened around the LOT parameter of BIP 8 is the fact that a flag day
finale of the deployment had the *optics* of being developer controlled,
despite the fact that upgrades are voluntary and opt in since there is no
automatic upgrade mechanism within Bitcoin Core. So I am generally
unconvinced by that argument. The alternative (the advancement of various
LOT=false proposals including ST) is that we are letting the miners
*actually* exert a level of control over the protocol by imbuing them with
the power to veto these proposals. At a 10% signalling threshold needed to
block the upgrade, in addition to the fact that mining is an enterprise
engaged in by a comparatively small portion of the user base, the
constituency required to block proposals under this model is remarkably
small. I don't think anyone is satisfied with the idea of indefinitely
letting miners have veto power over proposals. So the question is, "what
then?".

The particular gripe I have with the situation is that I generally believe
that *any* LOT=false deployment that is ultimately upheld (not reattempted)
is a ceding of control of the protocol to the miners, at the very least in
that instance if not more generally due to the possible emboldening of more
non-cooperative actors for personal gain at the expense of other network
participants. On the other hand, any LOT=false deployment that is
immediately followed up with something approximating a LOT=true/flag
day/UASF deployment is ultimately a charade and can be handily replaced
with LOT=true with an extended timeline. The common rebuttal I hear from
this is that a LOT=false deployment "helps us learn something" and is
usually an appeal to the difference between stated and revealed preference.
However, I have not been able to get past the idea of exactly what we plan
to learn from this? If we do not trust stated preferences for protocol
changes, and we *only* trust revealed preference, then it seems like the
only thing we can learn is that "miners will not voluntarily make this
upgrade" and any arguments about why are ultimately as speculative as the
stated preferences prior to any Soft Fork deployment at all. As such, I
don't think we learn basically anything useful.

Against the above backdrop, it is hard for me to think that a LOT=false
policy is *ever* sensible. In this case, I have advocated for ST because I
think it will either activate quickly, or prove out in general that a
LOT=false policy just doesn't work (quickly). In all of the discussions I
have been party to, I have not once witnessed someone who would accept the
final upholding of a miner veto on Taproot in particular, yet in the same
session, many of those participants are advocating for LOT=false anyway.
This seems incoherent to me.

So what are we actually solving for?

The idea as I understand it is to distribute power among Bitcoin's
different constituencies in such a way that we A. block "bad upgrades"
always, and B. do our best to activate "good upgrades". However, I feel it
necessary to caution against the fetishization of triumvirate governance
for its own sake. In my mind, anything that doesn't generally respect the
notion that the users are in control of the protocol is not too dissimilar
from the decentralization theater that some Bitcoiners criticize various
altcoins for.

So why not just UASF from the getgo?

This I have started to understand more clearly but still get hung up on
certain details. I think the only universal claim about this is "avoid deep
chain splits at all costs" where "deep" is anything that is going to ruin
upper layers of Bitcoin including institutional operations practices. For
as long as I've been developing applications on top of Bitcoin the
"generally accepted default" seems to be that 6 block reorgs are the
"untouchable limit". However, this is where the details become really
important. By definition, blocks conforming to soft-fork rules are accepted
by the pre-fork implementations. In the other direction, there exists a
subset of blocks that can be produced by pre-fork miners that will not be
accepted by the soft-fork rules. So, in this case, the risk incurred by the
SF clients is that the chain halts. The risk incurred by the pre-fork
implementations is that there are reorgs that result in economic losses.
However, in order for the reorgs to actually occur, miners have to
persistently mine invalid blocks. The reason I make the distinction about
"persistently" is that pre-fork implementations *will accept* soft-fork
compliant blocks. This means that the same incentive structures that cause
miners to switch to mining atop new blocks instead of continuing to mine
atop older ones, apply just as much in this case. Chain convergence is
*guaranteed* with as little as 51% (though this is eventual, and may not be
an acceptable threshold if you're trying to keep the probability of deep
reorgs to some astronomically low threshold). Furthermore, chain
convergence is probable with far less than that threshold, though I'm less
sure of this claim and am open to a rebuttal on this.

The only wrench in the above line of reasoning is the issue of forced
signaling. I've never been a fan of forced signaling. The only thing it
guards against is apathy, which I think is solved with even the threat of
having your mined blocks being reorged out by including invalid
transactions in them.

So to summarize my understanding of what's going on here: we don't want
MASF due to concentration of power, we don't want UASF because of the risk
of chain split. The only way to reconcile this is to show miners that users
more strongly value work done on a chain enforcing the soft fork rules than
one that doesn't, ideally with costly signaling. This is the only way users
can effectively exercise an override on miner preference in a backwards
compatible way and substantially reduces or eliminates chain split risk. I
don't think the proposal above does anything to alleviate this issue. In
fact, if I read it correctly, it rules out the possibility of any UASF
altogether by making it such that a failed MASF forces the users to
hard-fork the protocol. I think that this would be rather tragic since
avoidance of hard forks is one of the biggest victories that came out of
2017. I apologize for not addressing the proposal in a more direct way and
engaging with the details, however, I don't think that discussing the
process makes any sense until we can agree what is being solved, and what
risks are flat out unacceptable. Perhaps laying those out in a
followup response can make it easier for me to engage with the actual
process.

In my mind any proposal that doesn't include a mechanism for gathering
reliable information about user-preference in a way that is sybil resistant
doesn't make any meaningful progress on a consensus change process. I have
only two ideas in this area and both of them don't seem fantastic. The
first one, fork futures, has been discussed at length by people more
experienced than I. The second, is a mechanism that makes mining
transactions that express fork preference (in an as-yet undefined way)
contingent upon the block signaling for it. The details of how to pull that
off may make it such that it is DOA, but it would at least give users a way
to broadcast a costly signal of their preference.

Keagan

On Thu, Apr 22, 2021 at 4:57 PM Jeremy via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> This letter is particularly aimed at addressing Rusty Russell's quest for
> a development process that respects all groups in a balance of powers.
> However, in the spirit of open discussion, I'm sending it directly to the
> list.
>
> This proposal is aimed to be compatible with Taproot's ST, and I hope will
> help us form some rough consensus around what we try next. Some of the
> concepts here are synthesized from what I've seen discussed, but I haven't
> included citations of anyone's specific ideas as I'm not sure of the exact
> provenance -- I won't claim to have invented this per se, I'm trying to
> capture the zeitgeist of what anyone might think to be the process if
> pressed to draw it out. Lemme know how I did.
>
> The specific parameters are up for debate, but I'm trying to make sure
> I've captured the relevant state transitions.
>
> In this diagram time flows left-to-right, and transitions happen at the
> beginning, end, or middle of a block of time. It should be relatively clear
> when things happen, but if not, please ask to clarify.
>
> [image: Activation.png]
>
> Clarifications:
> - ST: Speedy trial, whereby > T% signals on a block activate the rule
> - neg-ST: Speedy Trial, whereby >X% signals on a block Reject the rule
> - neg-ST and ST at the same time on different bits: 11 or 00 are "abstain"
> votes and are discarded. only 10 or 01 are counted. The purpose of
> simultaneous bits is to allow both earlier lock in and to permit early
> failure, rather than just one or the other.
> - PoW Fork: If a new rule is active, but there is insufficient hashrate,
> the rule must be abandoned or PoW must be changed to minimize disruption.
> In order to minimize disruption, a node will consider an alternative PoW
> chain if < 1/4 of the typical hash rate is seen for a day. Alternative PoW
> is defined as SHA-256 10,000 layers, and starts at low difficulty. This is
> selected to be maximally similar to Bitcoin's existing PoW, but
> sufficiently different to obviate extant ASICS. A node will consider the
> new PoW to be equal in value to the old PoW, and will select between the
> two based on most-work. Work can be either within a single chain. The new
> PoW should have a difficulty adjustment every day for the first month, at
> which point, it will relax to every 2 weeks. The details of this should be
> described in a separate BIP.
> - PoW Fork Lockin: PoW fork is only *required* once the new rule is
> active. Thus it's not required in the case of mandatory signalling to force
> the signalling contemporaneously, but it can be used to commit to forking
> the PoW at some time in the future. It may make sense to not activate the
> new rule till the new PoW is active. The game theory of this should be
> studied carefully, it is my opinion that the safest option is to PoW fork
> during signalling as otherwise miners may protest progress at all.
> - Changes: Any time the underlying activation proposal is changed, the
> process is restarted. E.g., suppose taproot is rejected because Quantum
> Scaries, and we hash the key. The process restarts from the beginning.
> Restarts can only occur during quieting periods.
> - Quieting Period 1: In the first quieting period, if reached, the
> "Bitcoin Core Community" can release the next step, or change the BIP. I
> left out failing in this period as a change or a redeployment should always
> be attempted.
> - Quieting Period 2: In the second quieting period, the outcome is either
> to reject the change entirely or to agree to force it. The "Bitcoin Core
> Community" may also prepare the release at this stage, and sign, but should
> re-label the client as "Bitcoin Community's <Feature> Forcing Client".  A
> release labeled "Bitcoin Core" may also be made without mandatory
> signalling and without forced activation can also be made, such a client
> should have (depending on if the flag day is to use signalling) either
> ability to activate in response to signalling or a hidden
> <feature>activeathash parameter to allow clients to enable the feature
> post-hoc of the activating block.
> - Forced Signalling: It's unclear to me the merit of forced signalling
> being 90% of 2016 blocks v.s. 90% of 100 blocks. A shorter forced signaling
> assuages certain concerns around lost hashrate -- 1 day of disruption is a
> lot better than 2 weeks.
> - Timeline: As spec'd above, this whole process takes about 2 years worst
> case. ST1 0 months; Quieting 1 at 3 months; ST2 at 6 months; Quieting 2 at
> 9 months, final attempt at 1 year. The Forcing client period should
> probably be 1 yr till active. This is *a bit* slower than the "BIP8
> LOT=True UASF Client", but I think not so much slower that it's unworkable.
>
> The most contentious part of this I intuit to be the PoW Fork -- please,
> let's avoid discussing the mechanic of how to most safely accomplish this.
> The main point of including it in this diagram is to emphasize that if you
> commit to being on a minority chain with because of a rule activation with,
> say, 5% hashrate, you would experience very tangible disruption. In theory,
> every fork upgrade (even signalled) entails such a risk, but we assume some
> level of miner honesty (unfortunately!) that they never signal falsely.
> This may be a bad assumption with mandatory signalling. The alternative is
> to permit hard forks in our diagram, and allow users to downgrade their
> client and deactivate this rule. Since this can lead to loss of funds, we
> do not consider this a safe option, and it is a hardfork as well so is
> technically compatible with the "PoW fork" branch.
>
>
> Questions I have:
>
> 1) What state transitions are missing from this diagram? Are there other
> paths that should be included?
> 2) Is it ever feasible to make a change to the upgrade and not restart the
> whole process?
> 3) How long should all of the periods be? Can the 1st 2 ST's be 3 month?
> Should we make the second one 6 month? Does it depend on previous signalled
> hashrate?
> 4) Do we ever adjust signalling thresholds?
> 5) Does forced signalling need to be 2016 blocks?
> 6) In the second ST should the min active height be allowed to be within
> signalling time if > 3 mo?
> 7) Under what circumstances would we *want* to skip the second ST period
> and directly signal? What would we lose by committing to not skipping it,
> ever (6 months?).
> 8) I purposefully left the purple edge from ST2 bit 2 to Quieting 2: in
> theory, this edge is not there because it is overruled by the neg-ST
> failing to fail. Under what circumstances might we give this precedence
> over neg-ST? E.g., signalling activate < 50%?
> 9) How much parameter flexibility do we have during Quieting periods?
> Should we be fixed beforehand?
> 10) who wants to write the software for any of this... *noses*
> 11) do we need to hard-code the PoW fork ahead of time? Or can it just be
> "prepared" as an alt binary in case of emergency?
>
>
>
> Best,
>
> Jeremy
>
>
>
> --
> @JeremyRubin <https://twitter.com/JeremyRubin>
> <https://twitter.com/JeremyRubin>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #1.2: Type: text/html, Size: 21443 bytes --]

[-- Attachment #2: Activation.png --]
[-- Type: image/png, Size: 81125 bytes --]