Re: [Bitcoin-development] questions about bitcoin-XT code fork & non-consensus hard-fork

[Adam Back <adam@cypherspace.org>]

Hi Mike

Well thank you for replying openly on this topic, its helpful.

I apologise in advance if this gets quite to the point and at times
blunt, but transparency is important, and we owe it to the users who
see Bitcoin as the start of a new future and the$3b of invested funds
and $600m of VC funds invested in companies, we owe it to them that we
be open and transparent here.

I would really prefer on a personal nor professional basis to be
having this conversation period, never mind in public, but Mike - your
and Gavin's decision to promote a unilateral hard-fork and code fork
are extremely high risk for bitcoin and so there remains little
choice.  So I apologise again that we have to have this kind of
conversation on a technical discussion list.  This whole thing is
hugely stressful and worrying for developers, companies and investors.

I strongly urge that we return to the existing collaborative
constructive review process that has been used for the last 4 years
which is a consensus by design to prevent one rogue person from
inserting a backdoor, or lobbying for a favoured change on behalf of a
special interest group, or working for bad actor (without accusing you
of any of those - I understand you personally just want to scale
bitcoin, but are inclined to knock heads and try to force an issue you
see, rather than work collaboratively).

For you (and everyone)

- Should there be a summit of some kind, that is open attendance, and
video recorded so that people who are unable to attend can participate
too, so that people can present the technical proposals and risks in
an unbiased way?

(It is not theoretical question, I may have a sponsor and host - not
Blockstream, an independent, its a question for everyone, developers,
users, CTOs, CEOs.)



So here I come back to more frank questions:

Governance

The rest of the developers are wise to realise that they do not want
exclusive control, to avoid governance centralising into the hands of
one person, and this is why they have shared it with a consensus
process over the last 4 years.  No offence but I dont think you
personally are thinking far enough ahead to think you want personal
control of this industry.  Maybe some factions dont trust your
motives, or they dont mind, but feel more assured if a dozen other
people are closely reviewing and have collective review authority.

- Do you understand that attempting to break this process by
unilateral hard-fork is extremely weakening of Bitcoin's change
governance model?

- Do you understand that change governance is important, and that it
is important that there be multiple reviewers and sign-off to avoid
someone being blackmailed or influenced by an external party - which
could potentially result in massive theft of funds if something were
missed?

- Secondarily do you understand that even if you succeed in a
unilateral fork (and the level of lost coins and market cap and damage
to confidence is recoverable), that it sets a precedent that others
may try to follow in the future to introduce coercive features that
break the assurances of bitcoin, like fungibility reducing features
say (topically I hear you once proposed on a private forum the concept
of red-lists, other such proposals have been made and quickly
abandoned), or ultimately if there is a political process to obtain
unpopular changes by unilateral threat, the sky is the limit - rewrite
the social contract at that point without consensus, but by
calculation that people will value Bitcoin enough that they will
follow a lead to avoid risk to the system?


Security

As you probably know some extremely subtle bugs in Bitcoin have at
times slipped past even the most rigorous testings, often with
innocuous but unexpected behaviours, but some security issues  Some
extremely intricate and time-sensitive security defect and incident
response happens from time to time which is not necessarily publicly
disclosed until after the issue has been rolled out and fixed, which
can take some time due to the nature of protocol upgrades,
work-arounds, software upgrade via contacting key miners etc.  We
could take an example of the openSSL bug.

- How do you plan to deal with security & incident response for the
duration you describe where you will have control while you are
deploying the unilateral hard-fork and being in sole maintainership
control?

- Are you a member of the bitcoin security reporting list?

On 15 June 2015 at 11:56, Mike Hearn <mike@plan99.net> wrote:
> I will review both and mostly delegate to Gavin's good taste around the
> details, unless there is some very strong disagreement. But that seems
> unlikely.
> ...
> Feedback will be read. There are no NACKS in Bitcoin XT. Patch requests
> aren't scored in any way. The final decision rests with the maintainer as in
> ~all open source projects.

As you know the people who have written 95% of the code (and reviewed,
and tested, and formally proved segments etc) are strenuously advising
not to push any consensus code into public use without listening to
and addressing review questions which span beyond rigorous code &
automated guided fuzz testers, simulation and sometimes formal proofs,
but also economics, game-theory and critically very subtle
determinism/consensus safety that they have collectively 4-5 years
experience of each.

- Will you pause your release plans if all of the other developers
insist that the code or algorithm is defective?

- Please don't take this the wrong way, and I know your bitcoinj work
was a significant engineering project which required porting bitcoin
logic.  But If the answer to the above question is no, as you seemed
to indicate in your response, as you not have not written much bitcoin
core code yourself (I think 3 PRs in total), do you find yourself more
qualified than the combination of peer review of the group of people
who have written 95% of it, and maintained it and refactored most of
it over the last 4-5 years?

I presume from your security background you are quite familiar with
the need for review of crypto protocol changes & rigorous code review.
That is even more the case with Bitcoin given the consensus
criticality.

>> - On the idea of a non-consensus hard-fork at all, I think we can
>> assume you will get a row of NACKs.  Can you explain your rationale
>> for going ahead anyway?  The risks are well understood and enormous.
>
> If Bitcoin runs out of capacity it will break and many of our users will
> leave. That is not an acceptable outcome for myself or the many other
> wallet, service and merchant developers who have worked for years to build
> an ecosystem around this protocol.

That you are frustrated, is not a sufficient answer as to why you are
proposing to go ahead with a universally acknowledged extreme network
divergence danger unilateral hard-fork, lacking wide-spread consensus.
People are quite concerned about this.  Patience, caution and prudence
is necessary in a software system with such high assurance
requirements.

So I ask again:

- On the idea of a non-consensus hard-fork at all, I think we can
assume you will get a row of NACKs.  Can you explain your rationale
for going ahead anyway?  The risks are well understood and enormous.

Note the key point is that you are working on a unilateral hard-fork,
where there is a clear 4 year established process for proposing
improvements and an extremely well thought out and important change
management governance process.  While there has been much discussion,
you nor Gavin, have not actually posted a BIP for review.  Nor
actually was much of the discussion even conducted in the open: it was
only when Matt felt the need to clear the air and steer this
conversation into the open that discussion arose here.  During that
period of private discussion you and Gavin were largely unknown to
most of us lobbying companies with your representation of a method
that concerns everyone of the Bitcoin users.  Now that the technical
community aware aware they are strenuously discouraging you on the
basis of risks.


Openness

- Do you agree that bitcoin technical discussions should happen in the open?

- As this is a FOSS project, do you agree that companies should also
be open, about their requirements and trade-offs they would prefer?

- Can you disclose the list of companies you have lobbied in private
whether they have spoken publicly or not, and whether they have
indicated approval or not?

- Did you share a specific plan, like a BIP or white paper with these
companies, and if so can we see it?

- If you didnt submit a plan, could you summarise what you asked them
and what you proposed, and if you discussed also the risks?  (If you
asked them if they would like Bitcoin to scale, I expect almost
everyone does, including every member of the technical community, so
that for example would not fairly indicate approval for a unilateral
hard-fork)

I and others will be happy to talk with the CTO and CEOs of companies
you have lobbied in private, for balance to assure ourselves and the
rest of the community that their support was given - and with full
understanding of the risks of doing it unilaterally, without peer
review, benefit of maintenance and security inidence management, and
what exactly they are being quoting as having signed up for.

(This maybe more efficiently and openly achieved by the open process,
on a mailing list, maybe a different one even special purpose to this
topic, with additional option of the open public meeting I proposed at
the top).

- Do you agree that it would be appropriate, that companies be aware
of both the scaling opportunities (of course, great everyone wants
scalability) as well as the technical limits and risks with various
approaches?  And that these be presented by parties from a range of
views to ensure balance?

- Do you consider your expression of issues to hold true to the ideal
of representing balanced nuanced view of all sides of a technical
debate, even when under pressure or feeling impatient about the
process?

You may want to review the opening few minutes of your epicenter 82
bitcoin for example where you claimed and I quote "[the rest of the
technical community] dont want capacity to ever increase and want it
to stay where it is and when it fills up people move to other
systems".

- Do you think that is an accurate depiction of the complex trade-offs
we have been discussing on this list?

(For the record I am not aware of a single person who has said they do
not agree with scaling Bitcoin.  Changing a constant is not the
hard-part.  The hard part is validating a plan and the other factors
that go into it.  It's not a free choice it is a security/scalability
tradeoff.  No one will thank us if we "scale" bitcoin but break it in
hard to recover ways at the same time.)

- Were you similarly balanced in your explanations when talking to
companies in private discussions?

- Do you understand that if we do not work from balanced technical
discussion, that we may end up with some biased criteria?

Authority

Neither you nor Gavin have any particular authority here to speak on
behalf of Bitcoin (eg you acknowledge in your podcast that Wladimir is
dev lead, and you and Gavin are both well aware of the 4 year
established change management consensus decision making model where
all of the technical reviewers have to come to agreement before
changes go in for security reasons explained above).  I know Gavin has
a "Chief Scientist" title from the Bitcoin Foundation, but sadly that
organisation is not held in as much regard as it once was, due to
various irregularities and controversies, and as I understand it no
longer employs any developers, due to lack of funds.  Gavin is now
employed by MIT's DCI project as a researcher in some capacity.  As
you know Wladimir is doing the development lead role now, and it seems
part of your personal frustration you said was because he did not
agree with your views.  Neither you nor Gavin have been particularly
involved in bitcoin lately, even Gavin, for 1.5 years or so.

- Do you agree that if you presume to speak where you do not have
authority you may confuse companies?

> If Bitcoin runs out of capacity it will break and many of our users will
> leave. That is not an acceptable outcome for myself or the many other
> wallet, service and merchant developers who have worked for years to build
> an ecosystem around this protocol.

But I think this is a false dichotomy.  As I said in previous mail I
understand people are frustrated that it has taken so long, but it is
not the case that no progress has been made on scalability.

I itemised a long list of scalability work which you acknowledged as
impressive work (CPU, memory, network bandwidth/latency) and RBF, CPFP
fee work, fee-estimation, and so on, which you acknowledged and are
aware of.

There are multiple proposals and BIPs under consideration on the list right now.

- what is the reason that you (or Gavin) would not post your BIP along
side the others to see if it would win based on technical merit?

- why would you feel uniquely qualified to override the expert opinion
of the rest of the technical community if your proposal were not
considered to have most technical merit? (Given that this is not a
simple market competition thing where multiple hard-forks can be
considered - it is a one only decision, and if it is done in a
divisive unilateral way there are extreme risks of the ledger
diverging.)

Network Divergence Risk

>> - How do you propose to deal with the extra risks that come from
>> non-consensus hard-forks?  Hard-forks themselves are quite risky, but
>> non-consensus ones are extremely dangerous for consensus.
>
> The approach is the same for other forks. Voting via block versions and then
> when there's been >X% for Y time units the 1mb limit is lifted/replaced.

But this is not a soft-fork, it is a hard-fork.  Miner voting is only
peripherally related.  Even if in the extremis 75% of miners tried a
unilateral hard-fork but 100% of the users stayed on the maintained
original code, no change would occur other than those miners losing
reward (mining fork-coins with no resale value) and the difficulty
would adjust.  The miners who made an error in choice would lose money
and go out of business or rejoin the chain.

However if something in that direction happens with actual users and
companies on both sides of it users will lose money, the ledger will
diverge as soon as a single double-spend happens, and never share a
block again, companies will go instantly insolvent, and chaos will
break out.  This is the dangerous scenario we are concerned about.

So the same question again:

- How do you propose to deal with the extra risks that come from
non-consensus hard-forks?  Hard-forks themselves are quite risky, but
non-consensus ones are extremely dangerous for consensus.


Being sensitive to alarming the market

It is something akin to Greece or Portugal or Italy exiting the euro
currency in a disorderly way.  Economists and central bank policy
makers are extremely worried about such an eventuality and talk about
related factors in careful, measured terms, watch Mario Draghi when he
speaks.

Imagine that bitcoin is 10x or 100x bigger.  Bitcoin cant have people
taking unilateral actions such as you have been proposing.  It is not
following the consensus governance process, and not good policy and it
is probably affecting bitcoin confidence and price at this moment.

>> - Do you have contingency plans for what to do if the non-consensus
>> hard-fork goes wrong and $3B is lost as a result?
>
> Where did you get the $3B figure from? The fork either doesn't happen, or it
> happens after quite a long period of people knowing it's going to happen -
> for example because their full node is printing "You need to upgrade"
> messages due to seeing the larger block version, or because they read the
> news, or because they heard about it via some other mechanisms.

This is not a soft-fork, and the community will not want to take the
risks once they understand them, and they have months in which to
understand them and at this point you've motivated and wasted 100s of
developer man hours such that we will feel impelled to make sure that
no one opts into a unilateral hard-fork without understanding the
risks.  It would be negligent to allow people to do that.  Before this
gets very far FAQs will be on bitcoin.org etc explaining this risk I
would imagine.  Its just starting not finished.

What makes you think the rest of the community may not instead prefer
Jeff Garzik's BIP after revisions that he is making now with review
comments from others?

Or another proposal.  Taken together with a deployment plan that sees
work on decentralisation tying into that plan.

- If you persisted anyway, what makes you think bitcoin could not make
code changes defensively relating to your unilateral fork?
(I am sure creative minds can find some ways to harden bitcoin against
a unilateral fork, with a soft-fork or non-consensus update can be
deployed much faster than a hard-fork).

I tried to warn Gavin privately that I thought he was under-estimating
the risk of failure to his fork proposal due to it being unilateral.
Ie as you both seem sincere in your wish to have your proposal
succeed, then obviously the best way to do that is to release a BIP in
the open collaborative process and submit it to review like everyone
else.  Doing it unilaterally only increases its chance of failure.

The only sensible thing to do here is submit a BIP and stop the
unilateral fork threat.

Scalability Plans

> Let me flip the question around. Do you have a contingency plan if Bitcoin
> runs out of capacity and significant user disruption occurs that results in
> exodus, followed by fall in BTC price? The only one I've seen is "we can
> perform an emergency hard fork in a few weeks"!

Yes people have proposed other plans.  Bryan Bishop posted a list of them.

Jeff Garzik has a proposal, BIP-100 which seems already better than
Gavin's having benefit of peer review which he has been incorporating.

I proposed several soft-fork models which can be deployed safely and
immediately, which do not have ledger risk.

I have another proposal relating to simplified soft-fork one-way pegs
which I'll write up in a bit.

I think there are still issues in Jeff's proposal but he is very open
and collaborating and there maybe related but different proposals
presently.

>> As you can probably tell I think a unilateral fork without wide-scale
>> consensus from the technical and business communities is a deeply
>> inadvisable.
>
> Gavin and I have been polling many key players in the ecosystem. The
> consensus you seek does exist. All wallet developers (except Lawrence), all
> the major exchanges, all the major payment processors and many of the major
> mining pools want to see the limit lifted (I haven't been talking to pools,
> Gavin has).

It does not seem to me that you understand the issue.  Of course they
want to increase the scalability of bitcoin.  So does everyone else on
this mailing list.

That they would support that is obvious.  If you presented your
unilateral action plan without explaining the risks too.

I think I covered this further above.  If you would like to share the
company list, or we can invite them to the proposed public physical
meeting, I think it would be useful for them to have a balanced view
of the ledger divergence risks, and alternative in-consensus proposals
underway, as well as the governance risks, maintenance risks, security
incident risks.

Note that other people talk to companies too, as part of their day to
day jobs, or from contacts from being in the industry.  You have no
special authority or unique ability to talk with business people.  Its
just that the technical community did not know you were busy doing
that.

I can not believe that any company that would listen to their CTO, CSO
or failing that board would be ok with the risks implied by what you
are proposing on full examination.

> This notion that the change has no consensus is based on you polling the
> people directly around you and people who like to spend all day on this
> mailing list. It's not an accurate reflection of the wider Bitcoin community
> and that is one of the leading reasons there is going to be a fork. A small
> number of people have been flatly ignoring LOTS of highly technical and
> passionate developers who have written vast amounts of code, built up the
> Bitcoin user base, designed hardware and software, and yes built companies.

I know you want scale bitcoin, as I said everyone here does. I think
what you're experiencing is that you've had more luck explaining your
pragmatic unilateral plan to non-technical people without peer review,
and so not experienced the kind of huge pushback you are getting from
the technical community.  The whole of bitcoin is immensely
complicated such that it takes an uber-geek CS genius years to
catchup, this is not a slight of any of the business people who are
working hard to deploy Bitcoin into the world, its just complicated
and therefore not easy to understand the game-theory, security,
governance and distributed system thinking.  I have a comp sci PhD in
distributed systems, implemented p2p network systems and have 2
decades of applied crypto experience with a major interest in
electronic cash crypto protocols, and it took me a several years to
catchup and even I have a few hazy spots on low-level details, and I
addictively into read everything I could find.  Realistically all of
us are still learning, as bitcoin combines so many fields that it
opens new possibilities.

What I am expecting that yourself and Gavin are thinking is that
you'll knock heads and force the issue and get to consensus.

However I think you have seriously misjudged the risks and have not
adequately explained them to companies you are talking with.  Indeed
you do not fully seem to acknowledge the risks, nor to have a well
thought out plan here of how you would actually manage it, nor the
moral hazards of having a lone developer in hugely divisive
circumstances in sole control of bitcoins running code.  Those are
exactly the reasons for the code change governance process!

Even though you are trying to help, the full result is you are not
helping achieve anything by changing a constant and starting a
unilateral hard-fork (not to trivialise the work of making a patch to
do that).

The work to even make the constant change be feasible was a result of
1000s of hours of work by others in the development community, that is
emphatically and unilaterally telling you that hard-forks are hugely
inadvisable.

You are trying to break the code change governance security procedure
that were put in place for good reason for the security of $3b of
other peoples money, even if you have a pragmatic intent to help, this
is flat out unacceptable.

There are also security implications to what you are proposing, which
I have heard you attempting to trivialise, that are core to Bitcoins
security and core functionality.

>  the overwhelming impression I get from a few
> others here is that no, they don't want to scale Bitcoin. They already
> decided it's a technological dead end.

I think this is a significant mischaracterisation, and I think almost
everybody is on board with a combination plan:

1. work to improve decentralisation (specific technical work already
underway, and education)
2. create a plan to increase block-size in a slow fashion to not cause
system shocks (eg like Jeff is proposing or some better variant)
3. work on actual algorithmic scaling

In this way we can have throughput needed for scalability and security
work to continue.

As I said you can not scale a O(n^2) broadcast network by changing
constants, you need algorithmic improvements.

People are working on them already.  All of those 3 things are being
actively worked on RIGHT NOW, and in the case of algorithmic scaling
and improve decentralisation have been worked on for months.

You may have done one useful thing which is to remind people that
blocks are only 3x-4x below capacity such that we should look at it.

But we can not work under duress of haste, nor unilateral ultimatums,
this is the realm of human action that leads to moral hazard, and
ironically reminds us of why Satoshi put the quote in the genesis
block.

Bitcoin is too complex a system with too much at stake to be making
political hasty decisions, it would be negligent to act in such a way.

Again please consider that you did your job, caused people to pay
attention, but return to the process, submit a BIP, retract the
unilateral hard-fork which is so dangerous and lets have things be
calm, civil and collaborative in the technical zone of Bitcoin and not
further alarm companies and investors.

Adam