Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gregory Maxwell <gmaxwell@gmail.com>]

On Wed, Sep 30, 2015 at 9:01 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> I coined the term SPV so I know exactly what it means, and bitcoinj

The term comes from the Bitcoin whitepaper.

> On the other hand, full nodes all claim they run scripts. Users expect that
> and may be relying on it. The unstated assumption here is that the nodes run
> them correctly. A soft fork breaks this assumption.

They run them correctly with respect to the rules that they implement,
nothing about a soft-fork changes that.

The system could have been designed in a way that wasn't full of nice
compatibility features. The history of bitcoin could have been that
past improvements were all performed with hard forks instead of soft
forks.

But neither of these things are true. (And I think it's very likely
that there would have been fewer improvements if that were the case).

> I'm going to ignore the rest of the stuff you wrote  about "design decisions
> to lack security" or "cheaply avoidable lack of validation". When you have
> sat down and written an SPV implementation by yourself, then shipped it to a
> couple of million users, you might have better insight into basic
> engineering costs.

At the end of the day we need to deliver software to our users that
delivers on their needs and doesn't undermine their privacy or
security;  even if its really hard.  So when someone calls out
something that I'm working on that could use improvement, my response
isn't to tell them how much I'm not going to listen to them because
I've accomplished some long list of things that they haven't; that
software I've written runs on hundreds of millions of devices; ...
rather my response is to hear out their concerns, even when due to
extensive context I'm confident that they are probably confused or
dishonestly motivated; because there is always a potential to learn,
and always a potential to do better.  I have found this to be pretty
productive, as even when both parties walk away with the same
positions they started with, I usually learn something along the way
just because I paid attention.

> Until then, I find your criticisms of code you think was
> missing due to "stonewalling" and so on to be seriously lacking real world
> experience.

BIP16 was published on 2012-01-01. Enforcement on the network began on
April 1st 2012.

Support for merely sending to P2SH addresses was merged into BitcoinJ,
Nov 30th _2013_, after it was written by Mike Belshe.

In the interim you spent considerable time arguing against
implementing it, e.g. in one example incident:

--- Day changed Thu Sep 12 2013
10:03 < TD> heck, if a recipient really really wants to receive a p2sh
payment for some reason, they can just put the p2sh output into the
payment request message
....
10:17 <@gmaxwell> TD: In any case, P2SH has been deployed for
something like two years now. Your arguement seems to be basically we
should be creating a false tying between payment protocol messages and
things like escrow usage in order to coerce people to adopt the
payment protocol in places where an address would do.
10:17 <@gmaxwell> I think thats a bit cruddy.
10:19 < TD> no, i'm saying p2sh is a feature that just isn't usable. i
did point this out at the time it was merged - gavin believed that
apps to do complicated multi-device wallets would appear before a
payment protocol did, and people wouldn't like the look of long
addresses. that was pretty much the rationale given. that didn't
happen, obviously
10:20 < TD> developing features that are used only by bitcoind
developers, isn't the right way to go, and p2sh definitely falls into
that category
...
10:35 < TD> i'm not going to do it myself because anyone who is
capable of producing and running something that uses p2sh is capable
of working with the payment protocol as well, and that gives a better
user experience overall. but if someone else wants to, go for it.

I don't think there is anything wrong about calling this stonewalling.
At least thats how it came across to myself and others. I'm sorry if I
judged too harshly there.

To be clear, by pointing out your past opposition and non-deployment
in this message I am _not_ trying to attacking you for failing to
support P2SH.

I pointing out that right or wrong..... That you actively argued
against it. That you chose not to implement it, and only accepted a
patch for it a year and a half later. From your own words it seems
clear that you didn't implement it due to actual opposition, but even
if the non-implementation was simply engineering priorities, the fact
remains that you didn't implement for a very long time.

And that is _okay_, we still got it anyways, and today tens of
thousands of transactions per day use it and P2SH secures about 10% of
all Bitcoin value.  This is possible because with a soft fork users
using other software can gain functionality which might be critical to
them (As Jgarzik was saying about Bitpay in the discussion I was
quoting from) that you don't have the time or interest to implement in
your own software.


> What? This is nonsensical. P2SH was added to the full verification code
> quite quickly,

Yes, Matt Corallo added it to code which by your admission no one was
using.  I agree this is not relevant.

> resolve. Like I said - write the amount of code I've written, unpaid in your
> evenings and weekends, and then you can criticise bitcoinj for lacking
> features.

I think it's likely that I've spent significant more time unpaid on my
evenings and weekends creating software for others than you have (and
continue to do so; as well as having donated years worth of income
supporting other people's Free Software work), but it's a bit of an
unfair comparison: I'm a fair bit older than you. :)

And yet, I think that is all irrelevant here because I'm not current
criticizing bitcoinj for lacking features!   Quite the opposite, I am
pointing out that the advantage of soft-forks is that its OKAY for
software to lack soft-fork features,  which means that participants
who code only on evenings and weeks are free to continue participating
with the priorities they choose!

Forcing _all_ upgrades to be via hard-fork takes away the freedom to
make that trade-off; and concurrently reduces the collection of fixes
upgrades we could potentially deploy; because will always be
implementations out there like BitcoinJ in 2012 that didn't have the
resources (or interest) to fully implement this feature or that
feature, at least not right away.

And for many things, they simply don't have to, and that should be okay.

> Yes, a hypothetical full node could fork on the version bits. I would be
> quite happy with the version number in the header being an enforced
> consensus rule: it'd make hard forks easier to trigger. But it hasn't been
> done that way, and wishing away the behaviour of existing software in the
> field is no good. Luckily, for introducing a new opcode, the same effect can
> be achieved by using a non-allocated opcode number.

We handle this in Bitcoin Core. Our chosen and intentional way to
handle this is setting a notice. This gives users the freedom to do
what they like, while also behaving in a reasonably sane way by
default.

You don't have to like it, you can behave differently in your own
software or on your own hosts-- all the data is available to you.

(I wouldn't object out of principle to a default config option to take
more aggressive action on unexpected versions... but no one has ever
asked for one... and I'm doubtful anyone would ever do so.)

> The rest of your argument boils down to "people don't have to upgrade if
> they don't want to", which is addressed in the article I wrote already, and
> multiple responses on this thread. Yes, they do, otherwise they aren't
> getting the security level they were before.

They continue to enforce all the same rules as before. With the soft
fork Bitcoin Core users are informed that unexpected things are going
on, and they are free to look at whats going on and decide how to
handle it, or just accept that the new thing is almost certainly
something they don't care about (after all, the rules they signed up
for before are all still in effect, and at any time miners could be
silently imposing new 'soft fork' like rules without their knowledge--
having a big reaction to ones the network was kind enough to tell them
about doesn't seem that reasonable).

For many users and many soft-forks there is no substantial security
implication, and you cannot say that they were not getting the
security level they were getting before. But regardless, even what it
is different, they're free to decide on the cost tradeoff with
upgrading, and they're not forced onto an upgrade hamsterwheel that
disenfranchises their role in the system.

If you have a specific generalized security implication in mind,
you're failing to state it. In your writings you like to assert that I
 "did not respond" or were "not convincing"-- that is not generally my
style, I don't usually think anyone owes me point by point answers,
but I think on this point it seems clear that there is some
implication which is in your head that is a mystery to at least myself
and Jgarzik.

> 75% is a fine activation threshold. By definition if support is at 75% then
> bigger blocks is "winning", but if support fell, then the SPV wallets would
> just reorg back onto the 1mb-blocks chain.

A 75% measurement doesn't actually mean 75% support, due to variance.
Even ignoring that-- you recognize the acceptability of reorgs. The
situation is no worse for an SPV client for a soft-fork; and it's
better because (1) convergence is still guaranteed with exponential
probability (a hard fork can be mutual and no convergence may be
possible-- as is the case for more conceivable hard forks), and (2)
for BIP65 (and current soft forks generally) a _much_ more
conservative threshold is set (because in Bitcoin Core and the general
community around here considers 75% to be too low to achieve high
stability, based on our past experience).

> Re: demonstrated track record. They "work" only if you ignore the actual
> problems that have resulted. P2SH-invalid blocks were being mined for weeks
> after the flag day. That's not good no matter how you slice it: even if you
> didn't hear about any fraud resulting, it is still risk that can be avoided.

A couple points:

That same invalid blocks for weeks (actually months) from BIP16 is the
behavior you will get with a hard fork, for at least the same reasons
(miners asleep at the switch). Much more for a controversial hard-fork
as there will be principled objections.

Blocks get produced that get orphaned every day and this is
unavoidable, so users already must deal with occasional cases where
confirmations get undone.

More recent soft-forks have reduced the incidence of invalid blocks by
substantially increasing the threshold, including better notification
in Bitcoin core, communicating directly with miners more, and making
non-conforming transactions non-standard in advance. These mitigations
have been effective in practice; and we have not seen the same
behavior (which, as, noted is not known to have enabled any fraud in
any case -- in part because to non-upgraded wallets it looks just like
the orphaning that normally happens but with somewhat increased
frequency.).  I think it's unfortunate that people proposing hard
forks have not learned the same lessons, even though the stakes are
higher and the self-resolution of the system is greatly diminished.