From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 861CA1749 for ; Wed, 30 Sep 2015 22:59:24 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f178.google.com (mail-ig0-f178.google.com [209.85.213.178]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6C75224A for ; Wed, 30 Sep 2015 22:59:23 +0000 (UTC) Received: by igbkq10 with SMTP id kq10so2634799igb.0 for ; Wed, 30 Sep 2015 15:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=V6nMBbN4t7sY2F+6T8CesMwMQxpIoX8bclgCXi3+W1I=; b=oD/I0uQguIe1dX793U1nGQJjwYmct1eydcweaBXZgiRYzQ5mBa4i6AS9lseH/1nRXC VCUhfwZOxfkmBVZrVyiwDTesCpmrsYIR2fwMmUBmz3qJtizg0XdkLjRHMwOcEm9ZUPly 5D6KAdnZtRuYGkvhOrO08SrZrM70UVMpMpxQFJNELwADeQDxsC11A+Z4vADhA3gwQBAw mGbhbJeFhp23K89iau7yOQPS8VdxrP7vOUTbXwwG9Suov1Wm/RUyOdmsqXuPHgch9n96 AyctqjDvCfooVXlz1YIIJi9ukYFGPIAUKenzw0dKWUMXty12MjQ7xJyle0dv7JJtw0xP g4RA== MIME-Version: 1.0 X-Received: by 10.50.142.7 with SMTP id rs7mr32456828igb.62.1443653962848; Wed, 30 Sep 2015 15:59:22 -0700 (PDT) Received: by 10.107.19.30 with HTTP; Wed, 30 Sep 2015 15:59:22 -0700 (PDT) In-Reply-To: References: <20150927185031.GA20599@savin.petertodd.org> Date: Wed, 30 Sep 2015 22:59:22 +0000 Message-ID: From: Gregory Maxwell To: Mike Hearn Content-Type: text/plain; charset=UTF-8 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY! X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2015 22:59:24 -0000 On Wed, Sep 30, 2015 at 9:01 PM, Mike Hearn wrote: > I coined the term SPV so I know exactly what it means, and bitcoinj The term comes from the Bitcoin whitepaper. > On the other hand, full nodes all claim they run scripts. Users expect that > and may be relying on it. The unstated assumption here is that the nodes run > them correctly. A soft fork breaks this assumption. They run them correctly with respect to the rules that they implement, nothing about a soft-fork changes that. The system could have been designed in a way that wasn't full of nice compatibility features. The history of bitcoin could have been that past improvements were all performed with hard forks instead of soft forks. But neither of these things are true. (And I think it's very likely that there would have been fewer improvements if that were the case). > I'm going to ignore the rest of the stuff you wrote about "design decisions > to lack security" or "cheaply avoidable lack of validation". When you have > sat down and written an SPV implementation by yourself, then shipped it to a > couple of million users, you might have better insight into basic > engineering costs. At the end of the day we need to deliver software to our users that delivers on their needs and doesn't undermine their privacy or security; even if its really hard. So when someone calls out something that I'm working on that could use improvement, my response isn't to tell them how much I'm not going to listen to them because I've accomplished some long list of things that they haven't; that software I've written runs on hundreds of millions of devices; ... rather my response is to hear out their concerns, even when due to extensive context I'm confident that they are probably confused or dishonestly motivated; because there is always a potential to learn, and always a potential to do better. I have found this to be pretty productive, as even when both parties walk away with the same positions they started with, I usually learn something along the way just because I paid attention. > Until then, I find your criticisms of code you think was > missing due to "stonewalling" and so on to be seriously lacking real world > experience. BIP16 was published on 2012-01-01. Enforcement on the network began on April 1st 2012. Support for merely sending to P2SH addresses was merged into BitcoinJ, Nov 30th _2013_, after it was written by Mike Belshe. In the interim you spent considerable time arguing against implementing it, e.g. in one example incident: --- Day changed Thu Sep 12 2013 10:03 < TD> heck, if a recipient really really wants to receive a p2sh payment for some reason, they can just put the p2sh output into the payment request message .... 10:17 <@gmaxwell> TD: In any case, P2SH has been deployed for something like two years now. Your arguement seems to be basically we should be creating a false tying between payment protocol messages and things like escrow usage in order to coerce people to adopt the payment protocol in places where an address would do. 10:17 <@gmaxwell> I think thats a bit cruddy. 10:19 < TD> no, i'm saying p2sh is a feature that just isn't usable. i did point this out at the time it was merged - gavin believed that apps to do complicated multi-device wallets would appear before a payment protocol did, and people wouldn't like the look of long addresses. that was pretty much the rationale given. that didn't happen, obviously 10:20 < TD> developing features that are used only by bitcoind developers, isn't the right way to go, and p2sh definitely falls into that category ... 10:35 < TD> i'm not going to do it myself because anyone who is capable of producing and running something that uses p2sh is capable of working with the payment protocol as well, and that gives a better user experience overall. but if someone else wants to, go for it. I don't think there is anything wrong about calling this stonewalling. At least thats how it came across to myself and others. I'm sorry if I judged too harshly there. To be clear, by pointing out your past opposition and non-deployment in this message I am _not_ trying to attacking you for failing to support P2SH. I pointing out that right or wrong..... That you actively argued against it. That you chose not to implement it, and only accepted a patch for it a year and a half later. From your own words it seems clear that you didn't implement it due to actual opposition, but even if the non-implementation was simply engineering priorities, the fact remains that you didn't implement for a very long time. And that is _okay_, we still got it anyways, and today tens of thousands of transactions per day use it and P2SH secures about 10% of all Bitcoin value. This is possible because with a soft fork users using other software can gain functionality which might be critical to them (As Jgarzik was saying about Bitpay in the discussion I was quoting from) that you don't have the time or interest to implement in your own software. > What? This is nonsensical. P2SH was added to the full verification code > quite quickly, Yes, Matt Corallo added it to code which by your admission no one was using. I agree this is not relevant. > resolve. Like I said - write the amount of code I've written, unpaid in your > evenings and weekends, and then you can criticise bitcoinj for lacking > features. I think it's likely that I've spent significant more time unpaid on my evenings and weekends creating software for others than you have (and continue to do so; as well as having donated years worth of income supporting other people's Free Software work), but it's a bit of an unfair comparison: I'm a fair bit older than you. :) And yet, I think that is all irrelevant here because I'm not current criticizing bitcoinj for lacking features! Quite the opposite, I am pointing out that the advantage of soft-forks is that its OKAY for software to lack soft-fork features, which means that participants who code only on evenings and weeks are free to continue participating with the priorities they choose! Forcing _all_ upgrades to be via hard-fork takes away the freedom to make that trade-off; and concurrently reduces the collection of fixes upgrades we could potentially deploy; because will always be implementations out there like BitcoinJ in 2012 that didn't have the resources (or interest) to fully implement this feature or that feature, at least not right away. And for many things, they simply don't have to, and that should be okay. > Yes, a hypothetical full node could fork on the version bits. I would be > quite happy with the version number in the header being an enforced > consensus rule: it'd make hard forks easier to trigger. But it hasn't been > done that way, and wishing away the behaviour of existing software in the > field is no good. Luckily, for introducing a new opcode, the same effect can > be achieved by using a non-allocated opcode number. We handle this in Bitcoin Core. Our chosen and intentional way to handle this is setting a notice. This gives users the freedom to do what they like, while also behaving in a reasonably sane way by default. You don't have to like it, you can behave differently in your own software or on your own hosts-- all the data is available to you. (I wouldn't object out of principle to a default config option to take more aggressive action on unexpected versions... but no one has ever asked for one... and I'm doubtful anyone would ever do so.) > The rest of your argument boils down to "people don't have to upgrade if > they don't want to", which is addressed in the article I wrote already, and > multiple responses on this thread. Yes, they do, otherwise they aren't > getting the security level they were before. They continue to enforce all the same rules as before. With the soft fork Bitcoin Core users are informed that unexpected things are going on, and they are free to look at whats going on and decide how to handle it, or just accept that the new thing is almost certainly something they don't care about (after all, the rules they signed up for before are all still in effect, and at any time miners could be silently imposing new 'soft fork' like rules without their knowledge-- having a big reaction to ones the network was kind enough to tell them about doesn't seem that reasonable). For many users and many soft-forks there is no substantial security implication, and you cannot say that they were not getting the security level they were getting before. But regardless, even what it is different, they're free to decide on the cost tradeoff with upgrading, and they're not forced onto an upgrade hamsterwheel that disenfranchises their role in the system. If you have a specific generalized security implication in mind, you're failing to state it. In your writings you like to assert that I "did not respond" or were "not convincing"-- that is not generally my style, I don't usually think anyone owes me point by point answers, but I think on this point it seems clear that there is some implication which is in your head that is a mystery to at least myself and Jgarzik. > 75% is a fine activation threshold. By definition if support is at 75% then > bigger blocks is "winning", but if support fell, then the SPV wallets would > just reorg back onto the 1mb-blocks chain. A 75% measurement doesn't actually mean 75% support, due to variance. Even ignoring that-- you recognize the acceptability of reorgs. The situation is no worse for an SPV client for a soft-fork; and it's better because (1) convergence is still guaranteed with exponential probability (a hard fork can be mutual and no convergence may be possible-- as is the case for more conceivable hard forks), and (2) for BIP65 (and current soft forks generally) a _much_ more conservative threshold is set (because in Bitcoin Core and the general community around here considers 75% to be too low to achieve high stability, based on our past experience). > Re: demonstrated track record. They "work" only if you ignore the actual > problems that have resulted. P2SH-invalid blocks were being mined for weeks > after the flag day. That's not good no matter how you slice it: even if you > didn't hear about any fraud resulting, it is still risk that can be avoided. A couple points: That same invalid blocks for weeks (actually months) from BIP16 is the behavior you will get with a hard fork, for at least the same reasons (miners asleep at the switch). Much more for a controversial hard-fork as there will be principled objections. Blocks get produced that get orphaned every day and this is unavoidable, so users already must deal with occasional cases where confirmations get undone. More recent soft-forks have reduced the incidence of invalid blocks by substantially increasing the threshold, including better notification in Bitcoin core, communicating directly with miners more, and making non-conforming transactions non-standard in advance. These mitigations have been effective in practice; and we have not seen the same behavior (which, as, noted is not known to have enabled any fraud in any case -- in part because to non-upgraded wallets it looks just like the orphaning that normally happens but with somewhat increased frequency.). I think it's unfortunate that people proposing hard forks have not learned the same lessons, even though the stakes are higher and the self-resolution of the system is greatly diminished.