From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 170A1B94 for ; Tue, 11 Jul 2017 21:11:41 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-vk0-f43.google.com (mail-vk0-f43.google.com [209.85.213.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1AAE0193 for ; Tue, 11 Jul 2017 21:11:40 +0000 (UTC) Received: by mail-vk0-f43.google.com with SMTP id r125so2610222vkf.1 for ; Tue, 11 Jul 2017 14:11:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:content-transfer-encoding; bh=UoHPQfK0KHxk47lSuj9jDmhXPAd5OvWQB3q9e4EKLv8=; b=tL7m9Ag8YrjauJ6mhDdeeCWopVoiefrdPjzwrgcngjlUSwy+P/yZXutcJ9rXgHBJLe HDFyKHaXfdwoaSLffc1KwyE1IX66hep6+6cHZrlRe44UD9uRiL+sRQA3QN1scRNTQFpP ERRowe6m7wehSwtPeOcjzET/FVNkZf88i2FTW0ysSiqOwdy8+6OFzkflX+FBv/V+MIXu 7QVB0+6B0+kypQMLQZfAngMOwfsxZp+yP/B8qpBWc/g/GPywLTIhGKeoEkKKR17/tk8X jgIF3CsGQeb3GqplrgEFCEAgRL61wJytIIClirFimnvDaG7YLpCU67B774CEW+C5X6Q/ zPkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:content-transfer-encoding; bh=UoHPQfK0KHxk47lSuj9jDmhXPAd5OvWQB3q9e4EKLv8=; b=W1uBCtwOd5JD1RybQ4ocy8APqBRnC/6PReIbI1EVuXcaPNRzLBtksVe5vsCgwewuot 3nW2C1uWbDZ8vm6GuUAu7rtfWyTRCy1Twji05qZG8Ts/CoZgN/5UltLfI2+t0uqXtUTE BZQ8DfzOfNWdmrF0ePD587OH2Qd4xvIA60AjvDJBlGUifh5xULysZEp5ytTiRPdNLvzU SW050sk811XDNf7X1rtPInB3ko8NmWxhE577s3l4V8LeaWAgyntr8swJcKOtcBTV/oJI oUjCqe1XeOvpdWMVB1AdP4ZhQ0AhlmTg/YbDUGjfQ0UV+46v+U+6Gfq3G06bW/AVRu7S jkCQ== X-Gm-Message-State: AIVw112KiIFGlwDETS8L0kkxsOpp9Ya9i+qhP3PAfsf6HZq2pyFv6l/r rBAysd8ULh9Uf/LkAGaMkwq+1MHssQ== X-Received: by 10.31.79.70 with SMTP id d67mr18506vkb.123.1499807499048; Tue, 11 Jul 2017 14:11:39 -0700 (PDT) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 10.103.40.2 with HTTP; Tue, 11 Jul 2017 14:11:38 -0700 (PDT) In-Reply-To: <0119661e-a11a-6d4b-c9ec-fd510bd4f144@gmail.com> References: <0119661e-a11a-6d4b-c9ec-fd510bd4f144@gmail.com> From: Gregory Maxwell Date: Tue, 11 Jul 2017 21:11:38 +0000 X-Google-Sender-Auth: hQDiMMyBR63k6q-Vz2SD79lJTg4 Message-ID: To: Paul Sztorc , Bitcoin Protocol Discussion Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Tue, 11 Jul 2017 21:30:24 +0000 Subject: Re: [bitcoin-dev] Updating the Scaling Roadmap X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jul 2017 21:11:41 -0000 I think it's great that people want to experiment with things like drivechains/sidechains and what not, but their security model is very distinct from Bitcoin's and, given the current highly centralized mining ecosystem, arguably not very good. So positioning them as a major solution for the Bitcoin project is the wrong way to go. Instead we should support people trying cool stuff, at their own risk. So, given that although the vast majority of the things in the document are things I've been supporting for months (Please see Note 1 way down at the bottom) I cannot support your document. I think you may have have missed how much work I put into what I published before talking with people who actually work on the project to find out what they wouldn't object to before publishing the prior document-- and how much I left out that I would have loved to have in; and why I specifically held back from describing it as a roadmap or prompting people to sign onto it (though they did of their own accord). On a more meta-subject, I think grandly stated "top down" roadmaps in highly collaborative development are of minimal utility at best and actively misleading at worst. Fundamentally, it misunderstands the nature of peer collaboration. It's kind of like asking for a roadmap for the development of fusion power; individual practitioners have their own roadmaps, but the collaboration of science does not. Consider an example, The Linux kernel is one of the largest and best funded open source projects, which produces the most widely used operating system kernel in the world and one of the most widely used pieces of software of all time, and it produces _no_ roadmaps. Quoting Andrew Morton, "Instead of a roadmap, there are technical guidelines. Instead of a central resource allocation, there are persons and companies who all have a stake in the further development of the Linux kernel, quite independently from one another: People like Linus Torvalds and I don=E2=80=99t plan the kernel evolution. We don=E2=80= =99t sit there and think up the roadmap for the next two years, then assign resources to the various new features. That's because we don=E2=80=99t have any resources. The resources are all owned by the various corporations who use and contribute to Linux, as well as by the various independent contributors out there. It's those people who own the resources who decide." Linus remarked, "I look at the current release and the next one, as I don't think planning 10 years ahead is sane." Yet the Linux kernel still has every advantage over us: They have far more contributing resources from far more sources, they have a fairly centralized model and control over their own destiny because they have a much more functional pathway to disagreement than we have in Bitcoin for consensus rules. IMO the way to do "roadmaps" in Bitcoin is to roadmap the finalization and release process once the basic technology is done; because it's only past that point that guarantees can really start being made. But that isn't what your document does-- it names a lot of things which are still various shades of research at this point (much of it research I'm working on and strongly support, in fact--) We can also talk in a visionary sense about the sorts of things we're exploring, but it just isn't possible to nail down when they'll be ready until they are: If it's not something the linux kernel can do, it's not something we can do. These are neat and existing ideas, but not a roadmap. At least not as a group. Individually contributing parties have a lot more visibility and control into their own activities, and there is virtue in forecasting at that level. Redhat, for example, has roadmaps. They primarily deal with stabilization and support of already existing technology that you could get in the raw from the various upstream sources (fedora, kernel, etc.). (see for an example, http://www.slideshare.net/albertspijkers/red-hat-enterprise-linux-roadmap ) Separately, what we can do stably in Core is have timely reliable releases. Juniper made it 10 years with regular timed releases that did not slip by more than IIRC three days and which were _all_ production deployable (things changed later, but thats another story). This was an incredible benefit to our customers, but the only way it was possible was because _features_ were not guaranteed in a release. If a feature failed during the final testing and it needed more than the most trivial of fixes, it was _removed_ or disabled. As soon as there are multiple in-flight deliverables it becomes more important to be timely over-all even that that significantly delays single deliverables. This is effectively at odds with hard deadlines on functionality, even before getting into the fact that for consensus features delivery in software is irrelevant until activation which is a public election. (E.g. we shipped segwit almost a year ago, but it still hasn't arrived for users). >From the perspective of Bitcoin I think what people are actually asking for us to do is to (help) drive the Bitcoin Story-- the actual technology and its timelines is usually not that relevant: if it were, they'd be stepping up with resources to contribute to it. There are many ways to do that, -- we don't have to use highly authoritarian methods that wouldn't even work for the Linux kernel. [And all these projects you listed, your help would be more than welcome!] This can be done by a mixture of talking about research _as research_ not a done deal, and by moving discussions of non-research things to where they're actually more forecastable. 98% of the public discussion about segwit was before the pull request, -- there were solid political reasons for this-- but it was the wrong timing. On the case of CSV and CLTV, the general public didn't hear about them until they were merged -- pretty much-- and the timing then was much more compatible with 'roadmapping' +/- activation uncertainty. Some of this is driven by competitive pressure with things like Ethereum or other altcoins (e.g. dash, god save us) that have a lot lower commitment to engineering integrity or even logical consistency. We can't compete with technobabble bullshit, and we shouldn't try and as a side effect back ourselves into a corner where we're making remarkable accomplishments but regarded as failures in payment (because we either forecast it taking N years, which is the best we could promise, or because we forecast the best we might achieve and it was both still too long and the timeframe got missed). One of the big screwups with segwit handling was people sticking some random unrealistic date on it it which was done AFAIK, by well meaning folks who took some random numbers from people working on it for when they'd be done with the development-- not the testing, not the integration, and certainly not deployment and published it as The Date. Then even though the development was actually done by them, segwit was portrayed as massively delayed, because what matters to the users is deployment-- which we can't control. I see you've done this yourself with signature aggregation, sticking Q4 201= 6 right on it, which as far as I can tell is a figure that comes from mars. (Well not quite mars, see Note 1) Probably we'll have the research and an implementation done by then, but with so much uncertainty around segwit activation, I doubt anyone can be about when users will be able to use it (which is what they care about!) It's also not really appropriate to ask people to sign onto stuff when they can't even review it. Perhaps the signature aggregation stuff is insecure, patent encumbered, or practically useless... (It won't be but no one could tell that from your post; because it doesn't even exist as a concrete propo= sal) I think people would rightly protest about a number of these things-- espec= ially things like the the agg sigs and tx compaction, "wtf, I've not heard of this. Who are you to insist this goes into Bitcoin?" In any case; I can repeat the same story for major RFCs, FWIW. Collaborati= ve innovation is _very_ hard to stick to a tight schedule. And road-maps of totally prospective technology that no one has the actual authority to m= ake happen aren't a productive thing for the industry. In a few weeks you'll start seeing articles on the major new things coming in Bitcoin Core 0.15, -- now that we can do, because the work is done, and the outcome is very predictable. There are some awesome improvements around it-- ones we can rally around; and know will be delivered for sure. [ Note 1: I think it is important to disclose that several of the items in this list appear to be more or less quoted out of my own blockstream-internal descriptions of things we've been working on in Bitcoin. A while back Adam Back asked me to publish something which contained significant chunks of this document more or less verbatim, and I declined saying that I personally disagree with some of his points and didn't think that Blockstream attempting to redirect the Bitcoin project (esp towards drivechains) was appropriate-- along with my (above) views on roadmaps (which I have included here a private email thread on the subject). I feel it's important to disclose this, and that the document was not otherwise created with the input of project contributors (except Luke-Jr, apparently). I wasn't previously aware that Adam had been working with Paul on this, had I been I would have also encouraged people to be a little more transparent about it. ]