From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BCACA723 for ; Thu, 17 Nov 2016 10:10:31 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f169.google.com (mail-pf0-f169.google.com [209.85.192.169]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D874E248 for ; Thu, 17 Nov 2016 10:10:30 +0000 (UTC) Received: by mail-pf0-f169.google.com with SMTP id c4so36917260pfb.1 for ; Thu, 17 Nov 2016 02:10:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=lAHa2NKX88AyEc5RX8kwU+G/eSeRVhjjiB6KtbicSbA=; b=MBvghedVEAbaFqyau824jPOOLxTHgnsBuYGArPg1+VbgJf5TWZ/ZaC4w3fF80jaaLH TG9RDh4MXzmBAVGRK16nlhWCUHQ0g4/S0O1a/ca93yqS660KmV/OjVqE6lpz0IXOh2GR 4yBQSATs/I0DRAolPRW4dWyk2jD/iGCn2RagMGYJDraQkxV6IQV+qFoFWEhKdkHe1SsA DuWBv85pTxvUuqArY/g1AxaA3cKU6VjbZV2kUEAsmgtxdufbFQX9/NEw+SsJy0PhZ90G kYWPz92DpDivJH1C3/CjrrJVUUZCJwD77eAE4dpAzPNOjN4EbW/ZME54lQsxA+XhaN+Y qG/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=lAHa2NKX88AyEc5RX8kwU+G/eSeRVhjjiB6KtbicSbA=; b=KpYGcLsaqMig5lN+fx1Hes4F7cj3U+FFcOn/UkwXorOaznMU3VwTug1Oyic+0M550C CrFYGKHulb21Oqj3dM5HnvbttQVnPuYYt0tuJtTBj1ohj6RVQsCNzqsop10TBIc3T/FP v8naGnQYS3cYXiPrIuMtE/ym0beRskintHAxUdtj7Gm1ktp20ezeLbEUxoP4nRFCKF97 elYj39kmInzrPCml9ua0jSSMmtI1mCgA1kQa1lPrk1btCtRs4GDxCfDRotxmPcYZ4Al3 LlFWnGs7kbI+e+ytK47qFJuQULqIxMSuDVMQGSgGaJEC8yJcj+lj8zrtBCl1vWDTIaLv TXlQ== X-Gm-Message-State: ABUngvekC0Gt8MY+7oz8ma7A73f+zg8q2FdK3r/rDS5GagNQFssYGa5VqCaU1IoqnRZd/A== X-Received: by 10.98.32.151 with SMTP id m23mr3629353pfj.127.1479377430338; Thu, 17 Nov 2016 02:10:30 -0800 (PST) Received: from ?IPv6:2601:600:9000:d69e:8084:4206:2529:776d? ([2601:600:9000:d69e:8084:4206:2529:776d]) by smtp.gmail.com with ESMTPSA id r1sm5576748pfg.56.2016.11.17.02.10.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 Nov 2016 02:10:29 -0800 (PST) To: Pieter Wuille References: <33BFC318-0BB4-48DB-B5DC-08247FAC6E5A@voskuil.org> <0d66bf24-2ded-cd98-ec55-945e01b436d0@voskuil.org> From: Eric Voskuil X-Enigmail-Draft-Status: N0110 Message-ID: Date: Thu, 17 Nov 2016 02:10:30 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3RH56cXDefiqRbcTEG2VIxPsddaUheGvg" X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 17 Nov 2016 11:22:42 +0000 Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] [BIP Proposal] Buried Deployments X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2016 10:10:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3RH56cXDefiqRbcTEG2VIxPsddaUheGvg Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/16/2016 06:47 PM, Pieter Wuille wrote: > On Wed, Nov 16, 2016 at 6:16 PM, Eric Voskuil > wrote: >=20 > On 11/16/2016 05:50 PM, Pieter Wuille wrote: >=20 > > So are checkpoints good now? > > I believe we should get rid of checkpoints because they seem to b= e > misunderstood as a security feature rather than as an optimization.= >=20 > Or maybe because they place control of the "true chain" in the hand= s of > those selecting the checkpoints? It's not a great leap for the part= ies > distributing the checkpoints to become the central authority. >=20 > Yes, they can be used to control the "true chain", and this has happene= d > with various forks. But developers inevitably have this possibility, if= > you ignore review. If review is good enough to catch unintended > consensus changes, it is certainly enough to catch the introduction of > an invalid checkpoint. The risk you point out is real, but the way to > deal with it is good review and release practices. >=20 > I wish we had never used checkpoints the way we did, but here we are. > Because of this, I want to get rid of them. However, It's not because I= > think they offer an excessive power to developers - but because they're= > often perceived this way (partially as a result of how they've been > abused in other systems). > =20 > I recommend users of our node validate the full chain without > checkpoints and from that chain select their own checkpoints and pl= ace > them into config. From that point forward they can apply the > optimization. Checkpoints should never be hardcoded into the source= =2E >=20 > Having users with the discipline you suggest would be wonderful to have= =2E > I don't think it's very realistic, though, and I fear that the result > would be that everyone copies their config from one or a few websites > "because that's what everyone uses". Certainly, but embedding them in the code makes that a practical certainty. People cannot be prevented from doing dumb things, but let's not make it hard for them to be smart. > > I don't think buried softforks have that problem. >=20 > I find "buried softfork" a curious name as you are using it. You se= em to > be implying that this type of change is itself a softfork as oppose= d to > a hardfork that changes the activation of a softfork. It was my > understanding that the term referred to the 3 softforks that were b= eing > "buried", or the proposal, but not the burial itself. > I do not consider the practice of "buried softforks" to be a fork at > all. It is a change that modifies the validity of a theoretically > construable chain from invalid to valid. I was out at a Bitcoin meetup when I read this and I think beer actually came out of my nose. > However, a reorganization to > that theoretical chain itself is likely already impossible due to the > vast number of blocks to rewind, and economic damage that is far greate= r > than chain divergence itself. It's either possible or it is not. If it is not there is no reason for a proposal - just make the change and don't bother to tell anyone. The reason we are having this discussion is because it is not impossible. > Nevertheless, this proposal shouldn't have "that problem" because i= t is > clearly neither a security feature nor an optimization. That is the= > first issue that needs to be addressed. >=20 > It is clearly not a security feature, agreed. But how would you propose= > to avoid the ISM checks for BIP34 and BIP66 all the time? I'll call straw man on the question. It is not important to avoid the activation checks. The question is whether there is a material performance optimization in eliminating them. This would have to be significant enough to rise to the level of a change to the protocol. Having said that there are a few options: 1. The naive approach to activation is, for each new block, to query the store for the previous 1000 block headers (to the extent there are that many), and just do so forever, summing up after the query. This is the most straightforward but also the most costly approach. 2. A slightly less costly approach is, for each new block, to reverse iterate over the store until all decisions can be made. This would be an improvement below activation in that it would take it takes as little as 251 vs. 1000 queries to make the determinations. 3. A further improvement is available by caching the height of full activation of all three soft forks. Unless there is a subsequent reorg with a fork point prior that height, there is never a need to make another query. Once fully activated the activation height is cached to the store (otherwise just query the last 1000 versions at startup to determine the state), eliminating any ongoing material cost. 4. We may also be interested in optimizing initial block download. A cache of the last 1000 block versions can be maintained by adding each to a circular buffer as they are committed. This eliminates *all* querying for block versions unless: (1) there is a restart prior to full activation - in which case there is a query of up to 1000 versions to prime the cache. (2) there is a potential reorg after full activation, and the fork point precedes the saved full activation height - in which case the cache must be reprimed. (3) there is a potential reorg. before reaching full activation - in which case the cache must be backfilled with a query for a number of versions equal to the depth of the fork point. During initial block download potential reorgs are exceedingly rare (reorgs don't have potential unless they have sufficient work to overcome the long chain) and the cost of handling them as described above is trivial. The cost of priming the cache is immaterial in the context of a restart. So even with a full chain validation one is not likely to *ever* need to query the store. The memory cost of the cache is strictly 3 bits per block (375 bytes total). A simpler less memory-sensitive approach is to use one byte (1,000 bytes total). The computational cost is trivial. This should already be implemented. A protocol fork (or "change that modifies the validity of a theoretically construable chain from invalid to valid") to avoid doing so is not a performance optimization. > I feel this > approach is a perfectly reasonable choice for code that likely won't > ever affect the valid chain again. I find it to be completely unsupportable as there is no security, performance, or feature benefit in it. e --3RH56cXDefiqRbcTEG2VIxPsddaUheGvg Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJYLYIXAAoJEDzYwH8LXOFOYIcH/AqhsrMDDtcxGsG7ZEYPpRNN BPiYFV1W+c8siCqMP7oVjKWjqfLaYNt5II/nM8mYYK7R8o/o6j3gB44BBkNQYn1d BEiEe1I6mCDcC6/iMxBeNNvzZSuAhndMTpX4HOeWroPdYLnyOGv1SkcjLpVvG878 rPJD8aQPDKvIwOPZdGf1mfWXGgk5tEiFEQBFYwHpjlRHnSsHO0pUWJ8Aqtn4IoAT DKSivizV8DUuaanOhHNvbTHmhzOabxIUhSyfskfyrbIkuuKF6RDXC4BGgNLUIpEw 6QUvLr56Y3QRfX7wvC52ShdbmSheKFdrf0Jrfm5VqiDpRX+mB9dyJ5kFvquWhP0= =31NW -----END PGP SIGNATURE----- --3RH56cXDefiqRbcTEG2VIxPsddaUheGvg--