From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Fri, 02 May 2025 15:24:42 -0700 Received: from mail-qv1-f60.google.com ([209.85.219.60]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1uAyoD-0003RT-PT for bitcoindev@gnusha.org; Fri, 02 May 2025 15:24:42 -0700 Received: by mail-qv1-f60.google.com with SMTP id 6a1803df08f44-6e8f9057432sf50681936d6.1 for ; Fri, 02 May 2025 15:24:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1746224676; cv=pass; d=google.com; s=arc-20240605; b=kTLDXuPoRvC2CuuK4S8GQA8/WGBeZO+HshRTvCmSBiWbHBJ7WcEMcWNx9M0A3Lho94 7Sl4eW8yhokdEOm1jY11gY7VSwRHRz68L6dH7pLkTXO080smEG2suB6KwcPV9kdzPJ3c GZg1M5KGwKFx9f81YZce7U57lYLHjX09kNpLbAjDWOCrTdftQ8C7Nv2llWxDCAJFrQ66 wWR6mY+YX4LPchhjZ+s/FYeImEnc2aMPg9XB9i1n/G6Vt9aMvFEEHW9r0hamrpIIHZ6r 57p3q02wj118sqP4pDb2j8yzLWWSsjaXbYzWN7YF3TIbqNJB3m4SHHbUmFud0azG9IBt H+Vw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:thread-index:content-language :content-transfer-encoding:mime-version:message-id:date:subject :in-reply-to:references:to:from:sender:dkim-signature; bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=; fh=mEMpKTToPHo71zUiQs/6palwAcz8VSe6UBH+W7JN3M0=; b=MpOzY16TszSXdyEEGO+7BED0JxSVc7Rpx8sZVxsYx0pf8sYt18DMS9zclzHoum13oV l+fxd6iqpSG7RmFUSGVfk676Q5OxRCsLgwxonnobBmH6jwJBpw50I55niUMYHvi5PRxy 6CJYfkckg52muYFCVn955hEteduHwT0qyhigYx1lm4x2rN/U2jEa/2AHuC4Sc/OS2zEy 7QjhKkkN/bfS4sGuF1Wk50IZeWV5ixv+OS0YNTjpHawd8PV1nC0ryNaXinl033LLlwSt cuxR8cSjOvMEIHLplSDRUm2GwHtQiGSR4WeLyzWWFqYbQd4mflf+nTBLheNO3E5j4QnZ wQ3g==; darn=gnusha.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601 header.b=0YHeZ3g7; spf=none (google.com: eric@voskuil.org does not designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org; dara=pass header.i=@googlegroups.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1746224676; x=1746829476; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:thread-index:content-language :content-transfer-encoding:mime-version:message-id:date:subject :in-reply-to:references:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=; b=G+r5Ryim9fudKFCTL0PlTWsveLwh2ZJ8eJrMomJ6vDJoSqzJlns+32qSshQeBEHXQl s77wI5LOY102cmescO18m+c5L5z2aqH1NbowygZVGyPYQ+VkiJvnkSYHTM/CiOjfLVLu sxhiOrtgmk6jrEDBIMwuv53uZx2EgDd1egcXpNvD6aIYcdHkyNxR2/DFnFkDNgppUyos oV7BdS45JL+cfvqqnlaoklgBQ0p+AoNufbttcJExkOj6rlfi1qPyECWffxtWd56AEBfR /1foQsnqXzqiOMxoqEeL9RpsLc9/8YpUhEbbsJEcSQBFaR40pasJF/ReTz7sacpBaGYI 5tpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746224676; x=1746829476; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:thread-index:content-language :content-transfer-encoding:mime-version:message-id:date:subject :in-reply-to:references:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=; b=Z90wDemjzIqRnhklajnvQihQYbeToijP8o0XBPijUN7YMDzWYzcD0XCTiaZjDFH2Ho JUp5QX03apEb+Wf58va33PoxVJ5AYVQwJKMGLazel1yJzVluKYQB1zquWkfNmXaH3DDx cbstGJDNV0GZlt1SuZzWUoizCN5vetVyM8QkZ//ZZ6BlGQJEvJzp9lRtJXl64HDis7sa jikDR3wqwJEct3SxWF9WMtuzZZTI0v3Z6Y9ewCsqnkiyo+pA/lnuT9sw1npJxlMxb5iD dkX6JBBrWrOldkb+FEzvm3hbwjqiw4bRl1E5naO7dDU2HhJs0TmPMMXaL2/LnMIAAqwx iAdg== Sender: bitcoindev@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUwuJmUfyykc8801UH6hYSCBuA7PkW6GKdIvE9w2ZVIbzAAR2OXBmR/vAzzOXjzBKTVIXu/jLEuV922@gnusha.org X-Gm-Message-State: AOJu0YzqrTQkRrubUVda3SgUQnJB7yEcoyVGX9Z7QwL04yVlpkhlUvjh 7cf9QU3g7Ih2rQpEBB0EReQe4g65I2V1iXMAXwJspIyrBQz8QHow X-Google-Smtp-Source: AGHT+IFz8LGAPn5jb18p/zbk8giOQDYlRcKuJu8H64OGF7H/mftiKy/qqWwvQv7SF+3akg3GNBUUBg== X-Received: by 2002:a05:6214:224e:b0:6e8:fcc9:a291 with SMTP id 6a1803df08f44-6f51538de61mr87067876d6.23.1746224675889; Fri, 02 May 2025 15:24:35 -0700 (PDT) X-BeenThere: bitcoindev@googlegroups.com; h=AVT/gBGgQErraniDC4vuByCWPLuprMDGyHRNSRE/2gqA9OvcCA== Received: by 2002:a05:6214:189:b0:6e8:efc0:7a39 with SMTP id 6a1803df08f44-6f508538c67ls33487336d6.2.-pod-prod-06-us; Fri, 02 May 2025 15:24:32 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXlR1IuAMWKAMpwburvZO2NJweMnLgQ+vTt016ZJVE3ATwus2pNxV+XyycO0DcJCCVjOo8GICE+60QF@googlegroups.com X-Received: by 2002:a05:620a:6209:b0:7ca:df98:2f6 with SMTP id af79cd13be357-7cadf98069fmr106807185a.43.1746224672563; Fri, 02 May 2025 15:24:32 -0700 (PDT) Received: by 2002:a05:620a:3602:b0:7b6:d2da:e6ae with SMTP id af79cd13be357-7cad5cae937ms85a; Fri, 2 May 2025 14:16:45 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU8jEQqlVexhZB3ZXo0odaKDXA3QP5ehMrqsUzDD2Z8MW5COvlbLEKFtTmtf78z37yZLjwV8wMWCe4h@googlegroups.com X-Received: by 2002:a05:6214:1bcb:b0:6eb:28e4:8519 with SMTP id 6a1803df08f44-6f51538d979mr68736826d6.21.1746220604434; Fri, 02 May 2025 14:16:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1746220604; cv=none; d=google.com; s=arc-20240605; b=aRF13qLOygKNq0of4/hHDya/Ou9tjk+SqR6CBvHSHqrMeAOUlSaSdXQzbdbeptEAgp wZ63Dor2mopkLjrG6988fHcJr6wG+cyXv6s8CEBlRXT/5+511mmFj29uceb7PPO+c8GR 0ppbb8XzUolz7PpytWJnMRBj1D32Isgm+TIHU6HVQme9ucscBGS0N1ZizHcU0xdaJ2aY czIG4ShnrDc58qjHj6bk5HcZxiZj6pKxGOe4+lsIHVlyW7AvIVI49VrG8zBh6bJb05hw S9FmDVCFGzcJ7LvAA94A8xQDLrJkoe9nPect6YuUv546BGVrFWdLfFo2JuDJ3Ipy7+CA y16Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=thread-index:content-language:content-transfer-encoding :mime-version:message-id:date:subject:in-reply-to:references:to:from :dkim-signature; bh=rXCfnXNe4w3E4vZjr1ejc18dzAaxAhVPhEtyuYcWIfw=; fh=fI++9PkZmWm5vlxQWjWLdPbe4TNFNhp7TfiBb7Dkdcs=; b=e7hOkXavcy5eQOG+NnQnLOC5RQCHpDpXdXcRMGV1xUXTdQqYOle3bWZ278AITwMUh0 2fc7QfnPmgz59mXo8roGjRfwLTncNRCLV3JkDHFOzUTegUXKLWqJKi6TEbwHOLYR/FOV ONBBIIJpnQ9ar4/XWxZ4J6oG/It2wkgg2KES62bUaMYbLI7jHIk+DeTWE+FjZwPTPkLC 5jBWMei/9zXerNUkYIoBCGp6aC/HE3y9G5bhB5v1OpVEs2EVWM79RyBSosQ/Sdji6jd9 f65qBfxNBpdHQ63Co0VQcyyVpJ7p9XC+Q1TQYMjU447iOv12sY0xyDapS9MUIwGrJW1x EGDg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601 header.b=0YHeZ3g7; spf=none (google.com: eric@voskuil.org does not designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org; dara=pass header.i=@googlegroups.com Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com. [2607:f8b0:4864:20::72a]) by gmr-mx.google.com with ESMTPS id 6a1803df08f44-6f50f3acf36si542836d6.1.2025.05.02.14.16.44 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 May 2025 14:16:44 -0700 (PDT) Received-SPF: none (google.com: eric@voskuil.org does not designate permitted sender hosts) client-ip=2607:f8b0:4864:20::72a; Received: by mail-qk1-x72a.google.com with SMTP id af79cd13be357-7cadd46ea9aso86264985a.1 for ; Fri, 02 May 2025 14:16:44 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCVOGgcP+2yIpwIYZEYMz+9dQA2mmCBRaPKWyFsxpMyzHbTWF4hVOiIS4lr5OMOjAI6+bqAQqaxYnECr@googlegroups.com X-Gm-Gg: ASbGncs+d/f9pButmmIB2xg8w/x7OZEmHPJDctwLJNgai7L2jFOHmW3K54R6KaSVgXT hxT/DxxAEa3fM+Vrhz7Bu4XXXu7jqsWaQLUwQ8TYVRNdCr2700x/y5Fy9Itzv+z9g081nRax/rM SXJ1Dv+m7iYrvXk/CBaBo+pOBMXM00H2bBoFu6WmLmxJNCadVPj7sjsWVzveO6lGbJS8nZLPApa Re25tywInqDOyuYIjFud2ImflE1EDrQTV7DV8pbDSK3b3o9nrhRyXO/XB1vSqTv7XgOhdrrcxqU NrpPZLH1+TweUluPqcF2JcQ9gmQdyYab1ZM6kHVKffDAXvKV6LX4YbpDJg7fbeSs9ci3YuT/IrR qp/CTyQ== X-Received: by 2002:a05:620a:400c:b0:7c5:4be5:b0b3 with SMTP id af79cd13be357-7cad5ba38a3mr682442085a.48.1746220603725; Fri, 02 May 2025 14:16:43 -0700 (PDT) Received: from ERICDESKTOP (c-73-227-67-43.hsd1.nh.comcast.net. [73.227.67.43]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7cad23d24besm236865285a.58.2025.05.02.14.16.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 May 2025 14:16:43 -0700 (PDT) From: To: "'Sjors Provoost'" , "'Bitcoin Development Mailing List'" References: <8E4DFC2E-23D4-4D22-87C5-415A3CFC7E57@sprovoost.nl> In-Reply-To: <8E4DFC2E-23D4-4D22-87C5-415A3CFC7E57@sprovoost.nl> Subject: RE: [bitcoindev] Removing checkpoints in Bitcoin Core v30 Date: Fri, 2 May 2025 17:16:42 -0400 Message-ID: <035701dbbba7$807f23b0$817d6b10$@voskuil.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Content-Language: en-us Thread-Index: AQGpX3yTIPgtJr/2QclkLvTcE2GpaQIzdeJRtBLR2fA= X-Original-Sender: eric@voskuil.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601 header.b=0YHeZ3g7; spf=none (google.com: eric@voskuil.org does not designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org; dara=pass header.i=@googlegroups.com Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -0.8 (/) Hi Sjors, > In the context of BIP30 [0] Eric Voskuil brought up performance: I didn't originate the point on performance. I said (in the BIP30 thread): > One reference states that =E2=80=9Cassume valid=E2=80=9D speeds IBD, but = of course it does so by not validating. Which was in response to your OP of this thread, which provided that refere= nce, specifically: "Assumed Valid Blocks: a feature designed to replace the secondary use of c= heckpoints for (optionally) speeding up Initial Block Download (IBD) by ski= pping validation of signatures in old blocks. This was deployed in Bitcoin = Core 0.14" - David A. Harding This in turn references the following discussion: "Bitcoin 0.5.0 built upon those checkpoints to speed up IBD by skipping ver= ification of signatures in blocks that were earlier in the block chain than= the most recent checkpoint." https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks > > The top checkpoint is consensus for over 11 years and materially reduc= es > the validation cost of 295,000 blocks. >=20 > I don't think performance should be a consideration when removing > checkpoints. To be perfectly clear, I am not arguing against this hard fork because it r= educes IBD cost. I=E2=80=99m pointing out that one of the arguments *in fav= or* of removing checkpoints is that "assume valid" now serves this "seconda= ry use". However, as I pointed out, assume valid is not consensus, it achie= ves this outcome by trusting, not validating. The existing checkpoints are = consensus - they provide this advantage when fully validating. I'm not suggesting that checkpoints need to be added to improve performance= . I'm pointing out that removing them hurts performance. So performance is = obviously not a reason to accept such a hard fork. > Afaik checkpoints were not introduced as a performance feature, but rathe= r as > a DoS vulnerability fix. It appears from the above discussion that there was more than one reason. H= owever this isn't relevant. What matters is the consequences of removing th= em, not why they were introduced. > Even if they were, imo they shouldn't be used for performance enhancement= , > as it creates a temptation to add more - although Eric clearly says: >=20 > > I would never advocate for adding more >=20 > But perhaps someone else would fork Bitcoin Core (or Libbitcoin) and star= t > with honest checkpoints, in order to gain adoption due its excellent > performance - especially when combined with a UTXO snapshot. And then > suddenly it has an "upgrade". >=20 > But even if this wasn't a concern, This feels more like speculation than a reasonable concern. To date bitcoin= d has introduced two trust-based mechanisms (assume valid and assume utxo) = both with the objective of avoiding validation to improve IBD time. As I un= derstand it, BC ships with a very recent assume valid configuration by defa= ult. If you have this concern, it should be directed these current practice= s, not at ancient checkpoints. Removing the latter does not eliminate the f= ormer - which should be the current concern. I've recently seen declaration= s that the IBD problem is "solved" because of assume valid and/or assume ut= xo. That's not speculation. > the performance benefit of the latest > checkpoint from 2014 is just very small. That's because the early blocks = don't > have that many signatures to check. The first 5 years of Bitcoin represen= t only > 3% of all historical transactions. This assumes that signature validation is the major cost, which is not the = case for bitcoind architectures. > On an M4 MacBook I did an IBD up to the last checkpoint at height 295,000= . > This took 17.5 minutes. I have done this many times, on powerful machines, and it's generally at le= ast an hour - even with checkpoints (and headers excluded). > I then ran it again with -assumevalid=3D0, i.e. checking all signatures, = which took > 18 minutes. The major cost for the bitcoind architecture is unspent output accumulation= . There is only a modest distinction between full validation and assume val= id across the entire chain. As a result this measurement doesn't reflect th= e true (necessary) cost differential. I just performed a 1-295,000 block li= bbitcoin sync with and without checkpoints (and no milestone/assume valid).= The checkpointed time is 3.4 minutes and without is 8.3 minutes (on the sa= me machine BC takes 69.3 minutes with checkpoints). So checkpoints validate= d at 41% of the time vs. without (vs. your 97%). Again, this is not the bas= is of my comments, just clarifying for the record. > These times are from connecting the first block until connecting block > 295,000. They exclude the 1.5 minutes spent on the header pre-sync > mechanism, which is required to operate safely with or without (new) > checkpoints. Same in both cases. > Libbitcoin would probably find a different ratio, as might a machine with > severe CPU limitations, but I suspect it will be equally negliable. And w= ithout > new checkpoints the ratio keeps going down over time. Of course, the ratio is small and will keep decreasing. On this machine it'= s a difference of 4.9 minutes out of 4.4 hours for libbitcoin. The ratio is= smaller for BC, as it takes 24.5 hours to sync to the same height even wit= h assume valid. > That said, I've never been a fan of -assumevalid conceptually.=20 I agree, not a fan. We provide milestones (same security model) so that peo= ple can rapidly reproduce a previously synced node from the network by spec= ifying a milestone that they have previously validated. On my current low e= nd ($350) benchmark on a 2.3 Gbps Internet, this takes under 40 minutes to = block 850k, and checkpoints don't make a difference either way. It can be f= aster to sync from the p2p network than to copy the store, and of course do= esn't require a local copy. So it's a useful feature, but not a substitute = for validation. > Hopefully it can > be replaced by something like AssumeUTXO + SwiftSync [1] for the > background with full validation using block Undo data. I'm not so clear on why BC wants to give people the impression that a node = is usable when its chain hasn't been validated. I don't see this as the "ho= peful" outcome, but that's another topic I suppose. I also don't see the reason for all of this complexity to end up with a sec= urity model identical to assume valid (until it's actually validated). Why = not just sync the strong blocks and validate them in the background without= all of the fuss, extra bandwidth, and precomputations? In any case, I still do not see ANY justification for hard forking out the = checkpoints, and there are of course legitimate reasons to avoid doing so. https://groups.google.com/g/bitcoindev/c/aqHRfa0UWFo Best, Eric --=20 You received this message because you are subscribed to the Google Groups "= Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/= 035701dbbba7%24807f23b0%24817d6b10%24%40voskuil.org.