From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Tue, 02 Jul 2024 04:27:12 -0700 Received: from mail-yb1-f190.google.com ([209.85.219.190]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sObfD-0001Iy-54 for bitcoindev@gnusha.org; Tue, 02 Jul 2024 04:27:12 -0700 Received: by mail-yb1-f190.google.com with SMTP id 3f1490d57ef6-e036150efc6sf6836586276.2 for ; Tue, 02 Jul 2024 04:27:10 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1719919625; cv=pass; d=google.com; s=arc-20160816; b=lyEQiShgOv+p5AdEviOTAHhrCOJbLPTbQgXmC0/suPZJjjVGv7VIqSGt01UD4cUyHM 4a9j/+h6XggX80SIGkRzhHSLOGjonmG32UM5PKgBJ4yGR2Sah7g2UKVvRdbrm5oWeIpU U5+0Lq+SbpUCaqgucF5wDrKxabaodR0FNYGDpkyfLQUmBtN845nTLnmXETY+SMMZxWE2 YmdRUWCqKtc8ur8YVim56CvJ1xlqQSgnwJmedEkNzoO9KQrIuPPWF7WOOytdd4njm+R/ j9mvtl03uJlog90FmSkMesm1JjmoJQKfKGwEhQw/eXl0i6GnMmDaCR5UqAIJ7KuEX24r HdTg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:mime-version:feedback-id :references:in-reply-to:message-id:subject:cc:from:to:date :dkim-signature; bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=; fh=B8gD5/RfkTpsJzruBx/3IjkAQefkQFBaKnXLFpcB0Nw=; b=NFeqUhzMdorJ3eBZ4QobKp9Kh7GF9iUdVN3M7wEVMUqDjOExZw7xICVmx/oCFEIPc7 OT8RJBKxCfN50Hg/M/g3AOeOA02Lj3hKMBUOMm/8UiOIW+0f5dUPsLG9uBtRYJnlnj7Y pfynD7pU9fi8r5JfQ2MpdY/JI4723eLTGfSUst36HhIF3lcNkEi8N4uLgvTclrfCu9PV XzedpbpMHuLp+8Q8O1WDmTBNIVQjhRrX5ut6VgCtoJunTDspQShmEyL3FEPwYYEjpHqf bXCF43sPG9X1RIqsyuNsyUEjY7ssoApmZC0ViJLGdTSNBJdaCDFEdBpsAB6PDJHkBZ5F GpSw==; darn=gnusha.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb; spf=pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1719919625; x=1720524425; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:mime-version :feedback-id:references:in-reply-to:message-id:subject:cc:from:to :date:from:to:cc:subject:date:message-id:reply-to; bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=; b=DjpT1TAtkphUb99oFvEAmA8KwKi6dRxQ5PqUfEuQ1XInG4lG9ahJ67hKI4tZVC/Z8/ 2UB5R7/ZWuGRdcM8Yjxx78jJnHXM2R6ZzskcDzxNp+Od7M6fMKRshQzSHsrrIsU3sEBy 2/CJnFvmy47QPqvAHeRhO2Ju9VGrLbYDPpGQN7Li4TlL8XLR08/AoW4ysLPecDnln1b1 EiiD1HEpn7AB0O80SVQGBKnTjJmheBdkTQ4O7BMjdzLTnsXP+kv17qAa6rCNCY24jU6c ojw7hBxSROvjNSu5HEX1a3dnJziVNU2hWOVi6Di//KgWI9W9D55cE7yNrn/CeKzkXkQ4 +y/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719919625; x=1720524425; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:mime-version :feedback-id:references:in-reply-to:message-id:subject:cc:from:to :date:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=; b=C3c7xnSOJXhiw4vPabGSXJ1Chc70Zzc0WSY5VTEmymNUc/8LwRsOGhFLxcQRBiOA2u M6P+Vn73uI/oywSXMd2E7MZI79auA8B5vRBWSkT0Py+3JN201MrFXc/jKlaxIRvlm33A +reV2/aYIngGImbXRn9Flq3MeTbs/9rAb2SD0LzK39vrjogOj8bjv+vkIZ7zVOY9is7R T3yOSrP/hvX7/8uQZR2LW0Tm//BgHQkgDSP1SVtczBK7mKLhdQTAmxXsQAVw5Q8y1uph IM6ipfqMkLhpCwhqVWH3/KT1Qof7kugvii6vW0WzY3Kk7z06iUkqlionX0Vdla1xnutc /gLQ== X-Forwarded-Encrypted: i=2; AJvYcCWg2LbjKYrdST19nuOh7QRjKdSoZJn6lzJhywJ8bdO6d0uPyBdwWskBqmWy6SP7x96tn/tbxOQKiy9YfVsUDnYG56Eh8nE= X-Gm-Message-State: AOJu0Yw/PYlpbtpiTyk9YuJ3SUZihy1JAXkhs5K3t0W+Sg3mHKSWYSaH iW4RR29mDfVhZcbhTM0Ew0ghNfJhpLMw6zzPxGndtTADYS0C9VfZ X-Google-Smtp-Source: AGHT+IGIoFVbXsWyr03DUceJ7Pfy4N5KZyTUDElOr38kZbWRRSBogf3ZtJdhSH0+nbH9TsjgZqBIow== X-Received: by 2002:a25:b198:0:b0:e03:5b27:70bc with SMTP id 3f1490d57ef6-e039234a4fdmr3693880276.45.1719919624574; Tue, 02 Jul 2024 04:27:04 -0700 (PDT) X-BeenThere: bitcoindev@googlegroups.com Received: by 2002:ad4:5bec:0:b0:6b5:577:fbee with SMTP id 6a1803df08f44-6b59fcb6f75ls73677346d6.1.-pod-prod-06-us; Tue, 02 Jul 2024 04:27:03 -0700 (PDT) X-Received: by 2002:a05:6214:19ea:b0:6b5:b491:830a with SMTP id 6a1803df08f44-6b5b71242e7mr1380596d6.13.1719919623419; Tue, 02 Jul 2024 04:27:03 -0700 (PDT) Received: by 2002:a05:620a:917:b0:79d:5863:c65b with SMTP id af79cd13be357-79d99efb7fdms85a; Tue, 2 Jul 2024 03:24:00 -0700 (PDT) X-Received: by 2002:a05:651c:889:b0:2ec:440c:4e1c with SMTP id 38308e7fff4ca-2ee5e380e02mr61771641fa.11.1719915838459; Tue, 02 Jul 2024 03:23:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1719915838; cv=none; d=google.com; s=arc-20160816; b=yBh14/rnljjbamJKSxX5mfEPdAVau5XcNkhGgPiYDXxEuL3mK0uqa3hg6Bq1PCbTwY HrGs9Wcg6Ey1oD/5dDvf9DMw/7MEwAcLM4njUYqcfPReBiNJW71i5tjEg+LCD22sUEIG k8XNGgLi4i/uMSnOtubJQ3kGGJWPLDeESFxowhXP4nmfOMvz4/zAGheQejxD9zL64bMV JYNhfybpXxPiV3WyoAYhrmucYqR9vJGhU9CZcB7DxeS75RDXWVpN6n+hez049ChnnU5l NOTzOgFGETyN4jKYwqw0U0Z6skj17YvwJbh8EGBzSQsL0TF9D/5T9ol2JIkPNjbn3hgg vlkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:feedback-id:references:in-reply-to:message-id:subject :cc:from:to:date:dkim-signature; bh=SMWemyCP6c3lRHcbKXYFEG8Ep22IBd/uhhRPq6YQnX4=; fh=VUyRMGDsLDyKXHBc8DWjokFBiSMTvXavinKdBJZhUls=; b=tncEpbi66UWtlBrFgQ9wwrLjl6v8FAHvY9jsZwUF/ZCmQUKLCFQitovqWp5e+eSjVh yfuQJIoUk7KDyJVrlwXO7AShPvSD8u+Z+ojtu0i2pfKbJFfKise/za/QCN2PtxHR+7J7 tT2pO+hcg53VlGjWFxMPvxTJJShLuZAC2o17IW8A2UaQSrWydeGxeXN8bIm3/YdTXIKR Xfgr//dLknz1jwAnxe0X9/Vk1ClIVE5GuS6w4MYT0P2SLe8/p6p/mZx6FsPvZ9G/mTe8 pKyicCEpYtCAzAeXshpzVCoWSvhsdDdh1+IXGYkbjkMo9NKEIyRhNJdN2sYMuGBXqKq3 m36w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb; spf=pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com Received: from mail-40141.protonmail.ch (mail-40141.protonmail.ch. [185.70.40.141]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-425787f3b39si3304545e9.1.2024.07.02.03.23.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jul 2024 03:23:58 -0700 (PDT) Received-SPF: pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) client-ip=185.70.40.141; Date: Tue, 02 Jul 2024 10:23:51 +0000 To: Eric Voskuil From: "'Antoine Poinsot' via Bitcoin Development Mailing List" Cc: Bitcoin Development Mailing List Subject: Re: [bitcoindev] Re: Great Consensus Cleanup Revival Message-ID: In-Reply-To: <9a4c4151-36ed-425a-a535-aa2837919a04n@googlegroups.com> References: <72e83c31-408f-4c13-bff5-bf0789302e23n@googlegroups.com> <5b0331a5-4e94-465d-a51d-02166e2c1937n@googlegroups.com> <9a4c4151-36ed-425a-a535-aa2837919a04n@googlegroups.com> Feedback-ID: 7060259:user:proton X-Pm-Message-ID: 662ea3888d61a9c66124cb4e384213808b6d4377 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8" X-Original-Sender: darosior@protonmail.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb; spf=pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com X-Original-From: Antoine Poinsot Reply-To: Antoine Poinsot Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -1.0 (-) This is a multi-part message in MIME format. --b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable >>> This does not produce unmalleable block hashes. Duplicate tx hash malle= ation remains in either case, to the same effect. Without a resolution to b= oth issues this is an empty promise. >> Duplicate txids have been invalid since 2012 (CVE-2012-2459). > > I think again here you may have misunderstood me. I was not making a poin= t pertaining to BIP30. No, in fact you did. CVE-2012-2459 is unrelated to BIP30, it's the duplicat= e txids malleability found by forrestv in 2012. It's the one you are talkin= g about thereafter and the one relevant for the purpose of this discussion. For future reference, the full disclosure of CVE-2012-2459 can be found the= re: https://bitcointalk.org/?topic=3D102395. > The proposal does not enable that objective, it is already the case. No m= alleated block is a valid block. You are right. The advantage i initially mentioned about how making 64-byte= s transactions invalid could help caching block failures at an earlier stag= e is incorrect. Best, Antoine Poinsot On Friday, June 28th, 2024 at 7:14 PM, Eric Voskuil wrot= e: >>> It is not clear to me how determining the coinbase size can be done at = an earlier stage of validation than detection of the non-null coinbase. >> My point wasn't about checking the coinbase size, it was about being abl= e to cache the hash of a (non-malleated) invalid block as permanently inval= id to avoid re-downloading and re-validating it. > > This I understood, but I think you misunderstood me. Your point was speci= fically that, "it would let node implementations cache block failures at an= earlier stage of validation." Since you have not addressed that aspect I a= ssume you agree with my assertion above that the proposed rule does not act= ually achieve this. > > Regarding the question of checking coinbase size, the issue is of detecti= ng (or preventing) hashes mallied via the 64 byte tx technique. A rule agai= nst 64 byte txs would allow this determination by checking the coinbase alo= ne. If the coinbase is 64 bytes the block is invalid, if it is not the bloc= k hash cannot have been mallied (all txs must have been 64 bytes, see previ= ous reference). > > In that case if the block is invalid the invalidity can be cached. But bl= ock invalidity cannot actually be cached until the block is fully validated= . A rule to prohibit *all* 64 byte txs is counterproductive as it only adds= additional checks on typically thousands of txs per block, serving no purp= ose. > >>> It seems to me that introducing an arbitrary tx size validity may creat= e more potential implementation bugs than it resolves. >> The potential for implementation bugs is a fair point to raise, but in t= his case i don't think it's a big concern. Verifying no transaction in a bl= ock is 64 bytes is as simple a check as you can get. > > You appear to be making the assumption that the check is performed after = the block is fully parsed (contrary to your "earlier" criterion above). The= only way to determine the tx sizes is to parse each tx for witness marker,= input count, output count, input script sizes, output script sizes, witnes= s sizes, and skipping over the header, several constants, and associated bu= ffers. Doing this "early" to detect malleation is an extraordinarily comple= x and costly process. On the other hand, as I pointed out, a rational imple= mentation would only do this early check for the coinbase. > > Yet even determining the size of the coinbase is significantly more compl= ex and costly than checking its first input point against null. That check = (which is already necessary for validation) resolves the malleation questio= n, can be performed on the raw unparsed block buffer by simply skipping hea= der, version, reading input count and witness marker as necessary, offsetti= ng to the 36 byte point buffer, and performing a byte comparison against [0= 000000000000000000000000000000000000000000000000000000000000000ffffffff]. > > This is: > > (1) earlier > (2) faster > (3) simpler > (4) already consensus > >>> And certainly anyone implementing such a verifier must know many intric= acies of the protocol. >> They need to know some, but i don't think it's reasonable to expect them= to realize the merkle tree construction is such that an inner node may be = confused with a 64 bytes transaction. > > A protocol developer needs to understand that the hash of an invalid bloc= k cannot be cached unless at least the coinbase has been restricted in size= (under the proposal) -or- that the coinbase is a null point (presently or = under the proposal). In the latter case the check is already performed in v= alidation, so there is no way a block would presently be cached as invalid = without checking it. The proposal adds a redundant check, even if limited t= o just the coinbase. [He must also understand the second type of malleabili= ty, discussed below.] > > If this proposed rule was to activate we would implement it in a late sta= ge tx.check, after txs/blocks had been fully deserialized. We would not che= ck it an all in the case where the block is under checkpoint or milestone (= "assume valid"). In this case we would retain the early null point malleati= on check (along with the hash duplication malleation check) that we present= ly have, would validate tx commitments, and commit the block. In other word= s, the proposal adds unnecessary late stage checks only. Implementing it ot= herwise would just add complexity and hurt performance. > >>> I do not see this. I see a very ugly perpetual seam which will likely r= esult in unexpected complexities over time. >> What makes you think making 64 bytes transactions invalid could result i= n unexpected complexities? And why do you think it's likely? > > As described above, it's later, slower, more complex, unnecessarily broad= , and a consensus change. Beyond that it creates an arbitrary size limit - = not a lower or upper bound, but a slice out of the domain. Discontinuities = are inherent complexities in computing. The "unexpected" part speaks for it= self. > >>> This does not produce unmalleable block hashes. Duplicate tx hash malle= ation remains in either case, to the same effect. Without a resolution to b= oth issues this is an empty promise. >> Duplicate txids have been invalid since 2012 (CVE-2012-2459). > > I think again here you may have misunderstood me. I was not making a poin= t pertaining to BIP30. I was referring to the other form of block hash mall= eability, which results from duplicating sets of trailing txs in a single b= lock (see previous reference). This malleation vector remains, even with in= valid 64 byte txs. As I pointed out, this has the "same effect" as the 64 b= yte tx issue. Merkle hashing the set of txs is insufficient to determine id= entity. In one case the coinbase must be checked (null point or size) and i= n the other case the set of tx hashes must be checked for trailing duplicat= ed sets. [Core performs this second check within the Merkle hashing algorit= hm (with far more comparisons than necessary), though this can be performed= earlier and independently to avoid any hashing in the malleation case.] > > I would also point out in the interest of correctness that Core reverted = its BIP30 soft fork implementation as a consequence of the BIP90 hard fork,= following and requiring the BIP34 soft fork that presumably precluded it b= ut didn't, so it is no longer the case that duplicate tx hashes are invalid= in implementation. As you have proposed in this rollup, this requires fixi= ng again. > >> If 64 bytes transactions are also made invalid, this would make it impos= sible for two valid blocks to have the same hash. > > Aside from the BIP30/34/90 issue addressed above, it is already "impossib= le" (cannot be stronger than computationally infeasible) for two *valid* bl= ocks to have the same hash. The proposal does not enable that objective, it= is already the case. No malleated block is a valid block. > > The proposal aims only to make it earlier or easier or faster to check fo= r block hash malleation. And as I've pointed out above, it doesn't achieve = those objectives. Possibly the perception that this would be the case is a = consequence of implementation details, but as I have shown above, it is not= in fact the case. > > Given either type of malleation, the malleated block can be determined to= be invalid by a context free check. But this knowledge cannot ever be cach= ed against the block hash, since the same hash may be valid. Invalidity can= only be cached once a non-mallied block is validated and determined to be = invalid. Block hash malleations are and will remain invalid blocks with or = without the proposal, and it will continue to be necessary to avoid caching= invalid against the malleation. As you said: > >> it was about being able to cache the hash of a (non-malleated) invalid b= lock as permanently invalid to avoid re-downloading and re-validating it. > > This is already the case, and requires validating the full non-malleated = block. Adding a redundant invalidity check doesn't improve this in any way. > > Best, > Eric > > -- > You received this message because you are subscribed to the Google Groups= "Bitcoin Development Mailing List" group. > To unsubscribe from this group and stop receiving emails from it, send an= email to bitcoindev+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgi= d/bitcoindev/9a4c4151-36ed-425a-a535-aa2837919a04n%40googlegroups.com. --=20 You received this message because you are subscribed to the Google Groups "= Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/= bitcoindev/wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFeOZT7sHDlyrX= wJ5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw%3D%40protonmail.com. --b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
>> This does not produce unmalleable block hashes. Duplicate tx hash= =20 malleation remains in either case, to the same effect. Without a=20 resolution to both issues this is an empty promise.
> Dupl= icate txids have been invalid since 2012 (CVE-2012-2459).
I think again here you may have misunderstood me. I was not making a point= pertaining to BIP30.

= No, in fact you did. CVE-2012-2459 is unrelated to BIP30, it's the duplicat= e txids malleability found by forrestv in 2012. It's the one you are talkin= g about thereafter and the one relevant for the purpose of this discussion.=

For future reference, the full dis= closure of CVE-2012-2459 can be found there: https://bitcointalk.org/?topic=3D102395.

The proposal does not enable that objective, it is al= ready the case. No malleated block is a valid block.

You are right. The advantage i initia= lly mentioned about how making 64-bytes transactions invalid could help cac= hing block failures at an earlier stage is incorrect.
=
Best,
Antoine Poinsot
On Friday, June 28th, 2024 at 7:14 PM, Eric Voskuil <eric@voskui= l.org> wrote:
>> It is not clear to me how determining the coinbase siz= e can be done at an earlier stage of validation than detection of the non-n= ull coinbase.
> My point wasn't about checking the coinbase size, it = was about being able to cache the hash of a (non-malleated) invalid block a= s permanently invalid to avoid re-downloading and re-validating it.

= This I understood, but I think you misunderstood me. Your point was specifi= cally that, "it would let node implementations cache block failures at an e= arlier stage of validation." Since you have not addressed that aspect I ass= ume you agree with my assertion above that the proposed rule does not actua= lly achieve this.

Regarding the question of checking coinbase size, = the issue is of detecting (or preventing) hashes mallied via the 64 byte tx= technique. A rule against 64 byte txs would allow this determination by ch= ecking the coinbase alone. If the coinbase is 64 bytes the block is invalid= , if it is not the block hash cannot have been mallied (all txs must have b= een 64 bytes, see previous reference).

In that case if the block is = invalid the invalidity can be cached. But block invalidity cannot actually = be cached until the block is fully validated. A rule to prohibit *all* 64 b= yte txs is counterproductive as it only adds additional checks on typically= thousands of txs per block, serving no purpose.

>> It seems t= o me that introducing an arbitrary tx size validity may create more potenti= al implementation bugs than it resolves.
> The potential for implemen= tation bugs is a fair point to raise, but in this case i don't think it's a= big concern. Verifying no transaction in a block is 64 bytes is as simple = a check as you can get.

You appear to be making the assumption that = the check is performed after the block is fully parsed (contrary to your "e= arlier" criterion above). The only way to determine the tx sizes is to pars= e each tx for witness marker, input count, output count, input script sizes= , output script sizes, witness sizes, and skipping over the header, several= constants, and associated buffers. Doing this "early" to detect malleation= is an extraordinarily complex and costly process. On the other hand, as I = pointed out, a rational implementation would only do this early check for t= he coinbase.

Yet even determining the size of the coinbase is signif= icantly more complex and costly than checking its first input point against= null. That check (which is already necessary for validation) resolves the = malleation question, can be performed on the raw unparsed block buffer by s= imply skipping header, version, reading input count and witness marker as n= ecessary, offsetting to the 36 byte point buffer, and performing a byte com= parison against [0000000000000000000000000000000000000000000000000000000000= 000000ffffffff].

This is:

(1) earlier
(2) faster
(3) si= mpler
(4) already consensus

>> And certainly anyone impleme= nting such a verifier must know many intricacies of the protocol.
> T= hey need to know some, but i don't think it's reasonable to expect them to = realize the merkle tree construction is such that an inner node may be conf= used with a 64 bytes transaction.

A protocol developer needs to unde= rstand that the hash of an invalid block cannot be cached unless at least t= he coinbase has been restricted in size (under the proposal) -or- that the = coinbase is a null point (presently or under the proposal). In the latter c= ase the check is already performed in validation, so there is no way a bloc= k would presently be cached as invalid without checking it. The proposal ad= ds a redundant check, even if limited to just the coinbase. [He must also u= nderstand the second type of malleability, discussed below.]

If this= proposed rule was to activate we would implement it in a late stage tx.che= ck, after txs/blocks had been fully deserialized. We would not check it an = all in the case where the block is under checkpoint or milestone ("assume v= alid"). In this case we would retain the early null point malleation check = (along with the hash duplication malleation check) that we presently have, = would validate tx commitments, and commit the block. In other words, the pr= oposal adds unnecessary late stage checks only. Implementing it otherwise w= ould just add complexity and hurt performance.

>> I do not see= this. I see a very ugly perpetual seam which will likely result in unexpec= ted complexities over time.
> What makes you think making 64 bytes tr= ansactions invalid could result in unexpected complexities? And why do you = think it's likely?

As described above, it's later, slower, more comp= lex, unnecessarily broad, and a consensus change. Beyond that it creates an= arbitrary size limit - not a lower or upper bound, but a slice out of the = domain. Discontinuities are inherent complexities in computing. The "unexpe= cted" part speaks for itself.

>> This does not produce unmalle= able block hashes. Duplicate tx hash malleation remains in either case, to = the same effect. Without a resolution to both issues this is an empty promi= se.
> Duplicate txids have been invalid since 2012 (CVE-2012-2459).
I think again here you may have misunderstood me. I was not making a = point pertaining to BIP30. I was referring to the other form of block hash = malleability, which results from duplicating sets of trailing txs in a sing= le block (see previous reference). This malleation vector remains, even wit= h invalid 64 byte txs. As I pointed out, this has the "same effect" as the = 64 byte tx issue. Merkle hashing the set of txs is insufficient to determin= e identity. In one case the coinbase must be checked (null point or size) a= nd in the other case the set of tx hashes must be checked for trailing dupl= icated sets. [Core performs this second check within the Merkle hashing alg= orithm (with far more comparisons than necessary), though this can be perfo= rmed earlier and independently to avoid any hashing in the malleation case.= ]

I would also point out in the interest of correctness that Core re= verted its BIP30 soft fork implementation as a consequence of the BIP90 har= d fork, following and requiring the BIP34 soft fork that presumably preclud= ed it but didn't, so it is no longer the case that duplicate tx hashes are = invalid in implementation. As you have proposed in this rollup, this requir= es fixing again.

> If 64 bytes transactions are also made invalid= , this would make it impossible for two valid blocks to have the same hash.=

Aside from the BIP30/34/90 issue addressed above, it is already "im= possible" (cannot be stronger than computationally infeasible) for two *val= id* blocks to have the same hash. The proposal does not enable that objecti= ve, it is already the case. No malleated block is a valid block.

The= proposal aims only to make it earlier or easier or faster to check for blo= ck hash malleation. And as I've pointed out above, it doesn't achieve those= objectives. Possibly the perception that this would be the case is a conse= quence of implementation details, but as I have shown above, it is not in f= act the case.

Given either type of malleation, the malleated block c= an be determined to be invalid by a context free check. But this knowledge = cannot ever be cached against the block hash, since the same hash may be va= lid. Invalidity can only be cached once a non-mallied block is validated an= d determined to be invalid. Block hash malleations are and will remain inva= lid blocks with or without the proposal, and it will continue to be necessa= ry to avoid caching invalid against the malleation. As you said:

>= ; it was about being able to cache the hash of a (non-malleated) invalid bl= ock as permanently invalid to avoid re-downloading and re-validating it.
This is already the case, and requires validating the full non-malleat= ed block. Adding a redundant invalidity check doesn't improve this in any w= ay.

Best,
Eric

--
You received this message because you are subscribed to the Google Groups "= Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googl= egroups.com.
To view this discussion on the web visit https://groups.= google.com/d/msgid/bitcoindev/9a4c4151-36ed-425a-a535-aa2837919a04n%40googl= egroups.com.

--
You received this message because you are subscribed to the Google Groups &= quot;Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoind= ev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/b= itcoindev/wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFeOZT7sHDlyrXw= J5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw%3D%40protonmail.com.
--b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8--