From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id E8341C002D for ; Thu, 10 Nov 2022 21:24:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 374ED812AE for ; Thu, 10 Nov 2022 21:24:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 374ED812AE Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=wuille.net header.i=@wuille.net header.a=rsa-sha256 header.s=protonmail3 header.b=H9aZ1NKm X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.103 X-Spam-Level: X-Spam-Status: No, score=-2.103 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vb1aDmthmSvP for ; Thu, 10 Nov 2022 21:23:58 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 165C081260 Received: from mail-4317.proton.ch (mail-4317.proton.ch [185.70.43.17]) by smtp1.osuosl.org (Postfix) with ESMTPS id 165C081260 for ; Thu, 10 Nov 2022 21:23:57 +0000 (UTC) Date: Thu, 10 Nov 2022 21:23:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wuille.net; s=protonmail3; t=1668115434; x=1668374634; bh=APQR3hMu//8AOGBUMhX8bnHIkjDJhKcllWowW/9O0NA=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=H9aZ1NKm6wS5C5z+NuG8C/AaIU8Y5jR50uhzVBf9ZYcr+Mv5PpnX8PGTap/5OSP4x dzhTJSZso4OM8pHkIvgoawtheTzXXCHGH0jnDLajWfp0s0y5/b10PPvYRdqk0+Xfhx YHvcpu1BxrdRz6P6SwLHBMwf0hayBai8Z7o+ylrMe2H1T9o6puGpSzhhSeipcBvytf OCOOSgcl2Nf1IKNdnYCoeNGptci7e57r5b5lUlGK4XVxRkYOQv3/+3xAUYggPZqSWO ySc48llOH9IzmHoQvisM1cqV1NlH3nhVLpcz5d2LaaAK2joMfndGHnAH09lXAB9+1L VONpJchFC9IXg== To: Bitcoin Protocol Discussion From: Pieter Wuille Message-ID: In-Reply-To: References: <56677685-619a-691f-d5bc-54b69fdb6ed2@bip324.com> Feedback-ID: 19463299:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 10 Nov 2022 21:24:35 +0000 Subject: Re: [bitcoin-dev] Refreshed BIP324 X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2022 21:24:01 -0000 Hi, Thanks for the comments so far. I think these are all reasonable ideas. Comments inline below. On Thursday, November 3rd, 2022 at 1:53 PM, Murch wrote: > From what I understand we'll have about 35 message types on the network > with the addition of BIP324. 256 possible IDs sounds like plenty room to > grow, but perhaps we can be a bit more conservative: >=20 > We could use the first bit to signal a 2-byte message ID. That allows us > to express 128 IDs with 1 byte, but if we need more, we get a total of > 2^15 IDs across 2 bytes. Yeah, effectively treating the first 1 or 2 bytes as a simple variable length integer is a nice way of increasing the space at low cost. This also doesn't need to be decided now. The initial approach could just be avoiding allocating bytes in the 128-255 range until the need for more space arises. If and when that is the case, the choice could be to: * Just continue treating the first byte as the command. * Start treating the first upper bit as a sign that another command byte follows. * Switch to some form of explicit signalling (option 3 is my earlier mail). On Thursday, November 3rd, 2022 at 6:26 PM, Jonas Schnelli wrote: > There would be an alternative to preserve more 1 byte IDs on the cost > of a (much) smaller 2 byte ID space: Reserve the short ID 0xFF as an > indication for a 2 bytes short ID (additional 256 short IDs with 2 bytes)= . I don't think this is needed, because we arguably already have that! If the first byte is 0x01, then 1 more command byte follows in the current BIP324 draft. That mechanism is designed for alphabetic 1-character commands, but nothing prevents it from also being used for other things (by using a non-alphabetic byte there). > Maybe the BIP should state that only frequent sent messages should reserv= e > a short ID, though, the BIP itself assigns short IDs to all(?) message > types (including low frequent messages like SENDHEADERS). >=20 > Maybe exclude message types that expected to be only sent once from > assigning a short ID? I think that makes sense. Especially in combination with the idea avoiding bytes with the upper bit set there is a bit more pressure on the 1-byte space. Rarely-sent or at-most-once-sent commands don't really provide much benefit. I'd suggest scrapping from the list: * Version messages: version, verack * Negotiation messages: sendaddrv2, sendheaders, sendcmpct, wtxidrelay * Rarely-sent messages: mempool I'm not sure to what extent filteradd/filterload/filterclear/merkleblock are still actually used; perhaps they could be removed too? On Monday, November 7th, 2022 at 10:20 PM, Anthony Towns wrote: > I guess I think it would make sense to not start using a novel 1-byte > message unless you've done something to introduce that message first; > whether that's via approach (3) ("I'm going to use 0xE9 to mean pkgtxns") > or via a multibyte feature support message ("I sent sendaddrv3 as a > 10-byte message, that implies 0xA3 means addrv3 from now on"). That's fair, but I don't think it matters too much for allocation purposes; protocol designs should still not assign overlapping values, unless the protocols are known to never be used simultaneously? Unless... the assignment works like "whenever the sendaddrv3 is sent, the next available byte in range 48..127 gets allocated for addrv3". That means no explicit mapping is needed, as long as the total number of messages from simultaneously-active extensions isn't too large. > I do still think it'd be better to recommend against reserving a byte for > one-shot messages, and not do it for existing one-shot messages though. Agree. FWIW, if anyone was wondering about how much is actually saved by having 1-byte commands vs 12-byte commands, I've gathered statistics from two node= s (one with many inbound connections, one only outbound) for two weeks. This = is obviously very dependent on network topology and local implementation choic= es, but it may still give an idea: * Outbound-only node: * Around 4.5% of sent bytes are bytes 2-12 of the command. * Sent 979.98 MiB in total. * Outbound and inbound node: * Around 1.6% of sent bytes are bytes 2-12 of the command. * Sent 124.14 GiB in total. Cheers, --=20 Pieter