Re: [bitcoin-dev] Refreshed BIP324

[Pieter Wuille <bitcoin-dev@wuille.net>]

Hi,

Thanks for the comments so far. I think these are all reasonable ideas.
Comments inline below.

On Thursday, November 3rd, 2022 at 1:53 PM, Murch wrote:

> From what I understand we'll have about 35 message types on the network
> with the addition of BIP324. 256 possible IDs sounds like plenty room to
> grow, but perhaps we can be a bit more conservative:
> 
> We could use the first bit to signal a 2-byte message ID. That allows us
> to express 128 IDs with 1 byte, but if we need more, we get a total of
> 2^15 IDs across 2 bytes.

Yeah, effectively treating the first 1 or 2 bytes as a simple variable
length integer is a nice way of increasing the space at low cost.

This also doesn't need to be decided now. The initial approach could just
be avoiding allocating bytes in the 128-255 range until the need for more
space arises. If and when that is the case, the choice could be to:
* Just continue treating the first byte as the command.
* Start treating the first upper bit as a sign that another command byte
  follows.
* Switch to some form of explicit signalling (option 3 is my earlier
  mail).

On Thursday, November 3rd, 2022 at 6:26 PM, Jonas Schnelli wrote:

> There would be an alternative to preserve more 1 byte IDs on the cost
> of a (much) smaller 2 byte ID space: Reserve the short ID 0xFF as an
> indication for a 2 bytes short ID (additional 256 short IDs with 2 bytes).

I don't think this is needed, because we arguably already have that! If the
first byte is 0x01, then 1 more command byte follows in the current BIP324
draft. That mechanism is designed for alphabetic 1-character commands, but
nothing prevents it from also being used for other things (by using a
non-alphabetic byte there).

> Maybe the BIP should state that only frequent sent messages should reserve
> a short ID, though, the BIP itself assigns short IDs to all(?) message
> types (including low frequent messages like SENDHEADERS).
> 
> Maybe exclude message types that expected to be only sent once from
> assigning a short ID?

I think that makes sense. Especially in combination with the idea avoiding
bytes with the upper bit set there is a bit more pressure on the 1-byte
space. Rarely-sent or at-most-once-sent commands don't really provide much
benefit. I'd suggest scrapping from the list:
* Version messages: version, verack
* Negotiation messages: sendaddrv2, sendheaders, sendcmpct, wtxidrelay
* Rarely-sent messages: mempool

I'm not sure to what extent filteradd/filterload/filterclear/merkleblock
are still actually used; perhaps they could be removed too?

On Monday, November 7th, 2022 at 10:20 PM, Anthony Towns wrote:

> I guess I think it would make sense to not start using a novel 1-byte
> message unless you've done something to introduce that message first;
> whether that's via approach (3) ("I'm going to use 0xE9 to mean pkgtxns")
> or via a multibyte feature support message ("I sent sendaddrv3 as a
> 10-byte message, that implies 0xA3 means addrv3 from now on").

That's fair, but I don't think it matters too much for allocation purposes;
protocol designs should still not assign overlapping values, unless the
protocols are known to never be used simultaneously?

Unless... the assignment works like "whenever the sendaddrv3 is sent, the
next available byte in range 48..127 gets allocated for addrv3". That means
no explicit mapping is needed, as long as the total number of messages from
simultaneously-active extensions isn't too large.

> I do still think it'd be better to recommend against reserving a byte for
> one-shot messages, and not do it for existing one-shot messages though.

Agree.

FWIW, if anyone was wondering about how much is actually saved by having
1-byte commands vs 12-byte commands, I've gathered statistics from two nodes
(one with many inbound connections, one only outbound) for two weeks. This is
obviously very dependent on network topology and local implementation choices,
but it may still give an idea:
* Outbound-only node:
  * Around 4.5% of sent bytes are bytes 2-12 of the command.
  * Sent 979.98 MiB in total.
* Outbound and inbound node:
  * Around 1.6% of sent bytes are bytes 2-12 of the command.
  * Sent 124.14 GiB in total.

Cheers,

-- 
Pieter