[Bitcoin-development] Version bytes

[Bitcoin-development] Version bytes

[Pieter Wuille]

Hello everyone,

after a discussion on IRC, we decided to try to standardize the version bytes
used by bitcoin for several applications.

There are 3 components that seem meaningful:
* network? (realnet, testnet, alternate chains?)
* data class? (address, private key, master key, ...?)
* version? (real version, per data class defined)

There is no technical reason why different network and different data classes
would need separate version bytes, but i think it is a good thing to keep
them from colliding. People will mix them up, and when things are well
defined, a nice warning message could help a lot ("Oops it seems you entered
a private key instead of an address!").

So, first of all, there is already one actually used alternate chain, namely
namecoin, using version byte 52 for addresses. For this reason, i'd like to
reserve bit 16 in the version byte for "alternate chain". When bit 16 is set,
everything is up to the network itself, and no further semantics are defined.

When bit 16 isn't set:

Then remains the rest of the network. The problem is that testnet already uses
version 111, which is not a single bit. We can use a trick though, namely
choosing bit 1 for testnet, and if bit 1 is set, XOR the rest of the version
number with 111. Otherwise, we could reset testnet (not actually reset, just
change its addresses a bit), and simply state odd=testnet, even=realnet.

That leaves use with 6 more bits to play with, namely 128,64,32 and 8,4,2.
As 128 is already used for private keys, let's use (128,64,32) for data classes,
and (8,4,2) for versions.

So, in full:
* Bits 128/64/32 define data class
** 0 = address
** 32,64,96,160,192 = reserved for future use
** 128 = private key
** 224 = extended data class, another "data class" byte follows
* Bit 16 defines "private"
** 0 = bitcoin
** 16 = alternate chain
* Bits 8/4/2 define version number
** 0 = only thing used for now
** 2,4,6,8,10,12 = reserved for future use
** 14 = extended version, another version byte follows
* Bit 1 defines testnet
** 0 = realnet
** 1 = testnet (possibly using XOR 111, if not reset)

This whole discussion started when Stefan wanted to define a format for master keys from which
to derive deterministic wallet keys, i suggest using data class 192 for that, leaving the
lower numbers for more basic data, like public keys.

Any comments?

-- 
Pieter

Re: [Bitcoin-development] Version bytes

[Pieter Wuille]

On Thu, Jul 07, 2011 at 01:15:57PM +0200, Pieter Wuille wrote:
> Hello everyone,
> 
> after a discussion on IRC, we decided to try to standardize the version bytes
> used by bitcoin for several applications.

I realize my mail may have been a bit unclear. This is about the version bytes
used in addresses and other base58-encoded data structures. I'd like to see some
convention adopted before everyone starts defining their own.

The proposal in the previous mail could be summarized by the following functions
(for non-alternate chains). It is compatible with all currently-used version bytes
that i know of (testnet, realnet, addresses, private keys, namecoin, multicoin):

enum dataclass_t 
{
    address = 0,
    privkey = 4,
    masterkey = 6,
    extended = 7
}

int EncodeVersionByte(dataclass_t class, int nVersion, bool fTestNet)
{
    return (class << 5 + nVersion << 1) ^ fTestNet*111;
}

void DecodeVersionByte(int nByte, dataclass_t& class, int& nVersion, bool& fTestNet)
{
    fTestNet = false;
    if (nByte & 1)
    {
        fTestNet = true;
        nByte ^= 111;
    }
    class = (nByte & 224) >> 5;
    nVersion = (nByte & 14) >> 1;
}

-- 
Pieter

Re: [Bitcoin-development] Version bytes

[Stefan Thomas]

Hey Pieter,

> Otherwise, we could reset testnet (not actually reset, just
> change its addresses a bit), and simply state odd=testnet, even=realnet.

We could use the XOR hack for now and remove it the next time we reset 
testnet. But I do think the 111 is baggage we want to get rid of. Using 
the lsb as a simple flag is much cleaner.

Cheers,

Stefan


On 7/7/2011 1:15 PM, Pieter Wuille wrote:
> Hello everyone,
>
> after a discussion on IRC, we decided to try to standardize the version bytes
> used by bitcoin for several applications.
>
> There are 3 components that seem meaningful:
> * network? (realnet, testnet, alternate chains?)
> * data class? (address, private key, master key, ...?)
> * version? (real version, per data class defined)
>
> There is no technical reason why different network and different data classes
> would need separate version bytes, but i think it is a good thing to keep
> them from colliding. People will mix them up, and when things are well
> defined, a nice warning message could help a lot ("Oops it seems you entered
> a private key instead of an address!").
>
> So, first of all, there is already one actually used alternate chain, namely
> namecoin, using version byte 52 for addresses. For this reason, i'd like to
> reserve bit 16 in the version byte for "alternate chain". When bit 16 is set,
> everything is up to the network itself, and no further semantics are defined.
>
> When bit 16 isn't set:
>
> Then remains the rest of the network. The problem is that testnet already uses
> version 111, which is not a single bit. We can use a trick though, namely
> choosing bit 1 for testnet, and if bit 1 is set, XOR the rest of the version
> number with 111. Otherwise, we could reset testnet (not actually reset, just
> change its addresses a bit), and simply state odd=testnet, even=realnet.
>
> That leaves use with 6 more bits to play with, namely 128,64,32 and 8,4,2.
> As 128 is already used for private keys, let's use (128,64,32) for data classes,
> and (8,4,2) for versions.
>
> So, in full:
> * Bits 128/64/32 define data class
> ** 0 = address
> ** 32,64,96,160,192 = reserved for future use
> ** 128 = private key
> ** 224 = extended data class, another "data class" byte follows
> * Bit 16 defines "private"
> ** 0 = bitcoin
> ** 16 = alternate chain
> * Bits 8/4/2 define version number
> ** 0 = only thing used for now
> ** 2,4,6,8,10,12 = reserved for future use
> ** 14 = extended version, another version byte follows
> * Bit 1 defines testnet
> ** 0 = realnet
> ** 1 = testnet (possibly using XOR 111, if not reset)
>
> This whole discussion started when Stefan wanted to define a format for master keys from which
> to derive deterministic wallet keys, i suggest using data class 192 for that, leaving the
> lower numbers for more basic data, like public keys.
>
> Any comments?
>

Re: [Bitcoin-development] Version bytes

[John Smith]

[-- Attachment #1: Type: text/plain, Size: 3759 bytes --]

I agree. I think breaking compatiblity with older address (even testnet) is
not a

On Fri, Jul 8, 2011 at 6:36 AM, Stefan Thomas <moon@justmoon.de> wrote:

> Hey Pieter,
>
> > Otherwise, we could reset testnet (not actually reset, just
> > change its addresses a bit), and simply state odd=testnet, even=realnet.
>
> We could use the XOR hack for now and remove it the next time we reset
> testnet. But I do think the 111 is baggage we want to get rid of. Using
> the lsb as a simple flag is much cleaner.
>
> Cheers,
>
> Stefan
>
>
> On 7/7/2011 1:15 PM, Pieter Wuille wrote:
> > Hello everyone,
> >
> > after a discussion on IRC, we decided to try to standardize the version
> bytes
> > used by bitcoin for several applications.
> >
> > There are 3 components that seem meaningful:
> > * network? (realnet, testnet, alternate chains?)
> > * data class? (address, private key, master key, ...?)
> > * version? (real version, per data class defined)
> >
> > There is no technical reason why different network and different data
> classes
> > would need separate version bytes, but i think it is a good thing to keep
> > them from colliding. People will mix them up, and when things are well
> > defined, a nice warning message could help a lot ("Oops it seems you
> entered
> > a private key instead of an address!").
> >
> > So, first of all, there is already one actually used alternate chain,
> namely
> > namecoin, using version byte 52 for addresses. For this reason, i'd like
> to
> > reserve bit 16 in the version byte for "alternate chain". When bit 16 is
> set,
> > everything is up to the network itself, and no further semantics are
> defined.
> >
> > When bit 16 isn't set:
> >
> > Then remains the rest of the network. The problem is that testnet already
> uses
> > version 111, which is not a single bit. We can use a trick though, namely
> > choosing bit 1 for testnet, and if bit 1 is set, XOR the rest of the
> version
> > number with 111. Otherwise, we could reset testnet (not actually reset,
> just
> > change its addresses a bit), and simply state odd=testnet, even=realnet.
> >
> > That leaves use with 6 more bits to play with, namely 128,64,32 and
> 8,4,2.
> > As 128 is already used for private keys, let's use (128,64,32) for data
> classes,
> > and (8,4,2) for versions.
> >
> > So, in full:
> > * Bits 128/64/32 define data class
> > ** 0 = address
> > ** 32,64,96,160,192 = reserved for future use
> > ** 128 = private key
> > ** 224 = extended data class, another "data class" byte follows
> > * Bit 16 defines "private"
> > ** 0 = bitcoin
> > ** 16 = alternate chain
> > * Bits 8/4/2 define version number
> > ** 0 = only thing used for now
> > ** 2,4,6,8,10,12 = reserved for future use
> > ** 14 = extended version, another version byte follows
> > * Bit 1 defines testnet
> > ** 0 = realnet
> > ** 1 = testnet (possibly using XOR 111, if not reset)
> >
> > This whole discussion started when Stefan wanted to define a format for
> master keys from which
> > to derive deterministic wallet keys, i suggest using data class 192 for
> that, leaving the
> > lower numbers for more basic data, like public keys.
> >
> > Any comments?
> >
>
>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4716 bytes --]

Re: [Bitcoin-development] Version bytes

[John Smith]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

Sorry, that last message got broken off.

On Fri, Jul 8, 2011 at 6:36 AM, Stefan Thomas <moon@justmoon.de> wrote:
>
>> Hey Pieter,
>>
>> > Otherwise, we could reset testnet (not actually reset, just
>> > change its addresses a bit), and simply state odd=testnet, even=realnet.
>>
>> We could use the XOR hack for now and remove it the next time we reset
>> testnet. But I do think the 111 is baggage we want to get rid of. Using
>> the lsb as a simple flag is much cleaner.
>>
>

I agree it is cleaner, but I think breaking compatiblity with older address
(even testnet) is not a good idea right now. It is important to build an
image of stability and backward compatibility.

So I vote for the XOR 111 hack :)

JS

[-- Attachment #2: Type: text/html, Size: 1210 bytes --]

Re: [Bitcoin-development] Version bytes

[Pieter Wuille]

On Fri, Jul 08, 2011 at 08:18:19AM +0000, John Smith wrote:
> 
> On Fri, Jul 8, 2011 at 6:36 AM, Stefan Thomas <moon@justmoon.de> wrote:
> >
> >> Hey Pieter,
> >>
> >> > Otherwise, we could reset testnet (not actually reset, just
> >> > change its addresses a bit), and simply state odd=testnet, even=realnet.
> >>
> >> We could use the XOR hack for now and remove it the next time we reset
> >> testnet. But I do think the 111 is baggage we want to get rid of. Using
> >> the lsb as a simple flag is much cleaner.
> 
> I agree it is cleaner, but I think breaking compatiblity with older address
> (even testnet) is not a good idea right now. It is important to build an
> image of stability and backward compatibility.
> 
> So I vote for the XOR 111 hack :)

It does have another advantage: it makes testnet codes visually (after base58
encoding) different from realnet ones, which is probably the reason why the
relatively large number 111 was chosen.

The only small change that can cause the first base58 character to remain equal,
is a modification to nVersion of less than 5 in absolute value.

PS: +/- 111 is also possible, instead of XOR 111.

-- 
Pieter