[Bitcoin-development] Export format for xpub

[Bitcoin-development] Export format for xpub

[Levin Keller]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

Hello everyone,

I think this is my first email to this mailinglist so I will shortly
introduce myself:

I am Levin and the CEO of Coyno (www.coyno.com). Based in Berlin,
mathematician. Bitcoiner since 2011.

And now the reason for this email: Andreas (Schildbach) just released a new
update of his wallet. It now provides an export functionality for the m/0'
key in order to run read only copies on other devices. We already support
the format on our website. Of course we would love for this to become
standard. I also updated the Wiki article for Andreas' Wallet:
https://en.bitcoin.it/wiki/Bitcoin_Wallet

How do you like it? How does this format get standard? Shall I try to get a
pull request to BIP32 passed?

Cheers

Levin

[-- Attachment #2: Type: text/html, Size: 997 bytes --]

Re: [Bitcoin-development] Export format for xpub

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2536 bytes --]

We generally don't edit BIPs like that after they've been written except to
add helpful links, examples etc and other things that don't add new
functionality. For this you'd write a new BIP. It doesn't have to be hard.
The process is:

1) Adapt the template BIP and fill it out with your motivation, design,
rationale and ideally some examples.

2) Post it here and ask Gregory for a BIP number. He will select one
through some magic algorithm I am still reverse engineering ;)

3) People will give feedback and try to spot problems in your spec.

I looked at what Andreas is doing a few days ago - basically it's just an
xpub string with a ?a=b&c=d query string added to the end, with a hierarchy
type specifier and something else I forgot, encoded into a QR code. So it
should be a very easy BIP to add.

Whilst you're at it you might want to add an HTTP POST based method,
though. Web apps scanning QR codes is kind of clunky compared to just
picking Coyno from a list in the wallet app and having it all
auto-magically activate.

On Mon, Feb 2, 2015 at 9:56 AM, Levin Keller <post@levinkeller.de> wrote:

> Hello everyone,
>
> I think this is my first email to this mailinglist so I will shortly
> introduce myself:
>
> I am Levin and the CEO of Coyno (www.coyno.com). Based in Berlin,
> mathematician. Bitcoiner since 2011.
>
> And now the reason for this email: Andreas (Schildbach) just released a
> new update of his wallet. It now provides an export functionality for the
> m/0' key in order to run read only copies on other devices. We already
> support the format on our website. Of course we would love for this to
> become standard. I also updated the Wiki article for Andreas' Wallet:
> https://en.bitcoin.it/wiki/Bitcoin_Wallet
>
> How do you like it? How does this format get standard? Shall I try to get
> a pull request to BIP32 passed?
>
> Cheers
>
> Levin
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is
> your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 3550 bytes --]

Re: [Bitcoin-development] Export format for xpub

[Wladimir]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1022 bytes --]


On Mon, 2 Feb 2015, Levin Keller wrote:

> Hello everyone,
> I think this is my first email to this mailinglist so I will shortly introduce myself:
> 
> I am Levin and the CEO of Coyno (www.coyno.com). Based in Berlin, mathematician. Bitcoiner since 2011.
> 
> And now the reason for this email: Andreas (Schildbach) just released a new update of his wallet. It now provides an export functionality for the m/0' key in order to run read only copies
> on other devices. We already support the format on our website. Of course we would love for this to become standard. I also updated the Wiki article for Andreas'
> Wallet: https://en.bitcoin.it/wiki/Bitcoin_Wallet

Yes, standardizing on a format could be useful.

> How do you like it? How does this format get standard? Shall I try to get a pull request to BIP32 passed?

Just administrative trivia: this would be a new BIP, and not an 
amandment to BIP32. Excluding small language errors and 
clarifications in examples, BIPs are not changed after the fact.

Wladimir

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/02/2015 12:33 PM, Mike Hearn wrote:

> I looked at what Andreas is doing a few days ago - basically it's just
> an xpub string with a ?a=b&c=d query string added to the end, with a
> hierarchy type specifier and something else I forgot, encoded into a QR
> code.

It's h=bip32 for the hierarchy used and
c=123456 for the creation date of the wallet (in seconds since epoch).

I pondered about if we should add a scheme (e.g. "bitcoin-xpub:") but
decided to start without. The only advantage I currently see would be it
could be used on Android to dispatch intents to the right app(s).

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 02/02/15 12:33, Mike Hearn wrote:
> We generally don't edit BIPs like that after they've been written except to

I think this could end up in BIP43, which deals with hierarchies and is
still in Draft state although widely used. Allocating new BIP seems like
a overkill to me.


-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 02/02/15 13:38, Andreas Schildbach wrote:
> It's h=bip32 for the hierarchy used and

Do I understand this correctly and h=bip32 hierarchy means that both

xpub/0/i and xpub/1/j chains are scanned? (So it applies to BIP44
generated xpubs as well?)

> c=123456 for the creation date of the wallet (in seconds since epoch).

Uff, I would expect YYYYMMDD there so it's human readable as well.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/02/2015 01:59 PM, Pavol Rusnak wrote:
>> It's h=bip32 for the hierarchy used and
>
> Do I understand this correctly and h=bip32 hierarchy means that both
>
> xpub/0/i and xpub/1/j chains are scanned? (So it applies to BIP44
> generated xpubs as well?)

Yes, except that BIP32-hierarchy and BIP44 differ in some requirements
(e.g. gap limit).

>> c=123456 for the creation date of the wallet (in seconds since epoch).
>
> Uff, I would expect YYYYMMDD there so it's human readable as well.

Those strings are not meant to be read by humans. YYYYMMDD is more
complicated than necessary, given that Bitcoin deals with seconds since
epoch everywhere.

Re: [Bitcoin-development] Export format for xpub

[vv01f]

On 02.02.2015 15:17, Andreas Schildbach wrote:
>> Uff, I would expect YYYYMMDD there so it's human readable as well.
> 
> Those strings are not meant to be read by humans. YYYYMMDD is more
> complicated than necessary, given that Bitcoin deals with seconds since
> epoch everywhere.

First that is a pitty .. as its simply a waste of storage.

but back to Pavol's point: IMHO no harm to anything, as Bitcoin never
has any valid timestamp below ~1230768000 (jan2009) and thus will always
have 10 digits.. you can easily identify 8 char long timestamp as the
proposed format.
And there never is anything wrong with having a transparent, human
readable option - especially when it saves 2 bytes in e.g. qr-codes.

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 02/02/15 15:17, Andreas Schildbach wrote:
> Yes, except that BIP32-hierarchy and BIP44 differ in some requirements
> (e.g. gap limit).

Right.

To me it seems more important to describe how addresses should be
discovered (i.e. to scan xpub/0/i and xpub/1/j chains using gap limit G)
instead of how the xpub was created/obtained (bip32 vs bip44).

What do you thing about changing ?h=bip32 to something like

?t=01&g=20

- t=01 meaning that chains 0 and 1 should be scanned (feel free to
change "01" into any other descriptive string)
- g=20 meaning that gap 20 should be used

> Those strings are not meant to be read by humans. YYYYMMDD is more
> complicated than necessary, given that Bitcoin deals with seconds since
> epoch everywhere.

OK :-)

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/02/2015 03:47 PM, vv01f wrote:

>>> Uff, I would expect YYYYMMDD there so it's human readable as well.
>>
>> Those strings are not meant to be read by humans. YYYYMMDD is more
>> complicated than necessary, given that Bitcoin deals with seconds since
>> epoch everywhere.
> 
> First that is a pitty .. as its simply a waste of storage.
> 
> but back to Pavol's point: IMHO no harm to anything, as Bitcoin never
> has any valid timestamp below ~1230768000 (jan2009) and thus will always
> have 10 digits.. you can easily identify 8 char long timestamp as the
> proposed format.
> And there never is anything wrong with having a transparent, human
> readable option - especially when it saves 2 bytes in e.g. qr-codes.

Pavol's suggestion saves 2 chars only because its just a date. I think
the creation date should be at least precise to the hour, if not to the
minute.

But anyhow, if everyone prefers a human readble date format I will bow
to the majority.

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/02/2015 03:56 PM, Pavol Rusnak wrote:

> To me it seems more important to describe how addresses should be
> discovered (i.e. to scan xpub/0/i and xpub/1/j chains using gap limit G)
> instead of how the xpub was created/obtained (bip32 vs bip44).
> 
> What do you thing about changing ?h=bip32 to something like
> 
> ?t=01&g=20
> 
> - t=01 meaning that chains 0 and 1 should be scanned (feel free to
> change "01" into any other descriptive string)
> - g=20 meaning that gap 20 should be used

I don't think that parameterizing will work, we can't predict future
BIPs. It's the same as for BIP43, in the end we agreed on just putting
the BIP number.

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 03/02/15 01:05, Andreas Schildbach wrote:
> I don't think that parameterizing will work, we can't predict future
> BIPs. It's the same as for BIP43, in the end we agreed on just putting
> the BIP number.

Hm, let me put the questions the other way around:

What gap limit should a wallet use if it encounters h=bip32?

What h value should I use for myTREZOR wallets? Which is essentially a
BIP44 wallet that produces h=bip32 xpubs with gap limit 20 ...

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] Export format for xpub

[Levin Keller]

[-- Attachment #1: Type: text/plain, Size: 2525 bytes --]

Why even bother with the specific HD scheme such as BIP32  or BIP44. What
are the interesting parameters?

Required:

   - gap limit

Optional:

   - which node of the derivation chain is actually exported (m0' for
   BIP32, m44'0'account' for BIP44)
   - which subnodes are used for external and internal purposes
   - creation date

To import the data in a read only application it is not important which
node one actually gets and in all implementations the subnode of the
exported node "0" is used for external addresses and "1" for internal
addresses.

There is no usecase to export any higher node than m0' in BIP32 or
m44'0'account' as one can only derive any child nodes of the higher nodes *with
the private master key*. As for lower nodes (like further down the path)
there is also no need to export because in all implementations today they
will only give around half of the used addresses.

So I think a more general but very useful export scheme would be:

bitcoin-pub-export:xpub[gibberish]?gaplimit=[number]&path=[path in
derivation tree]&subchains=[numbers]&creationdate=[unixtimestamp]

Why not have more descriptive parameters? Saving on data?

I am a big fan of unix timestamps. Would vote for Andreas' format on the
creation date.

Cheers

Levin

2015-02-03 1:22 GMT+01:00 Pavol Rusnak <stick@gk2.sk>:

> On 03/02/15 01:05, Andreas Schildbach wrote:
> > I don't think that parameterizing will work, we can't predict future
> > BIPs. It's the same as for BIP43, in the end we agreed on just putting
> > the BIP number.
>
> Hm, let me put the questions the other way around:
>
> What gap limit should a wallet use if it encounters h=bip32?
>
> What h value should I use for myTREZOR wallets? Which is essentially a
> BIP44 wallet that produces h=bip32 xpubs with gap limit 20 ...
>
> --
> Best Regards / S pozdravom,
>
> Pavol Rusnak <stick@gk2.sk>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is
> your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 3557 bytes --]

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 03/02/15 10:33, Levin Keller wrote:
> bitcoin-pub-export:xpub[gibberish]?gaplimit=[number]&path=[path in
> derivation tree]&subchains=[numbers]&creationdate=[unixtimestamp]

I cannot come up with an usecase where "path" parameter would be needed.
FWIW childnumber and depth are already expressed in xpub itself.

I like the general idea of "subchains" parameter, but I would like to
further specify it:

a) parameter should contain values described as comma separated
   list of values (such as 0,1,2,3,4)

b) consecutive values can be shortened via dash (0,1,2,3 == 0-3)

c) should we allow non-consecutive values (e.g. 0,1,3,8)?
   I am not sure. If not the "subchains" param can contain just upper
   bound of indexes to scan (e.g. "3")

d) a wallet uses just the first specified chain to generate receiving
   addresses, uses the other chains just to add to the balance

   OR should a wallet be able to generate receiving address for second,
   third, etc. external chain? if yes, we should split "subchains" param
   into "external" and "internal" params both containing a list of
   numbers. this seems like an overkill to me and I am fine with using
   just the first chain as the external one.

> Why not have more descriptive parameters? Saving on data?

Yes. The longer the string, the bigger the QR code.

> I am a big fan of unix timestamps. Would vote for Andreas' format on the
> creation date.

I am not against Unix timestamps, I just said I expected something else
there. Unix timestamps have a lot of advantages. Another option that
might make sense is the block number.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/03/2015 01:22 AM, Pavol Rusnak wrote:

> Hm, let me put the questions the other way around:
> 
> What gap limit should a wallet use if it encounters h=bip32?

It should follow the spec. I know BIP32-hierarchy is short on gap
limits, which is why (amongst other reasons) I expect
BIP32-hierarchy-based wallets migrate to a better standard at some time.

> What h value should I use for myTREZOR wallets? Which is essentially a
> BIP44 wallet that produces h=bip32 xpubs with gap limit 20 ...

If it follows BIP32, h=bip32 is fine.

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/03/2015 10:33 AM, Levin Keller wrote:

> Why not have more descriptive parameters? Saving on data?

Yes. QR codes are very size sensitive.

Re: [Bitcoin-development] Export format for xpub

[Andreas Schildbach]

On 02/03/2015 11:10 AM, Pavol Rusnak wrote:

> Another option that might make sense is the block number.

Not really IMHO. Keys can be used on multiple blockchains.

Re: [Bitcoin-development] Export format for xpub

[Pavol Rusnak]

On 03/02/15 11:37, Andreas Schildbach wrote:
> Not really IMHO. Keys can be used on multiple blockchains.

Ah, correct. Timestamp it is.

Nitpick: They cannot be used on multiple blockchains according to BIP32.
In BIP43 we fixed that. :-)

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>