[Bitcoin-development] New BIP32 structure

[Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2322 bytes --]

Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
our BIP32 wallet structures would be compatible - and I discovered that
only I was planning to use the default structure.

Because I'm hopeful that we can get a lot of interoperability between
wallets with regards to importing 12-words paper wallets, we brainstormed
to find a structure acceptable to everyone and ended up with:

  /m/cointype/reserved'/account'/change/n

The extra levels require some explanation:

   - cointype:  This is zero for Bitcoin. This is here to support two
   things, one is supporting alt coins based off the same root seed. Right now
   nobody seemed very bothered about alt coins but sometimes feature requests
   do come in for this. Arguably there is no need and alt coins could just use
   the same keys as Bitcoin, but it may help avoid confusion if they don't.

   More usefully, cointype can distinguish between keys intended for things
   like multisig outputs, e.g. for watchdog services. This means if your
   wallet does not know about the extra protocol layers involved in this, it
   can still import the "raw" money and it will just ignore/not see the keys
   used in more complex transactions.

   - reserved is for "other stuff". I actually don't recall why we ended up
   with this. It may have been intended to split out multisig outputs etc from
   cointype. Marek, Thomas?

   - account is for keeping essentially wallets-within-a-wallet to avoid
   mixing of coins. If you want that.

   - change is 0 for receiving addresses, 1 for change addresses.

   - n is the actual key index

For bitcoinj we're targeting a deliberately limited feature set for hdw v1
so I would just set the first three values all to zero and that is a
perfectly fine way to be compatible.

The goal here is that the same seed can be written down once, and meet all
the users needs, whilst still allowing some drift between what wallets
support.

Pieter made the I think valid point that you can't really encode how keys
are meant to be used into just an HDW hierarchy and normally you'd need
some metadata as well. However, I feel interop between wallets is more
important than arriving at the most perfect possible arrangement, which
feels a little like bikeshedding, so I'm happy to just go with the flow on
this one.

[-- Attachment #2: Type: text/html, Size: 3280 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Andreas Schildbach]

Thanks for starting the discussion on finding a better structure.

For me, the most important thing is either we're 100% interoperable or
0%. There should not be anything inbetween, as users will delete seeds
without knowing there is still money in them on another implementation.
I heard from multiple sources that using this standard some wallets will
only see a subset of the addresses/keys of some other wallets.
Implementation differences can always happen (and should addresses as
bugs), but I think its unacceptable that this source of issues is by design.

I suggest we agree on an even simpler least common denominator and
wallets that want to implement some feature on top of that can do but
are encouraged to pick a totally different "cointype". I guess that
would mean removing reserved and account.

I'm still thinking it might be a good idea to have a separate chain for
"refunds". Refunds will be rarely used and thus need a much slower
moving window than receiving addresses or change.


On 03/26/2014 09:49 PM, Mike Hearn wrote:
> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
> our BIP32 wallet structures would be compatible - and I discovered that
> only I was planning to use the default structure.
> 
> Because I'm hopeful that we can get a lot of interoperability between
> wallets with regards to importing 12-words paper wallets, we
> brainstormed to find a structure acceptable to everyone and ended up with:
> 
>   /m/cointype/reserved'/account'/change/n
> 
> The extra levels require some explanation:
> 
>   * cointype:  This is zero for Bitcoin. This is here to support two
>     things, one is supporting alt coins based off the same root seed.
>     Right now nobody seemed very bothered about alt coins but sometimes
>     feature requests do come in for this. Arguably there is no need and
>     alt coins could just use the same keys as Bitcoin, but it may help
>     avoid confusion if they don't.
> 
>     More usefully, cointype can distinguish between keys intended for
>     things like multisig outputs, e.g. for watchdog services. This means
>     if your wallet does not know about the extra protocol layers
>     involved in this, it can still import the "raw" money and it will
>     just ignore/not see the keys used in more complex transactions.
> 
>   * reserved is for "other stuff". I actually don't recall why we ended
>     up with this. It may have been intended to split out multisig
>     outputs etc from cointype. Marek, Thomas?
> 
>   * account is for keeping essentially wallets-within-a-wallet to avoid
>     mixing of coins. If you want that.
> 
>   * change is 0 for receiving addresses, 1 for change addresses.
> 
>   * n is the actual key index
> 
> For bitcoinj we're targeting a deliberately limited feature set for hdw
> v1 so I would just set the first three values all to zero and that is a
> perfectly fine way to be compatible.
> 
> The goal here is that the same seed can be written down once, and meet
> all the users needs, whilst still allowing some drift between what
> wallets support.
> 
> Pieter made the I think valid point that you can't really encode how
> keys are meant to be used into just an HDW hierarchy and normally you'd
> need some metadata as well. However, I feel interop between wallets is
> more important than arriving at the most perfect possible arrangement,
> which feels a little like bikeshedding, so I'm happy to just go with the
> flow on this one.
> 
> 
> 
> ------------------------------------------------------------------------------
> 
> 
> 
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 

Re: [Bitcoin-development] New BIP32 structure

[Alan Reiner]

[-- Attachment #1: Type: text/plain, Size: 4868 bytes --]

This might be tangential, but the comment about "refund" chains reminded
me.  Armory will be implementing multi-sig/linked wallets where a each
device has a parallel HDW branch and produces P2SH addresses.  For those
types of wallets, I plan to allocate two chains /per signing
authority/.  If you have a shared 2-of-2 wallet split between your phone
and your spouse's phone, your phone would distribute addresses on P2SH
chain 0 and generate change addresses on P2SH chain 1.  Your spouse's
phone would use chains 2 and 3.

So if you and your spouse switch to a new app that supports M-of-N
linked wallets, it should search for coin history along the first 2*N
chains.
-Alan



On 03/26/2014 07:37 PM, Andreas Schildbach wrote:
> Thanks for starting the discussion on finding a better structure.
>
> For me, the most important thing is either we're 100% interoperable or
> 0%. There should not be anything inbetween, as users will delete seeds
> without knowing there is still money in them on another implementation.
> I heard from multiple sources that using this standard some wallets will
> only see a subset of the addresses/keys of some other wallets.
> Implementation differences can always happen (and should addresses as
> bugs), but I think its unacceptable that this source of issues is by design.
>
> I suggest we agree on an even simpler least common denominator and
> wallets that want to implement some feature on top of that can do but
> are encouraged to pick a totally different "cointype". I guess that
> would mean removing reserved and account.
>
> I'm still thinking it might be a good idea to have a separate chain for
> "refunds". Refunds will be rarely used and thus need a much slower
> moving window than receiving addresses or change.
>
>
> On 03/26/2014 09:49 PM, Mike Hearn wrote:
>> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
>> our BIP32 wallet structures would be compatible - and I discovered that
>> only I was planning to use the default structure.
>>
>> Because I'm hopeful that we can get a lot of interoperability between
>> wallets with regards to importing 12-words paper wallets, we
>> brainstormed to find a structure acceptable to everyone and ended up with:
>>
>>   /m/cointype/reserved'/account'/change/n
>>
>> The extra levels require some explanation:
>>
>>   * cointype:  This is zero for Bitcoin. This is here to support two
>>     things, one is supporting alt coins based off the same root seed.
>>     Right now nobody seemed very bothered about alt coins but sometimes
>>     feature requests do come in for this. Arguably there is no need and
>>     alt coins could just use the same keys as Bitcoin, but it may help
>>     avoid confusion if they don't.
>>
>>     More usefully, cointype can distinguish between keys intended for
>>     things like multisig outputs, e.g. for watchdog services. This means
>>     if your wallet does not know about the extra protocol layers
>>     involved in this, it can still import the "raw" money and it will
>>     just ignore/not see the keys used in more complex transactions.
>>
>>   * reserved is for "other stuff". I actually don't recall why we ended
>>     up with this. It may have been intended to split out multisig
>>     outputs etc from cointype. Marek, Thomas?
>>
>>   * account is for keeping essentially wallets-within-a-wallet to avoid
>>     mixing of coins. If you want that.
>>
>>   * change is 0 for receiving addresses, 1 for change addresses.
>>
>>   * n is the actual key index
>>
>> For bitcoinj we're targeting a deliberately limited feature set for hdw
>> v1 so I would just set the first three values all to zero and that is a
>> perfectly fine way to be compatible.
>>
>> The goal here is that the same seed can be written down once, and meet
>> all the users needs, whilst still allowing some drift between what
>> wallets support.
>>
>> Pieter made the I think valid point that you can't really encode how
>> keys are meant to be used into just an HDW hierarchy and normally you'd
>> need some metadata as well. However, I feel interop between wallets is
>> more important than arriving at the most perfect possible arrangement,
>> which feels a little like bikeshedding, so I'm happy to just go with the
>> flow on this one.
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Type: text/html, Size: 5632 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Mike Belshe]

[-- Attachment #1: Type: text/plain, Size: 7234 bytes --]

On Wed, Mar 26, 2014 at 6:01 PM, Alan Reiner <etotheipi@gmail.com> wrote:

>  This might be tangential, but the comment about "refund" chains reminded
> me.  Armory will be implementing multi-sig/linked wallets where a each
> device has a parallel HDW branch and produces P2SH addresses.  For those
> types of wallets, I plan to allocate two chains *per signing authority*.
> If you have a shared 2-of-2 wallet split between your phone and your
> spouse's phone, your phone would distribute addresses on P2SH chain 0 and
> generate change addresses on P2SH chain 1.  Your spouse's phone would use
> chains 2 and 3.
>
> So if you and your spouse switch to a new app that supports M-of-N linked
> wallets, it should search for coin history along the first 2*N chains.
>

In general with multisig, we should probably discourage using a common root
keychain for multiple keys in the same P2SH address.  The reason is because
if you have the single, root private key, you can sign them all. This
generally goes against the point of multisig - which was trying to
introduce a system which had multiple keys required to unlock :-)

For the BitGo BIP32 implementation we are doing the following:
   - users create 2 extended keys
   - bitgo creates 1 extended key for that user
   - the user can create any number of separate branches (accounts) of P2SH
addresses from those extended keys.
   - change/public addresses are generated from the 0/1 branches
   - new addresses are generated from there.
   - each time a new change address is generated, all 3 keys in the P2SH
are rotated.  This makes it so that for any chain depth *i* of the P2SH
address, we're using the *i*th key in each keychain.  For privacy reasons,
we don't need to rotate all keys.  But I think it makes management simpler
and probably more ready for interop.

So the paths in a user's keychain look exactly like BIP32's layout:
     m/<AccountIndex>/<ExternalOrInternal>/<AddressIndex>

Another issue unique to P2SH addresses is order of the keys. There has been
talk in this list of how to sort them.  While sorting is simplifying for
interop if you never use new change addresses, I believe sorting makes
wallet management more difficult when BIP32 is introduced.

Specifically, imagine a P2SH address with keys held by Bob, Charlie, and
Dana.  If you sort the pubkeys, then with each BIP32 chain, the order of
the keys could be different.  By maintaining key order, we always know
which of the pubkeys belongs to which user without having to do any lookups.

Obviously this can all be calculated too.  But the sorting seems gratuitous
to me - it adds complexity but offers little value.  On the other hand, it
can be really handy to know that key #0 for this address is always Bob's
key, regardless of how deep the chaining goes.

Mike



>
> -Alan
>
>
>
>
> On 03/26/2014 07:37 PM, Andreas Schildbach wrote:
>
> Thanks for starting the discussion on finding a better structure.
>
> For me, the most important thing is either we're 100% interoperable or
> 0%. There should not be anything inbetween, as users will delete seeds
> without knowing there is still money in them on another implementation.
> I heard from multiple sources that using this standard some wallets will
> only see a subset of the addresses/keys of some other wallets.
> Implementation differences can always happen (and should addresses as
> bugs), but I think its unacceptable that this source of issues is by design.
>
> I suggest we agree on an even simpler least common denominator and
> wallets that want to implement some feature on top of that can do but
> are encouraged to pick a totally different "cointype". I guess that
> would mean removing reserved and account.
>
> I'm still thinking it might be a good idea to have a separate chain for
> "refunds". Refunds will be rarely used and thus need a much slower
> moving window than receiving addresses or change.
>
>
> On 03/26/2014 09:49 PM, Mike Hearn wrote:
>
>  Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
> our BIP32 wallet structures would be compatible - and I discovered that
> only I was planning to use the default structure.
>
> Because I'm hopeful that we can get a lot of interoperability between
> wallets with regards to importing 12-words paper wallets, we
> brainstormed to find a structure acceptable to everyone and ended up with:
>
>   /m/cointype/reserved'/account'/change/n
>
> The extra levels require some explanation:
>
>   * cointype:  This is zero for Bitcoin. This is here to support two
>     things, one is supporting alt coins based off the same root seed.
>     Right now nobody seemed very bothered about alt coins but sometimes
>     feature requests do come in for this. Arguably there is no need and
>     alt coins could just use the same keys as Bitcoin, but it may help
>     avoid confusion if they don't.
>
>     More usefully, cointype can distinguish between keys intended for
>     things like multisig outputs, e.g. for watchdog services. This means
>     if your wallet does not know about the extra protocol layers
>     involved in this, it can still import the "raw" money and it will
>     just ignore/not see the keys used in more complex transactions.
>
>   * reserved is for "other stuff". I actually don't recall why we ended
>     up with this. It may have been intended to split out multisig
>     outputs etc from cointype. Marek, Thomas?
>
>   * account is for keeping essentially wallets-within-a-wallet to avoid
>     mixing of coins. If you want that.
>
>   * change is 0 for receiving addresses, 1 for change addresses.
>
>   * n is the actual key index
>
> For bitcoinj we're targeting a deliberately limited feature set for hdw
> v1 so I would just set the first three values all to zero and that is a
> perfectly fine way to be compatible.
>
> The goal here is that the same seed can be written down once, and meet
> all the users needs, whilst still allowing some drift between what
> wallets support.
>
> Pieter made the I think valid point that you can't really encode how
> keys are meant to be used into just an HDW hierarchy and normally you'd
> need some metadata as well. However, I feel interop between wallets is
> more important than arriving at the most perfect possible arrangement,
> which feels a little like bikeshedding, so I'm happy to just go with the
> flow on this one.
>
>
>
> ------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Bitcoin-development mailing listBitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>  ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing listBitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 9085 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 27/03/2014 00:37, Andreas Schildbach a écrit :
> Thanks for starting the discussion on finding a better structure.
>
> For me, the most important thing is either we're 100% interoperable or
> 0%. There should not be anything inbetween, as users will delete seeds
> without knowing there is still money in them on another implementation.

I believe you have a good point here: we should not advertise wallets as
compatible if they are not 100% compatible.

One issue that I have is bandwidth: Electrum (and mycelium) cannot
watch as many addresses as they want, because this will create too
much traffic on the servers. (especially when servers send utxo merkle
proofs for each address, which is not the case yet, but is planned)

For this reason Electrum imposes a constraint on the number of virgin
addresses provided to the user. Although the current strategy used by
Electrum can certainly be improved, it will not scale up to having every
client watching thousands of addresses.

This constraint is not so important for bloom-filter clients. So I guess 
that
it makes sense for Multibit to provide hundreds, or even thousands of 
virgin
addresses to the user, regardless of how they are used. Such a wallet will
in general not be recoverable in Electrum, unless the user "helps" the
recovery procedure. (or the seed has metadata telling the software that
this is a Multibit wallet). So we have a problem here, if we advertise 
these
wallets as compatible.

My opinion, as far as Electrum is concerned, is that merchant accounts
should behave differently from regular user accounts: While merchants
need to generate an unlimited number of receiving addresses, it is also
acceptable for them to have a slightly more complex wallet recovery 
procedure
(for example, the wallet might show an option to "search for more 
addresses",
and it might not need to watch "old" addresses anymore)

OTOH, I don't think we can ask regular users to do this, not only 
because it
adds complexity to the wallet recovery procedure (which makes it scarier),
but also because we want fully automated synchronization between different
instances of a wallet, using only no other source of information than 
the blockchain.

The first versions of Electrum allowed users to set the "gap limit" 
parameter
in their GUI preferences, but I removed it from GUI after I realized it 
was a bad
idea (users messed with it and did not understand what happened..)

With bloom filter clients I guess the distinction between these two use 
cases
is not really necessary, because watching addresses is cheap. So it 
would be
good to hear what you and Mike think about this problem. If you decide 
to let
the user create hundreds of unused addresses (and I think it perfectly 
makes
sense for you), then I guess it would be better for Electrum to give up on
compatibility, rather than running the risk of seeing only a subset of 
addresses.
Another option is to handle these seeds as "merchant" accounts.




> I heard from multiple sources that using this standard some wallets will
> only see a subset of the addresses/keys of some other wallets.
> Implementation differences can always happen (and should addresses as
> bugs), but I think its unacceptable that this source of issues is by design.
>
> I suggest we agree on an even simpler least common denominator and
> wallets that want to implement some feature on top of that can do but
> are encouraged to pick a totally different "cointype". I guess that
> would mean removing reserved and account.

>
> I'm still thinking it might be a good idea to have a separate chain for
> "refunds". Refunds will be rarely used and thus need a much slower
> moving window than receiving addresses or change.
>
>
> On 03/26/2014 09:49 PM, Mike Hearn wrote:
>> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
>> our BIP32 wallet structures would be compatible - and I discovered that
>> only I was planning to use the default structure.
>>
>> Because I'm hopeful that we can get a lot of interoperability between
>> wallets with regards to importing 12-words paper wallets, we
>> brainstormed to find a structure acceptable to everyone and ended up with:
>>
>>    /m/cointype/reserved'/account'/change/n
>>
>> The extra levels require some explanation:
>>
>>    * cointype:  This is zero for Bitcoin. This is here to support two
>>      things, one is supporting alt coins based off the same root seed.
>>      Right now nobody seemed very bothered about alt coins but sometimes
>>      feature requests do come in for this. Arguably there is no need and
>>      alt coins could just use the same keys as Bitcoin, but it may help
>>      avoid confusion if they don't.
>>
>>      More usefully, cointype can distinguish between keys intended for
>>      things like multisig outputs, e.g. for watchdog services. This means
>>      if your wallet does not know about the extra protocol layers
>>      involved in this, it can still import the "raw" money and it will
>>      just ignore/not see the keys used in more complex transactions.
>>
>>    * reserved is for "other stuff". I actually don't recall why we ended
>>      up with this. It may have been intended to split out multisig
>>      outputs etc from cointype. Marek, Thomas?
>>
>>    * account is for keeping essentially wallets-within-a-wallet to avoid
>>      mixing of coins. If you want that.
>>
>>    * change is 0 for receiving addresses, 1 for change addresses.
>>
>>    * n is the actual key index
>>
>> For bitcoinj we're targeting a deliberately limited feature set for hdw
>> v1 so I would just set the first three values all to zero and that is a
>> perfectly fine way to be compatible.
>>
>> The goal here is that the same seed can be written down once, and meet
>> all the users needs, whilst still allowing some drift between what
>> wallets support.
>>
>> Pieter made the I think valid point that you can't really encode how
>> keys are meant to be used into just an HDW hierarchy and normally you'd
>> need some metadata as well. However, I feel interop between wallets is
>> more important than arriving at the most perfect possible arrangement,
>> which feels a little like bikeshedding, so I'm happy to just go with the
>> flow on this one.
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2584 bytes --]

>
> One issue that I have is bandwidth: Electrum (and mycelium) cannot
> watch as many addresses as they want, because this will create too
> much traffic on the servers. (especially when servers send utxo merkle
> proofs for each address, which is not the case yet, but is planned)
>

This is surprising and the first time I've heard about this. Surely your
constraint is CPU or disk seeks? Addresses are small, I find it hard to
believe that clients uploading them is a big drain, and mostly addresses
that are in the lookahead region won't have any hits and so won't result in
any downloads?

This constraint is not so important for bloom-filter clients.


Bloom filters are a neat way to encode addresses and keys but they don't
magically let clients save bandwidth. A smaller filter results in less
upload bandwidth but more download (from the wallets perspective). So I'm
worried if you think this will be an issue for your clients: I haven't
investigated bandwidth usage deeply yet, perhaps I should.

FWIW the current bitcoinj HDW alpha preview pre-gens 100 addresses on both
receive and change branches. But I'm not sure what the right setting is.

We also have to consider latency. The simplest implementation from a
wallets POV is to step through each transaction in the block chain one at a
time, and each time you see an address that is yours, calculate the next
ones in the chain. But that would be fantastically slow, so we must instead
pre-generate a larger lookahead region and request more data in one batch.
Then you have to recover if that batch ends up using all the pre-genned
addresses. It's just painful.


> My opinion, as far as Electrum is concerned, is that merchant accounts
> should behave differently from regular user accounts: While merchants
> need to generate an unlimited number of receiving addresses, it is also
> acceptable for them to have a slightly more complex wallet recovery
> procedure
>

Maybe. I dislike any distinction between users and merchants though. I
don't think it's really safe to assume merchants are more sophisticated
than end users.


> but also because we want fully automated synchronization between different
> instances of a wallet, using only no other source of information than
> the blockchain.
>

I think such synchronization won't be possible as we keep adding features,
because the block chain cannot sync all the relevant data. For instance
Electrum already has a label sync feature. Other wallets need to compete
with that, somehow, so we need to build a way to do cross-device wallet
sync with non-chain data.

[-- Attachment #2: Type: text/html, Size: 3386 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 27/03/2014 12:39, Mike Hearn a écrit :
>     One issue that I have is bandwidth: Electrum (and mycelium) cannot
>     watch as many addresses as they want, because this will create too
>     much traffic on the servers. (especially when servers send utxo merkle
>     proofs for each address, which is not the case yet, but is planned)
>
>
> This is surprising and the first time I've heard about this. Surely your
> constraint is CPU or disk seeks? Addresses are small, I find it hard to
> believe that clients uploading them is a big drain, and mostly addresses
> that are in the lookahead region won't have any hits and so won't result
> in any downloads?


To be honest, I have not carried out a comprehensive examination of 
server performance. What I can see is that Electrum servers are often 
slowed down when a wallet with a large number (thousands) of addresses 
shows up, and this is caused by disk seeks (especially on my slow VPS).

The master branch of electrum-server is also quite wasteful in terms of 
CPU, because it uses client threads. I have another branch that uses a 
socket poller, but that branch is not widely deployed yet.

I reckon that I might have been a bit too conservative, in setting the 
number of unused receiving addresses watched by Electrum clients (until 
now, the default "gap limit" has always been 5). The reason is that, if 
I increase that number, then there is no way to go back to a smaller 
value, because it needs to be compatible with all previously released 
versions. However, Electrum servers performance has improved over time, 
so I guess it could safely be raised to 20 (see previous post to slush).

In terms of bandwidth, I am referring to my Android version of Electrum. 
When it runs on a 3G connection, it sometimes takes up to 1 minute to 
synchronize (with a wallet that has hundreds of addresses). However, I 
have not checked if this was caused by addresses or block headers.



>
>     This constraint is not so important for bloom-filter clients.
>
>
> Bloom filters are a neat way to encode addresses and keys but they don't
> magically let clients save bandwidth. A smaller filter results in less
> upload bandwidth but more download (from the wallets perspective). So
> I'm worried if you think this will be an issue for your clients: I
> haven't investigated bandwidth usage deeply yet, perhaps I should.
>
> FWIW the current bitcoinj HDW alpha preview pre-gens 100 addresses on
> both receive and change branches. But I'm not sure what the right
> setting is.


Heh, may I suggest 20 in the receive branch?

For the change branch, there is no need to watch a large number of 
unused addresses, because the wallet should try to fill all the gaps in 
the sequence of change.

(Electrum does that. It also watches 3 unused addresses at the end of 
that sequence, in order to cope with possible blockchain reorgs causing 
gaps. As an extra safety, it also waits for 3 confirmations before using 
a new change address, which sometimes results in address reuse, but I 
guess a smarter strategy could avoid that).



>
> We also have to consider latency. The simplest implementation from a
> wallets POV is to step through each transaction in the block chain one
> at a time, and each time you see an address that is yours, calculate the
> next ones in the chain. But that would be fantastically slow, so we must
> instead pre-generate a larger lookahead region and request more data in
> one batch. Then you have to recover if that batch ends up using all the
> pre-genned addresses. It's just painful.



>
>     My opinion, as far as Electrum is concerned, is that merchant accounts
>     should behave differently from regular user accounts: While merchants
>     need to generate an unlimited number of receiving addresses, it is also
>     acceptable for them to have a slightly more complex wallet recovery
>     procedure
>
>
> Maybe. I dislike any distinction between users and merchants though. I
> don't think it's really safe to assume merchants are more sophisticated
> than end users.

well, it depends what we mean by "merchant". I was thinking more of a 
website running a script, rather than a brick and mortar ice cream 
seller. :)


>
>     but also because we want fully automated synchronization between
>     different
>     instances of a wallet, using only no other source of information than
>     the blockchain.
>
>
> I think such synchronization won't be possible as we keep adding
> features, because the block chain cannot sync all the relevant data. For
> instance Electrum already has a label sync feature. Other wallets need
> to compete with that, somehow, so we need to build a way to do
> cross-device wallet sync with non-chain data.

Oh, I was not referring to label sync, but only to the synchronization 
of the list of addresses in the wallet. Label sync is an Electrum plugin 
that relies on a centralized server. Using a third party server is 
acceptable in that case, IMO, because you will not lose your coins if 
the server fails.

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

>
> To be honest, I have not carried out a comprehensive examination of
> server performance. What I can see is that Electrum servers are often
> slowed down when a wallet with a large number (thousands) of addresses
> shows up, and this is caused by disk seeks (especially on my slow VPS).
>

Yes that sounds more like what I expected.


> In terms of bandwidth, I am referring to my Android version of Electrum.
> When it runs on a 3G connection, it sometimes takes up to 1 minute to
> synchronize (with a wallet that has hundreds of addresses). However, I
> have not checked if this was caused by addresses or block headers.
>

An address is 160 bits. (1000 * 160) / 8 / 1024 = 19.5 kilobytes of data
which 3G should be able to transfer in <1 second easily. Of course the
encoding may not be optimal. But if it is, I suspect the issue is elsewhere.

[-- Attachment #2: Type: text/html, Size: 1392 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 3594 bytes --]

For every branch (say multiple accounts), how would a new wallet be able to
know how many sequence items to scan?  It seems like not only do you need
to have standard rules for the hierarchy, but how the usage can be
detected.  The other scanning seems pretty straightforward.  For accounts,
it seems like you could have a situation where you want to initially set up
10 different accounts, but only account #10 gets any transactions.  If a
new wallet was trying to scan with this seed, it would have to know to keep
scanning each account until it found the account.  The user would have to
be responsible for knowing how many accounts there are, or some rules would
need to be in place to not allow creating accounts until earlier accounts
can be proven to have existed in the blockchain.  Or I am missing something.

-Allen

On Wed, Mar 26, 2014 at 3:49 PM, Mike Hearn <mike@plan99.net> wrote:

> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
> our BIP32 wallet structures would be compatible - and I discovered that
> only I was planning to use the default structure.
>
> Because I'm hopeful that we can get a lot of interoperability between
> wallets with regards to importing 12-words paper wallets, we brainstormed
> to find a structure acceptable to everyone and ended up with:
>
>   /m/cointype/reserved'/account'/change/n
>
> The extra levels require some explanation:
>
>    - cointype:  This is zero for Bitcoin. This is here to support two
>    things, one is supporting alt coins based off the same root seed. Right now
>    nobody seemed very bothered about alt coins but sometimes feature requests
>    do come in for this. Arguably there is no need and alt coins could just use
>    the same keys as Bitcoin, but it may help avoid confusion if they don't.
>
>    More usefully, cointype can distinguish between keys intended for
>    things like multisig outputs, e.g. for watchdog services. This means if
>    your wallet does not know about the extra protocol layers involved in this,
>    it can still import the "raw" money and it will just ignore/not see the
>    keys used in more complex transactions.
>
>    - reserved is for "other stuff". I actually don't recall why we ended
>    up with this. It may have been intended to split out multisig outputs etc
>    from cointype. Marek, Thomas?
>
>    - account is for keeping essentially wallets-within-a-wallet to avoid
>    mixing of coins. If you want that.
>
>    - change is 0 for receiving addresses, 1 for change addresses.
>
>    - n is the actual key index
>
> For bitcoinj we're targeting a deliberately limited feature set for hdw v1
> so I would just set the first three values all to zero and that is a
> perfectly fine way to be compatible.
>
> The goal here is that the same seed can be written down once, and meet all
> the users needs, whilst still allowing some drift between what wallets
> support.
>
> Pieter made the I think valid point that you can't really encode how keys
> are meant to be used into just an HDW hierarchy and normally you'd need
> some metadata as well. However, I feel interop between wallets is more
> important than arriving at the most perfect possible arrangement, which
> feels a little like bikeshedding, so I'm happy to just go with the flow on
> this one.
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 5084 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Troy Benjegerdes]

On Wed, Mar 26, 2014 at 09:49:39PM +0100, Mike Hearn wrote:
> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
> our BIP32 wallet structures would be compatible - and I discovered that
> only I was planning to use the default structure.
> 
> Because I'm hopeful that we can get a lot of interoperability between
> wallets with regards to importing 12-words paper wallets, we brainstormed
> to find a structure acceptable to everyone and ended up with:
> 
>   /m/cointype/reserved'/account'/change/n
> 
> The extra levels require some explanation:
> 
>    - cointype:  This is zero for Bitcoin. This is here to support two
>    things, one is supporting alt coins based off the same root seed. Right now
>    nobody seemed very bothered about alt coins but sometimes feature requests
>    do come in for this. Arguably there is no need and alt coins could just use
>    the same keys as Bitcoin, but it may help avoid confusion if they don't.

Using the same keys across different altcoins seems like an exceedingly bad opsec
practice. Cointype is critical, as well as having a predictable and deterministic
mapping of alt coins to Cointype.

What should I be using for Catcoin, for instance? the CAT symbol all the 
exchanges use, or do we set up a 'registry', or some other mechanism?

I'd venture to guess the altcoin market is, or soon will be larger in US
dollar value trade volume than Bitcoin, so *some* of us are quite bothered
by the wailing and gnashing of teeth that occurs on this list at mere thought
of such heresy.


-- 
----------------------------------------------------------------------------
Troy Benjegerdes                 'da hozer'                  hozer@hozed.org
7 elements      earth::water::air::fire::mind::spirit::soul        grid.coop

      Never pick a fight with someone who buys ink by the barrel,
         nor try buy a hacker who makes money by the megahash

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 3487 bytes --]

We had a similar meeting with Andreas Schildbach (Android Bitcoin Wallet), Jan Moller, Andreas  Petersson (Mycelium), Thomas V (Electrum), Tamas Blummer, Tamas Bartfai (Bits of Proof)
at the Inside Bitcoin Conference in Berlin.

I remember that there were different opinions on how to use a hierarchy and it did seem to me they could eventually be "standardized" for the retail customer but definitelly not for corporate use,
where hierarchy will certainly map to organisational hierarchy or cost centres.

A notable suggestion was to instead of building a directory of magic numbers (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word "Bitcoin", "Litecoin", "Dogecoin", so collosion is unlikely and
cetral directory is not needed.

Regards,

Tamas Blummer
http://bitsofproof.com

On 26.03.2014, at 21:49, Mike Hearn <mike@plan99.net> wrote:

> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure our BIP32 wallet structures would be compatible - and I discovered that only I was planning to use the default structure.
> 
> Because I'm hopeful that we can get a lot of interoperability between wallets with regards to importing 12-words paper wallets, we brainstormed to find a structure acceptable to everyone and ended up with:
> 
>   /m/cointype/reserved'/account'/change/n
> 
> The extra levels require some explanation:
> cointype:  This is zero for Bitcoin. This is here to support two things, one is supporting alt coins based off the same root seed. Right now nobody seemed very bothered about alt coins but sometimes feature requests do come in for this. Arguably there is no need and alt coins could just use the same keys as Bitcoin, but it may help avoid confusion if they don't.
> 
> More usefully, cointype can distinguish between keys intended for things like multisig outputs, e.g. for watchdog services. This means if your wallet does not know about the extra protocol layers involved in this, it can still import the "raw" money and it will just ignore/not see the keys used in more complex transactions.
> 
> reserved is for "other stuff". I actually don't recall why we ended up with this. It may have been intended to split out multisig outputs etc from cointype. Marek, Thomas?
> 
> account is for keeping essentially wallets-within-a-wallet to avoid mixing of coins. If you want that.
> 
> change is 0 for receiving addresses, 1 for change addresses.
> 
> n is the actual key index
> For bitcoinj we're targeting a deliberately limited feature set for hdw v1 so I would just set the first three values all to zero and that is a perfectly fine way to be compatible.
> 
> The goal here is that the same seed can be written down once, and meet all the users needs, whilst still allowing some drift between what wallets support.
> 
> Pieter made the I think valid point that you can't really encode how keys are meant to be used into just an HDW hierarchy and normally you'd need some metadata as well. However, I feel interop between wallets is more important than arriving at the most perfect possible arrangement, which feels a little like bikeshedding, so I'm happy to just go with the flow on this one.
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #1.2: Type: text/html, Size: 7500 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 4282 bytes --]

At this point I'm not sure how much further work people want to do on this:
I got the impression that Trezor will ship soon, and Thomas V seemed
satisfied too. I'm not sure we can get all wallets to be fully
interoperable given the flexibility inherent in BIP32 and people's
differing use cases.

Andreas: good point but I really hope nobody ever deletes a seed after all
this work we put in to make backups so easy! I'm not sure we can really
stop it anyway: not unless we make the seed a full blown data structure
with hints to other apps that they should refuse to load it. And it's a bit
late for that now.



On Thu, Mar 27, 2014 at 8:09 AM, Tamas Blummer <tamas@bitsofproof.com>wrote:

> We had a similar meeting with Andreas Schildbach (Android Bitcoin Wallet),
> Jan Moller, Andreas  Petersson (Mycelium), Thomas V (Electrum), Tamas
> Blummer, Tamas Bartfai (Bits of Proof)
> at the Inside Bitcoin Conference in Berlin.
>
> I remember that there were different opinions on how to use a hierarchy
> and it did seem to me they could eventually be "standardized" for the
> retail customer but definitelly not for corporate use,
> where hierarchy will certainly map to organisational hierarchy or cost
> centres.
>
> A notable suggestion was to instead of building a directory of magic
> numbers (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word
> "Bitcoin", "Litecoin", "Dogecoin", so collosion is unlikely and
> cetral directory is not needed.
>
> Regards,
>
> Tamas Blummer
> http://bitsofproof.com
>
> On 26.03.2014, at 21:49, Mike Hearn <mike@plan99.net> wrote:
>
> Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure
> our BIP32 wallet structures would be compatible - and I discovered that
> only I was planning to use the default structure.
>
> Because I'm hopeful that we can get a lot of interoperability between
> wallets with regards to importing 12-words paper wallets, we brainstormed
> to find a structure acceptable to everyone and ended up with:
>
>   /m/cointype/reserved'/account'/change/n
>
> The extra levels require some explanation:
>
>    - cointype:  This is zero for Bitcoin. This is here to support two
>    things, one is supporting alt coins based off the same root seed. Right now
>    nobody seemed very bothered about alt coins but sometimes feature requests
>    do come in for this. Arguably there is no need and alt coins could just use
>    the same keys as Bitcoin, but it may help avoid confusion if they don't.
>
>    More usefully, cointype can distinguish between keys intended for
>    things like multisig outputs, e.g. for watchdog services. This means if
>    your wallet does not know about the extra protocol layers involved in this,
>    it can still import the "raw" money and it will just ignore/not see the
>    keys used in more complex transactions.
>
>    - reserved is for "other stuff". I actually don't recall why we ended
>    up with this. It may have been intended to split out multisig outputs etc
>    from cointype. Marek, Thomas?
>
>    - account is for keeping essentially wallets-within-a-wallet to avoid
>    mixing of coins. If you want that.
>
>    - change is 0 for receiving addresses, 1 for change addresses.
>
>    - n is the actual key index
>
> For bitcoinj we're targeting a deliberately limited feature set for hdw v1
> so I would just set the first three values all to zero and that is a
> perfectly fine way to be compatible.
>
> The goal here is that the same seed can be written down once, and meet all
> the users needs, whilst still allowing some drift between what wallets
> support.
>
> Pieter made the I think valid point that you can't really encode how keys
> are meant to be used into just an HDW hierarchy and normally you'd need
> some metadata as well. However, I feel interop between wallets is more
> important than arriving at the most perfect possible arrangement, which
> feels a little like bikeshedding, so I'm happy to just go with the flow on
> this one.
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>

[-- Attachment #2: Type: text/html, Size: 7747 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/27/2014 08:09 AM, Tamas Blummer wrote:
> A notable suggestion was to instead of building a directory of magic numbers (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word "Bitcoin", "Litecoin", "Dogecoin", so collosion is unlikely and
> cetral directory is not needed.

Nice idea, but keep in mind that you are hashing into 2^32 space, so
collisions will occur, unfortunately and we'll end up with directory
again :-/

Even if they did not occur you would need to keep up the registry of
names anyway (is it Peercoin or PPCoin, Testnet or TestNet ...)?

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Jeff Garzik]

On Thu, Mar 27, 2014 at 3:09 AM, Tamas Blummer <tamas@bitsofproof.com> wrote:
> A notable suggestion was to instead of building a directory of magic numbers
> (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word "Bitcoin",
> "Litecoin", "Dogecoin", so collosion is unlikely and
> cetral directory is not needed.

+1 good idea

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

Re: [Bitcoin-development] New BIP32 structure

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

Don't most of these coins have a magic number already assigned that is
unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for
Litecoin, etc...).  This seems like a good candidate for identifying coins,
and also supports Testnet cases well.  Maybe there are some alts without
such a magic number that might prevent that?

-Allen


On Thu, Mar 27, 2014 at 10:43 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:

> On Thu, Mar 27, 2014 at 3:09 AM, Tamas Blummer <tamas@bitsofproof.com>
> wrote:
> > A notable suggestion was to instead of building a directory of magic
> numbers
> > (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word
> "Bitcoin",
> > "Litecoin", "Dogecoin", so collosion is unlikely and
> > cetral directory is not needed.
>
> +1 good idea
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc.      https://bitpay.com/
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 2773 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/27/2014 04:57 PM, Allen Piscitello wrote:
> Don't most of these coins have a magic number already assigned that is
> unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for
> Litecoin, etc...).  This seems like a good candidate for identifying coins,
> and also supports Testnet cases well.  Maybe there are some alts without
> such a magic number that might prevent that?

That magic number is something I find very unfortunate and superflous in
BIP-32 design. Its only purpose is to distinguish BIP-32 trees for
various altcoins, but it doesn't make sense at all once you start
storing various altcoins in the same tree using the proposed
/m/cointype/reserved'/account'/change/n scheme.

I would love to see that removed from BIP-32 and use always
0x0488B21E/0x0488ADE4 (xpub/xpriv), but that is for different discussion
I guess.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 2094 bytes --]

The idea was to use the magic number as the source for cointype.  If it's
too big, as Tamas showed, perhaps a hash of it, and for coins without a
magic number, a hash of their name (or some unique identifier).

That being said, I agree with Andreas that something that is 90%
inter-operable seems very dangerous and will give people false expectations
when they miss the corner cases.  If the structure isn't going to be shared
completely and have all support shared, having it completely incompatible
along with a mechanism for converting part of it to another wallet seems
superior.  The worst types of losses will occur when someone tests out
something with a limited use case, sees that it appears to work, makes
dangerous assumptions, then gets burned when it's too late.

-Allen


On Thu, Mar 27, 2014 at 11:06 AM, Pavol Rusnak <stick@gk2.sk> wrote:

> On 03/27/2014 04:57 PM, Allen Piscitello wrote:
> > Don't most of these coins have a magic number already assigned that is
> > unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for
> > Litecoin, etc...).  This seems like a good candidate for identifying
> coins,
> > and also supports Testnet cases well.  Maybe there are some alts without
> > such a magic number that might prevent that?
>
> That magic number is something I find very unfortunate and superflous in
> BIP-32 design. Its only purpose is to distinguish BIP-32 trees for
> various altcoins, but it doesn't make sense at all once you start
> storing various altcoins in the same tree using the proposed
> /m/cointype/reserved'/account'/change/n scheme.
>
> I would love to see that removed from BIP-32 and use always
> 0x0488B21E/0x0488ADE4 (xpub/xpriv), but that is for different discussion
> I guess.
>
> --
> Best Regards / S pozdravom,
>
> Pavol Rusnak <stick@gk2.sk>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 2856 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 1573 bytes --]

I think not all alts (will) have magic numbers, at least not those defined e.g. with colored coins on top of an other chain.

Also note that the index should have MSB cleared as it would otherwise indicate private derivation. 

Regards,

Tamas Blummer
http://bitsofproof.com

On 27.03.2014, at 16:57, Allen Piscitello <allen.piscitello@gmail.com> wrote:

> Don't most of these coins have a magic number already assigned that is unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for Litecoin, etc...).  This seems like a good candidate for identifying coins, and also supports Testnet cases well.  Maybe there are some alts without such a magic number that might prevent that?
> 
> -Allen
> 
> 
> On Thu, Mar 27, 2014 at 10:43 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
> On Thu, Mar 27, 2014 at 3:09 AM, Tamas Blummer <tamas@bitsofproof.com> wrote:
> > A notable suggestion was to instead of building a directory of magic numbers
> > (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word "Bitcoin",
> > "Litecoin", "Dogecoin", so collosion is unlikely and
> > cetral directory is not needed.
> 
> +1 good idea
> 
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc.      https://bitpay.com/
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 


[-- Attachment #1.2: Type: text/html, Size: 6014 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

Just chiming in...

I'm not opposed to a more generic default key tree, but we need to
standardize this soon I believe. There are already existing code bases
that implement BIP32 wallets (and more are popping up...); just using
a separate one will result in lots of incompibilities.

That said, I'm not convinced about the extra layers. The "cointype" in
my opinion isn't necessary inside the derivation. There is already
support (4 bytes!) for magic bytes in the serialized form. Inside
applications/p2p it should always be known to which chain it applies,
and outside of that you shouldn't transfer raw keys. Maybe seeds need
some marker, but that's a separate case anyway. Mainnet and testnet
have specified magics here already - alts can define what they want
imho.

A 'reserved' field for future extensions may be useful, but as already
suggested by Mike, I don't believe we can encode how key chains are to
be used inside the derivation structure anyway. The most basic case
(not losing money in a wallet without special structure) can perhaps
be supported with just "the blockchain is your wallet", but I don't
believe this principle can scale to more advanced uses anyway, and you
need metadata in the wallet to deal with it.

In my view, your wallet just has a bunch of chains, and each chain
gets used for a particular purpose, fixing how the derivation beneath
it works. Either that is as a wallet, as part of a pair of multisig
keys, as a recurring payment receiver, ... or more complex things.
Some of these will require extra layers beneath, but that is
application specific. You would import a chain into your (advanced)
wallet with a particular extpub/extpriv code, and some metadata on how
to use it. Serialization formats for such designated extra uses sounds
better to me than trying to fit it into the derivation structure.

-- 
Pieter


On Thu, Mar 27, 2014 at 4:57 PM, Allen Piscitello
<allen.piscitello@gmail.com> wrote:
> Don't most of these coins have a magic number already assigned that is
> unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for
> Litecoin, etc...).  This seems like a good candidate for identifying coins,
> and also supports Testnet cases well.  Maybe there are some alts without
> such a magic number that might prevent that?
>
> -Allen
>
>
> On Thu, Mar 27, 2014 at 10:43 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
>>
>> On Thu, Mar 27, 2014 at 3:09 AM, Tamas Blummer <tamas@bitsofproof.com>
>> wrote:
>> > A notable suggestion was to instead of building a directory of magic
>> > numbers
>> > (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word
>> > "Bitcoin",
>> > "Litecoin", "Dogecoin", so collosion is unlikely and
>> > cetral directory is not needed.
>>
>> +1 good idea
>>
>> --
>> Jeff Garzik
>> Bitcoin core developer and open source evangelist
>> BitPay, Inc.      https://bitpay.com/
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/27/2014 05:14 PM, Pieter Wuille wrote:
> That said, I'm not convinced about the extra layers. The "cointype" in
> my opinion isn't necessary inside the derivation. There is already
> support (4 bytes!) for magic bytes in the serialized form. Inside

Cointype in path is for separation purposes, not for identification.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

On Thu, Mar 27, 2014 at 5:21 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> Cointype in path is for separation purposes, not for identification.

I don't understand what that gains you.

-- 
Pieter

Re: [Bitcoin-development] New BIP32 structure

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

The benefit I see is avoiding reuse of keys between coins while not having
each wallet implementation have to know about each coin in order to scan
for transactions.  Wallet X supports Doge and Bitcoin.  If both used a
shared sequence of keys, say the first two end up Bitcoin, then 10 Doge,
then some more Bitcoin.  If you took this seed to Wallet Y, which only
supports Bitcoin (either the wallet's support or what is installed on the
system it's being used), it will see a gap of 10 addresses, and presume no
more scanning with a 5 gap limit.  The alternative is to reuse keys for
each coin.

It also seems like a solution might be to only expect interoperability on a
single sequence, and provide backups of each final sequence to use between
different wallet implementations.  This allows flexibility in hierarchies
depending on needs and support of wallet, but allows sharing.  The short
seed would only be useful for the same wallet, but sharing between wallets
would use the longer keys.  That will give predictable behavior for users
(although less friendly) and lead to less errors.

-Allen


On Thu, Mar 27, 2014 at 11:28 AM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> On Thu, Mar 27, 2014 at 5:21 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> > Cointype in path is for separation purposes, not for identification.
>
> I don't understand what that gains you.
>
> --
> Pieter
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 2419 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

I agree that 'version' field of bip32 is not necessary and xpriv/xpub
should be enough for all cases; there's actually no need to use different
BIP32 roots for different altcoins.

I'm happily using one xpub for Bitcoin/Testnet/Litecoin at once, and by
having the "cointype" distinction in the bip32 path itself, I'm sure that I
don't reuse the same pubkey across blockchains which may be a privacy issue
otherwise.

Marek


On Thu, Mar 27, 2014 at 5:28 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> On Thu, Mar 27, 2014 at 5:21 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> > Cointype in path is for separation purposes, not for identification.
>
> I don't understand what that gains you.
>
> --
> Pieter
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 1759 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 1677 bytes --]

After some off-list discussion about details with wallet developers, it
seems that structure

m/<cointype>'/<account>'/<change>/<n>

fulfill requirements of all wallet developers around, including myTrezor,
Electrum, Multibit, Wallet32 and other software is willing to adapt once
anything will be standardized (i.e. they don't care).

Because I think that everybody told their comments to the topic already and
because it seems that there's quite wide agreement on that, I would like to
close the discussion and finally implement these paths into our software.

Cheers,
Marek


On Fri, Mar 28, 2014 at 3:59 PM, slush <slush@centrum.cz> wrote:

> I agree that 'version' field of bip32 is not necessary and xpriv/xpub
> should be enough for all cases; there's actually no need to use different
> BIP32 roots for different altcoins.
>
> I'm happily using one xpub for Bitcoin/Testnet/Litecoin at once, and by
> having the "cointype" distinction in the bip32 path itself, I'm sure that I
> don't reuse the same pubkey across blockchains which may be a privacy issue
> otherwise.
>
> Marek
>
>
> On Thu, Mar 27, 2014 at 5:28 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:
>
>> On Thu, Mar 27, 2014 at 5:21 PM, Pavol Rusnak <stick@gk2.sk> wrote:
>> > Cointype in path is for separation purposes, not for identification.
>>
>> I don't understand what that gains you.
>>
>> --
>> Pieter
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2898 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

I still don't understand the purpose of cointype. If you don't want to
risk reusing the same keys across different currencies, just don't use
the same seed or the same account? That is purely a client-side issue.

If the consensus is to add the cointype anyway, can we fix it to be
equal to the 4-byte magic in the serialization (after setting the high
bit to true)? That way there aren't two 4-byte magic codes that need
to be defined for each, and at the same time make it obvious from the
serialized form what it is for.



On Tue, Apr 8, 2014 at 2:43 PM, slush <slush@centrum.cz> wrote:
> After some off-list discussion about details with wallet developers, it
> seems that structure
>
> m/<cointype>'/<account>'/<change>/<n>
>
> fulfill requirements of all wallet developers around, including myTrezor,
> Electrum, Multibit, Wallet32 and other software is willing to adapt once
> anything will be standardized (i.e. they don't care).
>
> Because I think that everybody told their comments to the topic already and
> because it seems that there's quite wide agreement on that, I would like to
> close the discussion and finally implement these paths into our software.
>
> Cheers,
> Marek
>
>
> On Fri, Mar 28, 2014 at 3:59 PM, slush <slush@centrum.cz> wrote:
>>
>> I agree that 'version' field of bip32 is not necessary and xpriv/xpub
>> should be enough for all cases; there's actually no need to use different
>> BIP32 roots for different altcoins.
>>
>> I'm happily using one xpub for Bitcoin/Testnet/Litecoin at once, and by
>> having the "cointype" distinction in the bip32 path itself, I'm sure that I
>> don't reuse the same pubkey across blockchains which may be a privacy issue
>> otherwise.
>>
>> Marek
>>
>>
>> On Thu, Mar 27, 2014 at 5:28 PM, Pieter Wuille <pieter.wuille@gmail.com>
>> wrote:
>>>
>>> On Thu, Mar 27, 2014 at 5:21 PM, Pavol Rusnak <stick@gk2.sk> wrote:
>>> > Cointype in path is for separation purposes, not for identification.
>>>
>>> I don't understand what that gains you.
>>>
>>> --
>>> Pieter
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>
>
> ------------------------------------------------------------------------------
> Put Bad Developers to Shame
> Dominate Development with Jenkins Continuous Integration
> Continuously Automate Build, Test & Deployment
> Start a new project now. Try Jenkins in the cloud.
> http://p.sf.net/sfu/13600_Cloudbees
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

On Tue, Apr 8, 2014 at 3:18 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> I still don't understand the purpose of cointype. If you don't want to
> risk reusing the same keys across different currencies, just don't use
> the same seed or the same account? That is purely a client-side issue.
>
>
Of course it is purely client-side issue, but it matters.

There's actually no reason to generate, backup and store individual seeds
for various coins and purposes. User can handle all his identities and
altcoins with single seed, avoiding potential issues with using wrong seed
for other purposes.

Actually with accounts and cointypes in the path, you can have all your
crypto funds stored on single seed, which I see as very comfortable
solution.

But to gain advantages of such solution and avoid reusing the same path
across blockchains, we need to separate the space, which is achieved by
cointype.


> If the consensus is to add the cointype anyway, can we fix it to be
> equal to the 4-byte magic in the serialization (after setting the high
> bit to true)? That way there aren't two 4-byte magic codes that need
> to be defined for each, and at the same time make it obvious from the
> serialized form what it is for.
>
>
Serialization magic of bip32 seed is in my opinion completely unnecessary.
Most of software does not care about it anyway; You can use xprv/xpub pair
for main net, testnet, litecoin, dogecoin, whatevercoin.

Instead using the same seed (xprv) and then separate the chains *inside*
the bip32 path seems more useful to me.

Marek

[-- Attachment #2: Type: text/html, Size: 2249 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

tl;dr;

It is dangerous to expect that other seed than "xprv" does not contain
bitcoins or that "xprv" contains only bitcoins, because technically are
both situations possible. It is still safer to do the lookup; the magic
itself is ambiguous.

Marek

On Tue, Apr 8, 2014 at 3:40 PM, slush <slush@centrum.cz> wrote:

>
> Serialization magic of bip32 seed is in my opinion completely unnecessary.
> Most of software does not care about it anyway; You can use xprv/xpub pair
> for main net, testnet, litecoin, dogecoin, whatevercoin.
>
> Instead using the same seed (xprv) and then separate the chains *inside*
> the bip32 path seems more useful to me.
>
> Marek
>

[-- Attachment #2: Type: text/html, Size: 1210 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

I see the cause of our disagreement now.

You actually want to share a single BIP32 tree across different
currency types, but do it in a way that guarantees that they never use
the same keys.

I would have expected that different chains would use independent
chains, and have serializations encode which chain they belong to.

Let me offer an alternative suggestion, which is compatible with the
original default BIP32 structure:
* You can use one seed across different chains, but the master nodes
are separate.
* To derive the master node from the seed, the key string "Bitcoin
seed" is replaced by something chain-specific.
* Every encoded node (including master nodes) has a chain-specific
serialization magic.

This is in practice almost the same as your suggestion, except that
the m/cointype' in m/cointype'/account'/change/n is replaced by
different masters. The only disadvantage I see is that you do not have
a way to encode the "super master" that is the parent of all
chain-specific masters. You can - and with the same security
properties - encode the seed, though.

-- 
Pieter


On Tue, Apr 8, 2014 at 3:43 PM, slush <slush@centrum.cz> wrote:
> tl;dr;
>
> It is dangerous to expect that other seed than "xprv" does not contain
> bitcoins or that "xprv" contains only bitcoins, because technically are both
> situations possible. It is still safer to do the lookup; the magic itself is
> ambiguous.
>
> Marek
>
> On Tue, Apr 8, 2014 at 3:40 PM, slush <slush@centrum.cz> wrote:
>>
>>
>> Serialization magic of bip32 seed is in my opinion completely unnecessary.
>> Most of software does not care about it anyway; You can use xprv/xpub pair
>> for main net, testnet, litecoin, dogecoin, whatevercoin.
>>
>> Instead using the same seed (xprv) and then separate the chains *inside*
>> the bip32 path seems more useful to me.
>>
>> Marek
>
>

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

+1

I would prefer that solution...



Le 08/04/2014 15:53, Pieter Wuille a écrit :
> I see the cause of our disagreement now.
>
> You actually want to share a single BIP32 tree across different
> currency types, but do it in a way that guarantees that they never use
> the same keys.
>
> I would have expected that different chains would use independent
> chains, and have serializations encode which chain they belong to.
>
> Let me offer an alternative suggestion, which is compatible with the
> original default BIP32 structure:
> * You can use one seed across different chains, but the master nodes
> are separate.
> * To derive the master node from the seed, the key string "Bitcoin
> seed" is replaced by something chain-specific.
> * Every encoded node (including master nodes) has a chain-specific
> serialization magic.
>
> This is in practice almost the same as your suggestion, except that
> the m/cointype' in m/cointype'/account'/change/n is replaced by
> different masters. The only disadvantage I see is that you do not have
> a way to encode the "super master" that is the parent of all
> chain-specific masters. You can - and with the same security
> properties - encode the seed, though.
>

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 2830 bytes --]

Pieter,

your suggestion has charm since “Bitcoin seed” would even not need 
a global dictionary like the interpretation of the first level, since it would be self describing.

Regards,

Tamas Blummer
http://bitsofproof.com

On 08.04.2014, at 15:53, Pieter Wuille <pieter.wuille@gmail.com> wrote:

> I see the cause of our disagreement now.
> 
> You actually want to share a single BIP32 tree across different
> currency types, but do it in a way that guarantees that they never use
> the same keys.
> 
> I would have expected that different chains would use independent
> chains, and have serializations encode which chain they belong to.
> 
> Let me offer an alternative suggestion, which is compatible with the
> original default BIP32 structure:
> * You can use one seed across different chains, but the master nodes
> are separate.
> * To derive the master node from the seed, the key string "Bitcoin
> seed" is replaced by something chain-specific.
> * Every encoded node (including master nodes) has a chain-specific
> serialization magic.
> 
> This is in practice almost the same as your suggestion, except that
> the m/cointype' in m/cointype'/account'/change/n is replaced by
> different masters. The only disadvantage I see is that you do not have
> a way to encode the "super master" that is the parent of all
> chain-specific masters. You can - and with the same security
> properties - encode the seed, though.
> 
> -- 
> Pieter
> 
> 
> On Tue, Apr 8, 2014 at 3:43 PM, slush <slush@centrum.cz> wrote:
>> tl;dr;
>> 
>> It is dangerous to expect that other seed than "xprv" does not contain
>> bitcoins or that "xprv" contains only bitcoins, because technically are both
>> situations possible. It is still safer to do the lookup; the magic itself is
>> ambiguous.
>> 
>> Marek
>> 
>> On Tue, Apr 8, 2014 at 3:40 PM, slush <slush@centrum.cz> wrote:
>>> 
>>> 
>>> Serialization magic of bip32 seed is in my opinion completely unnecessary.
>>> Most of software does not care about it anyway; You can use xprv/xpub pair
>>> for main net, testnet, litecoin, dogecoin, whatevercoin.
>>> 
>>> Instead using the same seed (xprv) and then separate the chains *inside*
>>> the bip32 path seems more useful to me.
>>> 
>>> Marek
>> 
>> 
> 
> ------------------------------------------------------------------------------
> Put Bad Developers to Shame
> Dominate Development with Jenkins Continuous Integration
> Continuously Automate Build, Test & Deployment 
> Start a new project now. Try Jenkins in the cloud.
> http://p.sf.net/sfu/13600_Cloudbees
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 


[-- Attachment #1.2: Type: text/html, Size: 6387 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/08/2014 03:53 PM, Pieter Wuille wrote:
> Let me offer an alternative suggestion, which is compatible with the
> original default BIP32 structure:
> * You can use one seed across different chains, but the master nodes
> are separate.
> * To derive the master node from the seed, the key string "Bitcoin
> seed" is replaced by something chain-specific.
> * Every encoded node (including master nodes) has a chain-specific
> serialization magic.

This is possible, but I find it much more practical to use just one list
(assignment of coins to indexes) than to use two lists (assignment of
coins to key strings and to serialization magic).

Keeping two lists is harder and adds unnecessary friction. (Also I am
not very happy for the possibility we'll have to deal with key strings
"sCAMCo1N RULEZZZZ!!!! bRoUghT TO YoU bY M4rty" and serialization magic
that leads to prefix "lulz").

Also from wallet's implementer perspective it is a little easier to use
just one root node and then descend in tree as needed than to use method
you described.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]

On Tue, Apr 8, 2014 at 3:53 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> I see the cause of our disagreement now.
>
> You actually want to share a single BIP32 tree across different
> currency types, but do it in a way that guarantees that they never use
> the same keys.
>
> I would have expected that different chains would use independent
> chains, and have serializations encode which chain they belong to.
>
> Let me offer an alternative suggestion, which is compatible with the
> original default BIP32 structure:
> * You can use one seed across different chains, but the master nodes
> are separate.
> * To derive the master node from the seed, the key string "Bitcoin
> seed" is replaced by something chain-specific.
>

I've discussed the solution of "Litecoin seed" in BIP32 HMAC with Litecoin
devs already, and after long discussion we've concluded that it is
generally bad idea.

When changing "Bitcoin seed" constant to something different, same
*entropy* will produce different *master node*. That's actually the
opposite what's requested, because xprv serialization format stores *node*,
not *entropy*. By changing HMAC constant, you still won't be able to store
one node and derive wallets for multiple coins at same time.



> * Every encoded node (including master nodes) has a chain-specific
> serialization magic.
>
This is in practice almost the same as your suggestion, except that
> the m/cointype' in m/cointype'/account'/change/n is replaced by
> different masters. The only disadvantage I see is that you do not have
> a way to encode the "super master" that is the parent of all
> chain-specific masters. You can - and with the same security
> properties - encode the seed, though.
>
>
Actually I don't understand why there's such disagreement about "cointype"
level here, what it breaks? I see it as the cleanest solution so far. It is
forward and backward compatible, does need any special extension to bip32
(to be strict, bip32 says "Bitcoin seed", so client using "Litecoin seed"
cannot be "bip32 compatible").

Of course, the problem of "cointype" can be solved in zillion ways, but
still using cointype in bip32 path seem to be the most elegant way so far,
because it fullfill all requirements for single backup, for separating
pubkeys and for handling all coins by one master...

Marek

[-- Attachment #2: Type: text/html, Size: 3469 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

On Tue, Apr 8, 2014 at 5:41 PM, slush <slush@centrum.cz> wrote:
> I've discussed the solution of "Litecoin seed" in BIP32 HMAC with Litecoin
> devs already, and after long discussion we've concluded that it is generally
> bad idea.
>
> When changing "Bitcoin seed" constant to something different, same *entropy*
> will produce different *master node*. That's actually the opposite what's
> requested, because xprv serialization format stores *node*, not *entropy*.
> By changing HMAC constant, you still won't be able to store one node and
> derive wallets for multiple coins at same time.

Storing the seed is superior to storing the master node already
(whether coin specific or not), as it is smaller.

All this changes is making the seed the "super master" which allows
generating the coin-specific masters (which get an actual useful
function: revealing your entire-tree, but only one coin's subset of
it).

>> * Every encoded node (including master nodes) has a chain-specific
>> serialization magic.
>>
>> This is in practice almost the same as your suggestion, except that
>> the m/cointype' in m/cointype'/account'/change/n is replaced by
>> different masters. The only disadvantage I see is that you do not have
>> a way to encode the "super master" that is the parent of all
>> chain-specific masters. You can - and with the same security
>> properties - encode the seed, though.
>>
>
> Actually I don't understand why there's such disagreement about "cointype"
> level here, what it breaks? I see it as the cleanest solution so far. It is
> forward and backward compatible, does need any special extension to bip32
> (to be strict, bip32 says "Bitcoin seed", so client using "Litecoin seed"
> cannot be "bip32 compatible").

Fair enough, it would break strictly BIP32. Then again, BIP32 is a
*Bitcoin* improvement proposal, and not something that necessarily
applies to other coins (they can adopt it of course, I don't care).

What I dislike is that this removes the ability of using the magic in
the serialization to prevent importing a chain from the wrong coin.
The standard could just say that instead of "Bitcoin seed", you'd use
"Coin seed: " + magic, so you don't need an extra mapping from
cointype to seed strings.

-- 
Pieter

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

On Wed, Apr 23, 2014 at 7:42 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:
>
> Storing the seed is superior to storing the master node already
> (whether coin specific or not), as it is smaller.
>
>
...Except that you're loosing flexibility (serialization, deserialization)
which gives you BIP32 node.

I see "bip32 seed" as some transitional, internal state from raw entropy to
bip32 master node and this seed should not be handled by the end user in
any form. In the oposite, well-serialized bip32 node (in xpriv, or even in
mnemonic format) can be used very widely and have no downsides against
using raw "bip32 seed".


>
> Fair enough, it would break strictly BIP32. Then again, BIP32 is a
> *Bitcoin* improvement proposal, and not something that necessarily
> applies to other coins (they can adopt it of course, I don't care).
>
>
I also don't care too much about altcoins, but people want them so me, as
infrastructure developer, need to think about it. And I don't see any
reason for breaking compatibility between Bitcoin and other altcoins. I
would be happier if there will be another sentence than "Bitcoin seed", but
honestly, who cares. It is just some magic string for hashing the raw
seed...


> What I dislike is that this removes the ability of using the magic in
> the serialization to prevent importing a chain from the wrong coin.
>

The truth is that even existing software which handle bip32 don't care
about 'version' at all. I think that "xpub/xprv" distinction is the only
useful feature of version, so user se if it stores public or private
information.

But using prefixes which doesn't enforce anything is even more dangerous.
If somebody exports node "dogeblablabla", it creates false exceptations
that there's only dogecoin stored.

 Marek

[-- Attachment #2: Type: text/html, Size: 2624 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]

For those who don't follow github pull requests regularly; there's pull
request for BIP64 defining HD wallet structure as discussed in this thread:

https://github.com/bitcoin/bips/pull/52



On Wed, Apr 23, 2014 at 8:01 PM, slush <slush@centrum.cz> wrote:

>
>
>
> On Wed, Apr 23, 2014 at 7:42 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:
>>
>> Storing the seed is superior to storing the master node already
>> (whether coin specific or not), as it is smaller.
>>
>>
> ...Except that you're loosing flexibility (serialization, deserialization)
> which gives you BIP32 node.
>
> I see "bip32 seed" as some transitional, internal state from raw entropy
> to bip32 master node and this seed should not be handled by the end user in
> any form. In the oposite, well-serialized bip32 node (in xpriv, or even in
> mnemonic format) can be used very widely and have no downsides against
> using raw "bip32 seed".
>
>
>>
>> Fair enough, it would break strictly BIP32. Then again, BIP32 is a
>> *Bitcoin* improvement proposal, and not something that necessarily
>> applies to other coins (they can adopt it of course, I don't care).
>>
>>
> I also don't care too much about altcoins, but people want them so me, as
> infrastructure developer, need to think about it. And I don't see any
> reason for breaking compatibility between Bitcoin and other altcoins. I
> would be happier if there will be another sentence than "Bitcoin seed", but
> honestly, who cares. It is just some magic string for hashing the raw
> seed...
>
>
>> What I dislike is that this removes the ability of using the magic in
>> the serialization to prevent importing a chain from the wrong coin.
>>
>
> The truth is that even existing software which handle bip32 don't care
> about 'version' at all. I think that "xpub/xprv" distinction is the only
> useful feature of version, so user se if it stores public or private
> information.
>
> But using prefixes which doesn't enforce anything is even more dangerous.
> If somebody exports node "dogeblablabla", it creates false exceptations
> that there's only dogecoin stored.
>
>  Marek
>

[-- Attachment #2: Type: text/html, Size: 3349 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 3166 bytes --]

Different users could have different gap limit requirements.  20 seems very
low as the default.

A merchant could easily send 20 addresses in a row to customers and none of
them bother to actually buy anything.

Setting the gap limit to high is just a small extra cost in that case.

Bip-32 serialization doesn't have a way of adding meta data though.


On Wed, Apr 23, 2014 at 7:18 PM, slush <slush@centrum.cz> wrote:

> For those who don't follow github pull requests regularly; there's pull
> request for BIP64 defining HD wallet structure as discussed in this thread:
>
> https://github.com/bitcoin/bips/pull/52
>
>
>
> On Wed, Apr 23, 2014 at 8:01 PM, slush <slush@centrum.cz> wrote:
>
>>
>>
>>
>> On Wed, Apr 23, 2014 at 7:42 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:
>>>
>>> Storing the seed is superior to storing the master node already
>>> (whether coin specific or not), as it is smaller.
>>>
>>>
>> ...Except that you're loosing flexibility (serialization,
>> deserialization) which gives you BIP32 node.
>>
>> I see "bip32 seed" as some transitional, internal state from raw entropy
>> to bip32 master node and this seed should not be handled by the end user in
>> any form. In the oposite, well-serialized bip32 node (in xpriv, or even in
>> mnemonic format) can be used very widely and have no downsides against
>> using raw "bip32 seed".
>>
>>
>>>
>>> Fair enough, it would break strictly BIP32. Then again, BIP32 is a
>>> *Bitcoin* improvement proposal, and not something that necessarily
>>> applies to other coins (they can adopt it of course, I don't care).
>>>
>>>
>> I also don't care too much about altcoins, but people want them so me, as
>> infrastructure developer, need to think about it. And I don't see any
>> reason for breaking compatibility between Bitcoin and other altcoins. I
>> would be happier if there will be another sentence than "Bitcoin seed", but
>> honestly, who cares. It is just some magic string for hashing the raw
>> seed...
>>
>>
>>> What I dislike is that this removes the ability of using the magic in
>>> the serialization to prevent importing a chain from the wrong coin.
>>>
>>
>> The truth is that even existing software which handle bip32 don't care
>> about 'version' at all. I think that "xpub/xprv" distinction is the only
>> useful feature of version, so user se if it stores public or private
>> information.
>>
>> But using prefixes which doesn't enforce anything is even more dangerous.
>> If somebody exports node "dogeblablabla", it creates false exceptations
>> that there's only dogecoin stored.
>>
>>  Marek
>>
>
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 5003 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 08:39 PM, Tier Nolan wrote:
> A merchant could easily send 20 addresses in a row to customers and none of
> them bother to actually buy anything.

Such merchant would surely use some merchant system instead of generic
wallet software.

> Setting the gap limit to high is just a small extra cost in that case.

Not if you have 100 accounts on 10 different devices.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 376 bytes --]

On Wed, Apr 23, 2014 at 7:46 PM, Pavol Rusnak <stick@gk2.sk> wrote:

>
> > Setting the gap limit to high is just a small extra cost in that case.
>
> Not if you have 100 accounts on 10 different devices.
>

I meant for a merchant with a server that is handing out hundreds of
addresses.

The point is to have a single system that is compatible over a large number
of systems.

[-- Attachment #2: Type: text/html, Size: 752 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 1484 bytes --]

Using higher gap limit in the software is not prohibited, but then it
breaks the standard "as is", in mean that importing such wallet to another
BIP64 wallet won't recover the funds properly, without proper settings of
gap limit...

Gap limit 20 is the most sane defaults for majority of users (actually I
think it is a bit higher than is really necessary for 99% users) and
majority of users won't need to change it at any time.

Marek


On Wed, Apr 23, 2014 at 9:00 PM, Tier Nolan <tier.nolan@gmail.com> wrote:

> On Wed, Apr 23, 2014 at 7:46 PM, Pavol Rusnak <stick@gk2.sk> wrote:
>
>>
>> > Setting the gap limit to high is just a small extra cost in that case.
>>
>> Not if you have 100 accounts on 10 different devices.
>>
>
> I meant for a merchant with a server that is handing out hundreds of
> addresses.
>
> The point is to have a single system that is compatible over a large
> number of systems.
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 2484 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 260 bytes --]

Pieter suggested in IRC couple of months ago to append key birth to key serialization in xprv….@unixtime format.

What about picking this idea up in BIP64? It would greatly help the importing client.

Regards,

Tamas Blummer
http://bitsofproof.com



[-- Attachment #1.2: Type: text/html, Size: 3497 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

We do not want BIP64 to be incompatible with BIP32 in any way. BIP64 is
just set of some recommendations for wallet developers how to browse bip32
tree.

Modifying serialization format would break the compatibility.

However we have our solution for storing wallet birth time, which is out of
scope of BIP64, but we'll communicate it as soon as we'll write down that
idea to some more specific format.

Marek


On Wed, Apr 23, 2014 at 9:36 PM, Tamas Blummer <tamas@bitsofproof.com>wrote:

> Pieter suggested in IRC couple of months ago to append key birth to key
> serialization in xprv....@unixtime format.
>
> What about picking this idea up in BIP64? It would greatly help the
> importing client.
>
> Regards,
>
> Tamas Blummer
> http://bitsofproof.com
>
>
>

[-- Attachment #2: Type: text/html, Size: 3430 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 827 bytes --]

I built such a merchant system handing out BIP32 addresses. 

The gap size problem does not arise there since such a system has to have an extra database keeping track of requests, so there is no added cost of storing the key coordinates used by them. A scan is not needed the keys can be accessed at random order.

Tamas Blummer
http://bitsofproof.com

On 23.04.2014, at 21:00, Tier Nolan <tier.nolan@gmail.com> wrote:

> On Wed, Apr 23, 2014 at 7:46 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> 
> > Setting the gap limit to high is just a small extra cost in that case.
> 
> Not if you have 100 accounts on 10 different devices.
> 
> I meant for a merchant with a server that is handing out hundreds of addresses.
>  
> The point is to have a single system that is compatible over a large number of systems.


[-- Attachment #1.2: Type: text/html, Size: 3468 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 09:00 PM, Tier Nolan wrote:
> The point is to have a single system that is compatible over a large number
> of systems.

There is such system and it is called BIP32.

On the other hand, in BIP64 we try to put a set of restrictions and
rules on top of BIP32. There will always be some special usecases where
BIP64 is not a good fit and there's no reason why you cannot use BIP32
in a different manner using a different "purpose" field.

Examples: Electrum does not want to use accounts and they start to use
scheme m/65'/change/address (where change = 0 or 1). Or Andreas
Schildbach wants to have refunds chain so he uses m/66'/chain/address
(where chain = 0, 1 or 2).

We wanted to find one good solution that fits all, but unfortunately it
turned out everyone wants something a little bit different.

The point of the whole effort is that you can have ONE SINGLE BACKUP
(master node) for all these independent purposes and suddenly claims
such as "my wallet is BIP64 and BIP66 compatible" actually mean
something as opposed to "BIP32 compatible" which actually means nothing
except that the wallet author is capable of using HMAC in context of
generating the tree.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 7:29:04 PM Pavol Rusnak wrote:
> On 04/23/2014 09:00 PM, Tier Nolan wrote:
> > The point is to have a single system that is compatible over a large
> > number of systems.
> 
> There is such system and it is called BIP32.
> 
> On the other hand, in BIP64 we try to put a set of restrictions and
> rules on top of BIP32. There will always be some special usecases where
> BIP64 is not a good fit and there's no reason why you cannot use BIP32
> in a different manner using a different "purpose" field.
> 
> Examples: Electrum does not want to use accounts and they start to use
> scheme m/65'/change/address (where change = 0 or 1). Or Andreas
> Schildbach wants to have refunds chain so he uses m/66'/chain/address
> (where chain = 0, 1 or 2).
> 
> We wanted to find one good solution that fits all, but unfortunately it
> turned out everyone wants something a little bit different.

Why do clients need to use the features in BIP 64? If Electrum doesn't want to 
use accounts, then it can just use account 0 for everything. Refund chains are 
definitely a third case that should be added to the external and 
internal/change address division... and a wallet not implementing refund 
addresses would simply not use that chain.

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 09:44 PM, Luke-Jr wrote:
> Why do clients need to use the features in BIP 64? If Electrum doesn't want to 
> use accounts, then it can just use account 0 for everything. Refund chains are 

As Andreas wrote earlier in this thread: "There is no "bare minimum".
Either you implement the "BIP" fully or not."

What you suggest does not follow the principle of least surprise.
Suppose user imports his BIP64 compatible wallet into Electrum, which
claims it is BIP64 compatible, but actually implements just a subset of
the spec (sticking account index to 0). The user now sees just a
fraction of his coins and is puzzled.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 7:49:38 PM Pavol Rusnak wrote:
> On 04/23/2014 09:44 PM, Luke-Jr wrote:
> > Why do clients need to use the features in BIP 64? If Electrum doesn't
> > want to use accounts, then it can just use account 0 for everything.
> > Refund chains are
> 
> As Andreas wrote earlier in this thread: "There is no "bare minimum".
> Either you implement the "BIP" fully or not."
> 
> What you suggest does not follow the principle of least surprise.
> Suppose user imports his BIP64 compatible wallet into Electrum, which
> claims it is BIP64 compatible, but actually implements just a subset of
> the spec (sticking account index to 0). The user now sees just a
> fraction of his coins and is puzzled.

Any wallet should import all the coins just fine, it just wouldn't *use* any 
account other than 0. Remember addresses are used to receive bitcoins; once 
the UTXOs are in the wallet, they are no longer associated with the address or 
any other details of how they were received.

Luke

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 437 bytes --]

On Wed, Apr 23, 2014 at 9:55 PM, Luke-Jr <luke@dashjr.org> wrote:

> Any wallet should import all the coins just fine, it just wouldn't *use*
> any
> account other than 0. Remember addresses are used to receive bitcoins; once
> the UTXOs are in the wallet, they are no longer associated with the
> address or
> any other details of how they were received.
>

Wallet don't see UTXO until it scans all branches/accounts on HD node
import.

[-- Attachment #2: Type: text/html, Size: 786 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 7:57:46 PM slush wrote:
> On Wed, Apr 23, 2014 at 9:55 PM, Luke-Jr <luke@dashjr.org> wrote:
> > Any wallet should import all the coins just fine, it just wouldn't *use*
> > any
> > account other than 0. Remember addresses are used to receive bitcoins;
> > once the UTXOs are in the wallet, they are no longer associated with the
> > address or
> > any other details of how they were received.
> 
> Wallet don't see UTXO until it scans all branches/accounts on HD node
> import.

Yes, it should scan all possible (under the BIP-defined structure) branches 
regardless of which features it supports.

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 10:01 PM, Luke-Jr wrote:
> Yes, it should scan all possible (under the BIP-defined structure) branches 
> regardless of which features it supports.

So you suggest to scan for accounts, show balances but don't allow user
to spend them? Does not seem right to me.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 8:04:03 PM Pavol Rusnak wrote:
> On 04/23/2014 10:01 PM, Luke-Jr wrote:
> > Yes, it should scan all possible (under the BIP-defined structure)
> > branches regardless of which features it supports.
> 
> So you suggest to scan for accounts, show balances but don't allow user
> to spend them? Does not seem right to me.

Scan all branches for UTXOs, then you have the balance for the wallet. Account 
balances are metadata, so cannot be known from the seed alone. If you want to 
have a way to restore accounts, you must define some more detailed wallet file 
format (which could be built on top of this).

On Wednesday, April 23, 2014 8:04:35 PM you wrote:
> On 23.04.2014, at 22:02, Luke-Jr <luke@dashjr.org> wrote:
> > On Wednesday, April 23, 2014 8:01:16 PM Tamas Blummer wrote:
> >> The wallet has to know how it got the UTXO in order to be able to spend
> >> it.
> > 
> > No it doesn't... Just the assigned scriptPubKey and secret(s) required to
> > satisfy it.
> 
> To know the secret it needs to know which key coordinate to use that is
> logically the same as knowing the address it used to receive it.

Sure, it *knows* what address was used to receive it. But at the point it's a 
UTXO, that address is no longer relevant.

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 10:09 PM, Luke-Jr wrote:
> Scan all branches for UTXOs, then you have the balance for the wallet. Account 
> balances are metadata, so cannot be known from the seed alone. If you want to 
> have a way to restore accounts, you must define some more detailed wallet file 
> format (which could be built on top of this).

I think one of the following is happening:

a) you have not read the document
b) you don't understand what accounts are, because you have not read
   the document
c) you don't understand how restoring accounts work, because you
   have not read the document
d) you don't understand that accounts funds are not meant to be mixed
   together, because you have not read the document
e) I got your emails wrong because I am not a native speaker,
   but please read the the document ;-)

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 8:16:45 PM Pavol Rusnak wrote:
> On 04/23/2014 10:09 PM, Luke-Jr wrote:
> > Scan all branches for UTXOs, then you have the balance for the wallet.
> > Account balances are metadata, so cannot be known from the seed alone.
> > If you want to have a way to restore accounts, you must define some more
> > detailed wallet file format (which could be built on top of this).
> 
> I think one of the following is happening:
> 
> a) you have not read the document
> b) you don't understand what accounts are, because you have not read
>    the document
> c) you don't understand how restoring accounts work, because you
>    have not read the document
> d) you don't understand that accounts funds are not meant to be mixed
>    together, because you have not read the document
> e) I got your emails wrong because I am not a native speaker,
>    but please read the the document ;-)

You missed one:
f) I missed the part where BIP 32 redefined "account" to mean "subwallet" 
instead of what has traditionally been called "account" in Bitcoin.

:)

In that case, single-subwallet wallet software probably needs to have the user 
provide a subwallet number in addition to the seed.

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 10:32 PM, Luke-Jr wrote:
> f) I missed the part where BIP 32 redefined "account" to mean "subwallet" 
> instead of what has traditionally been called "account" in Bitcoin.

Ah, okay. The last time I saw Bitcoin-qt it was still using independent
addresses.

> In that case, single-subwallet wallet software probably needs to have the user 
> provide a subwallet number in addition to the seed.

Which brings us back to my original complaint that the user can be
confused because he doesn't see all his funds.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 8:35:50 PM Pavol Rusnak wrote:
> On 04/23/2014 10:32 PM, Luke-Jr wrote:
> > f) I missed the part where BIP 32 redefined "account" to mean "subwallet"
> > instead of what has traditionally been called "account" in Bitcoin.
> 
> Ah, okay. The last time I saw Bitcoin-qt it was still using independent
> addresses.

It is. Accounts have been a bitcoind feature since before 0.4.

> > In that case, single-subwallet wallet software probably needs to have the
> > user provide a subwallet number in addition to the seed.
> 
> Which brings us back to my original complaint that the user can be
> confused because he doesn't see all his funds.

I don't see how. The user knows he has money in different subwallets. As long 
as he has a way to specify which subwallet he is accessing in single-subwallet 
clients, there shouldn't be a problem.

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 10:41 PM, Luke-Jr wrote:
> I don't see how. The user knows he has money in different subwallets. As long 
> as he has a way to specify which subwallet he is accessing in single-subwallet 
> clients, there shouldn't be a problem.

Right. But these clients have no right to call themselves BIP64
compatible then.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

On Wed, Apr 23, 2014 at 10:43 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> On 04/23/2014 10:41 PM, Luke-Jr wrote:
>> I don't see how. The user knows he has money in different subwallets. As long
>> as he has a way to specify which subwallet he is accessing in single-subwallet
>> clients, there shouldn't be a problem.
>
> Right. But these clients have no right to call themselves BIP64
> compatible then.

Would you consider software which scans all accounts as specified by
BIP64, but has no user interface option to distinguish them in any
way, view them independently, and has no ability to keep the coins
apart... compatible with BIP64?

According to the argument here mentioned earlier ("all or nothing"),
it is, as it will not break interoperability with other BIP64
software. Still, it doesn't support the accounts feature, and perhaps
that's fine?

-- 
Pieter

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 10:54 PM, Pieter Wuille wrote:
> Would you consider software which scans all accounts as specified by
> BIP64, but has no user interface option to distinguish them in any
> way, view them independently, and has no ability to keep the coins
> apart... compatible with BIP64?

This is not a desired behavior. Do you have any idea how to make it even
more explicit in the spec? Currently we just have (in Account sectrion):

"This level splits the key space into independent user identities, so
the wallet never mixes the coins across different accounts."


I would like to make it obvious from the spec that if you mix funds
accross the accounts you are not doing a right thing and you are not
compliant to the spec.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 9:06:26 PM Pavol Rusnak wrote:
> On 04/23/2014 10:54 PM, Pieter Wuille wrote:
> > Would you consider software which scans all accounts as specified by
> > BIP64, but has no user interface option to distinguish them in any
> > way, view them independently, and has no ability to keep the coins
> > apart... compatible with BIP64?
> 
> This is not a desired behavior. Do you have any idea how to make it even
> more explicit in the spec? Currently we just have (in Account sectrion):
> 
> "This level splits the key space into independent user identities, so
> the wallet never mixes the coins across different accounts."
> 
> 
> I would like to make it obvious from the spec that if you mix funds
> accross the accounts you are not doing a right thing and you are not
> compliant to the spec.

How do you get the more expected/usual behaviour of mixing funds between 
accounts? Only a very niche user base needs funds isolated...

Luke

Re: [Bitcoin-development] New BIP32 structure

[Gregory Maxwell]

On Wed, Apr 23, 2014 at 2:18 PM, Luke-Jr <luke@dashjr.org> wrote:
> How do you get the more expected/usual behaviour of mixing funds between
> accounts? Only a very niche user base needs funds isolated...

Hopefully it would be clarified as only a MUST NOT do so silently...
"I have funds split across two wallets and it WONT LET ME SPEND THEM"
sounds like a terrible user experience. :)

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 11:22 PM, Gregory Maxwell wrote:
> Hopefully it would be clarified as only a MUST NOT do so silently...
> "I have funds split across two wallets and it WONT LET ME SPEND THEM"
> sounds like a terrible user experience. :)

That is a subjective matter. To me it makes PERFECT SENSE that funds
across accounts NEVER MIX. One can still send funds from one account to
another and then perform another spend.

That's what I consider a consistent and thus GOOD user experience.
What's the purpose of Accounts if wallet mixes coins among them anyway?

I know it's not good to use classic bank accounts as analogy, but that's
exactly how they work. Or have you every done ONE transaction from two
bank accounts simultaneously?

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

On Wed, Apr 23, 2014 at 11:33 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> On 04/23/2014 11:22 PM, Gregory Maxwell wrote:
>> Hopefully it would be clarified as only a MUST NOT do so silently...
>> "I have funds split across two wallets and it WONT LET ME SPEND THEM"
>> sounds like a terrible user experience. :)
>
> That is a subjective matter. To me it makes PERFECT SENSE that funds
> across accounts NEVER MIX. One can still send funds from one account to
> another and then perform another spend.

In that case, maybe it makes sense to define another purpose id
without accounts as well already.

I believe many simple wallets will find multiple subwallets too
burdening for the user experience, or not worth the technical
complexity.

-- 
Pieter

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 11:42 PM, Pieter Wuille wrote:
> In that case, maybe it makes sense to define another purpose id
> without accounts as well already.
> 
> I believe many simple wallets will find multiple subwallets too
> burdening for the user experience, or not worth the technical
> complexity.

Right. See my previous email where I describe hypothetical creation of
BIP65 and BIP66 which the exact thing you describe.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Gregory Maxwell]

On Wed, Apr 23, 2014 at 2:42 PM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
> In that case, maybe it makes sense to define another purpose id
> without accounts as well already.
>
> I believe many simple wallets will find multiple subwallets too
> burdening for the user experience, or not worth the technical
> complexity.

Or implement them but in a form where the different wallets can have
different security policies and thus wouldn't share a common piece of
private key material.  I can see it being pretty confusing to have
multiple wallets which are both sharing a private key and not sharing
a private key.

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 9:33:48 PM Pavol Rusnak wrote:
> On 04/23/2014 11:22 PM, Gregory Maxwell wrote:
> > Hopefully it would be clarified as only a MUST NOT do so silently...
> > "I have funds split across two wallets and it WONT LET ME SPEND THEM"
> > sounds like a terrible user experience. :)
> 
> That is a subjective matter. To me it makes PERFECT SENSE that funds
> across accounts NEVER MIX. One can still send funds from one account to
> another and then perform another spend.

Only by crossing the blockchain unnecessarily.

> That's what I consider a consistent and thus GOOD user experience.
> What's the purpose of Accounts if wallet mixes coins among them anyway?

To keep balances separated. For example, I might have an account for each of 
my children, move their allowance money to it every day/week (off-chain), and 
they can spend only what they have in their account. Or an exchange might have 
an account for each user's deposits. In neither case, would it make sense to 
pay special attention to which UTXOs are consumed in transactions from any of 
the account.

The only use case that requires tracking UTXOs per-account is when you want to 
have multiple uncoordinated copies of the wallet in use at the same time, and 
therefore need to immediately identify which account a transaction came from 
based on the UTXOs it consumed - although even then, you could still mix funds 
as long as you use only the first UTXO for tracking, so maybe there isn't even 
a niche use case! In any case, Trezor, being a hardware wallet, should never 
overlap with this niche...?

> I know it's not good to use classic bank accounts as analogy, but that's
> exactly how they work. Or have you every done ONE transaction from two
> bank accounts simultaneously?

Bad analogy. Banks *always* mix funds. You don't expect your bank wire to use 
exactly the specific dollar bills you deposited, do you??

Luke

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 04/23/2014 11:18 PM, Luke-Jr wrote:
> Only a very niche user base needs funds isolated...

Our users do and we are creating this BIP for them, because we love them. ;)

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

On Wed, Apr 23, 2014 at 10:54 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

>
> Would you consider software which scans all accounts as specified by
> BIP64, but has no user interface option to distinguish them in any
> way, view them independently, and has no ability to keep the coins
> apart... compatible with BIP64?
>
>
No, because one of requirement of BIP64 is to do not mix addressess between
accounts. Without handling those accounts fully, it won't pass this
requirement.

("This level [accounts] splits the key space into independent user
identities, so the wallet never mixes the coins across different accounts.")


Marek

[-- Attachment #2: Type: text/html, Size: 1273 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 8:43:57 PM Pavol Rusnak wrote:
> On 04/23/2014 10:41 PM, Luke-Jr wrote:
> > I don't see how. The user knows he has money in different subwallets. As
> > long as he has a way to specify which subwallet he is accessing in
> > single-subwallet clients, there shouldn't be a problem.
> 
> Right. But these clients have no right to call themselves BIP64
> compatible then.

Then BIP 64 is pretty restrictive. Most end users really have no need for 
subwallet support.

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

On 23.04.2014, at 22:09, Luke-Jr <luke@dashjr.org> wrote:

> On Wednesday, April 23, 2014 8:04:03 PM Pavol Rusnak wrote:
>> On 04/23/2014 10:01 PM, Luke-Jr wrote:
>>> Yes, it should scan all possible (under the BIP-defined structure)
>>> branches regardless of which features it supports.
>> 
>> So you suggest to scan for accounts, show balances but don't allow user
>> to spend them? Does not seem right to me.
> 
> Scan all branches for UTXOs, then you have the balance for the wallet. Account 
> balances are metadata, so cannot be known from the seed alone. If you want to 
> have a way to restore accounts, you must define some more detailed wallet file 
> format (which could be built on top of this).
> 
> On Wednesday, April 23, 2014 8:04:35 PM you wrote:
>> On 23.04.2014, at 22:02, Luke-Jr <luke@dashjr.org> wrote:
>>> On Wednesday, April 23, 2014 8:01:16 PM Tamas Blummer wrote:
>>>> The wallet has to know how it got the UTXO in order to be able to spend
>>>> it.
>>> 
>>> No it doesn't... Just the assigned scriptPubKey and secret(s) required to
>>> satisfy it.
>> 
>> To know the secret it needs to know which key coordinate to use that is
>> logically the same as knowing the address it used to receive it.
> 
> Sure, it *knows* what address was used to receive it. But at the point it's a 
> UTXO, that address is no longer relevant.
> 

In context of BIP32 one does not store secrets but re-create them on-the-fly if needed using key coordinates known to the UTXO.
Individual secrets per UTXO are about as irrelevant and accessible as addresses.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

I believe Luke means scanning chains as defined by the structure, but not
handing out addresses from other accounts than the first one.

That's certainly a possibly way to compatibly implement BIP64, but it
doesn't reduce all that much complexity. I hope people would choose that
over defining their own accountless structure though.
On 23 Apr 2014 22:06, "Pavol Rusnak" <stick@gk2.sk> wrote:

> On 04/23/2014 10:01 PM, Luke-Jr wrote:
> > Yes, it should scan all possible (under the BIP-defined structure)
> branches
> > regardless of which features it supports.
>
> So you suggest to scan for accounts, show balances but don't allow user
> to spend them? Does not seem right to me.
>
> --
> Best Regards / S pozdravom,
>
> Pavol Rusnak <stick@gk2.sk>
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 1926 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 1805 bytes --]

That's the point. BIP64 specifies such a structure, and you have to scan
all that it defines.

If you want to write wallet software that does not have the complexity to
deal with just one account, it is not BIP64 compliant. It could try to
define its own purpose system, with a hierarchy without accounts in it. I'm
not sure this is a very interesting use case, but I like how strict it is.
Construction of related chains for multisig addresses is perhaps a better
example of a different purpose structure.
 On 23 Apr 2014 22:03, "Luke-Jr" <luke@dashjr.org> wrote:

> On Wednesday, April 23, 2014 7:57:46 PM slush wrote:
> > On Wed, Apr 23, 2014 at 9:55 PM, Luke-Jr <luke@dashjr.org> wrote:
> > > Any wallet should import all the coins just fine, it just wouldn't
> *use*
> > > any
> > > account other than 0. Remember addresses are used to receive bitcoins;
> > > once the UTXOs are in the wallet, they are no longer associated with
> the
> > > address or
> > > any other details of how they were received.
> >
> > Wallet don't see UTXO until it scans all branches/accounts on HD node
> > import.
>
> Yes, it should scan all possible (under the BIP-defined structure) branches
> regardless of which features it supports.
>
> Luke
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 2474 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]


On 23.04.2014, at 21:55, Luke-Jr <luke@dashjr.org> wrote:
> Any wallet should import all the coins just fine, it just wouldn't *use* any 
> account other than 0. Remember addresses are used to receive bitcoins; once 
> the UTXOs are in the wallet, they are no longer associated with the address or 
> any other details of how they were received.

This is a bit idealistic. 
The wallet has to know how it got the UTXO in order to be able to spend it.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Luke-Jr]

On Wednesday, April 23, 2014 8:01:16 PM Tamas Blummer wrote:
> On 23.04.2014, at 21:55, Luke-Jr <luke@dashjr.org> wrote:
> > Any wallet should import all the coins just fine, it just wouldn't *use*
> > any account other than 0. Remember addresses are used to receive
> > bitcoins; once the UTXOs are in the wallet, they are no longer
> > associated with the address or any other details of how they were
> > received.
> 
> This is a bit idealistic.
> The wallet has to know how it got the UTXO in order to be able to spend it.

No it doesn't... Just the assigned scriptPubKey and secret(s) required to 
satisfy it.

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1: Type: text/plain, Size: 845 bytes --]

On 23.04.2014, at 22:02, Luke-Jr <luke@dashjr.org> wrote:

> On Wednesday, April 23, 2014 8:01:16 PM Tamas Blummer wrote:
>> On 23.04.2014, at 21:55, Luke-Jr <luke@dashjr.org> wrote:
>>> Any wallet should import all the coins just fine, it just wouldn't *use*
>>> any account other than 0. Remember addresses are used to receive
>>> bitcoins; once the UTXOs are in the wallet, they are no longer
>>> associated with the address or any other details of how they were
>>> received.
>> 
>> This is a bit idealistic.
>> The wallet has to know how it got the UTXO in order to be able to spend it.
> 
> No it doesn't... Just the assigned scriptPubKey and secret(s) required to 
> satisfy it.
> 

To know the secret it needs to know which key coordinate to use that is logically the same as knowing the address it used to receive it.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 23/04/2014 21:44, Luke-Jr a écrit :
> On Wednesday, April 23, 2014 7:29:04 PM Pavol Rusnak wrote:
>>
>> Examples: Electrum does not want to use accounts [...]

> Why do clients need to use the features in BIP 64? If Electrum doesn't want to 
> use accounts, [...]

To clarify:
Electrum plans to have bip32 accounts; Multibit will not, afaik.

I think Pavol just wanted to make an example.

Re: [Bitcoin-development] New BIP32 structure

[Pieter Wuille]

On Thu, Apr 24, 2014 at 8:54 AM, Thomas Voegtlin <thomasv1@gmx.de> wrote:
>> Why do clients need to use the features in BIP 64? If Electrum doesn't want to
>> use accounts, [...]
>
> To clarify:
> Electrum plans to have bip32 accounts; Multibit will not, afaik.

To clarify:
BIP64 has a much stricter definition for accounts than BIP32.

In BIP32, it is not well specified what accounts are used for. They
can be used for "subwallets", "receive accounts" (as in bitcoind's
account feature), "recurring payments", part of a chain used as
multisig addresses, ... determined individually for each index.

In BIP64, they are strictly used for subwallets, and can't be used by
anything else.

-- 
Pieter

Re: [Bitcoin-development] New BIP32 structure

[Gregory Maxwell]

On Thu, Apr 24, 2014 at 12:10 AM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
> On Thu, Apr 24, 2014 at 8:54 AM, Thomas Voegtlin <thomasv1@gmx.de> wrote:
>>> Why do clients need to use the features in BIP 64? If Electrum doesn't want to
>>> use accounts, [...]
>>
>> To clarify:
>> Electrum plans to have bip32 accounts; Multibit will not, afaik.
>
> To clarify:
> BIP64 has a much stricter definition for accounts than BIP32.
>
> In BIP32, it is not well specified what accounts are used for. They
> can be used for "subwallets", "receive accounts" (as in bitcoind's
> account feature), "recurring payments", part of a chain used as
> multisig addresses, ... determined individually for each index.
>
> In BIP64, they are strictly used for subwallets, and can't be used by
> anything else.

It doesn't appear to me that reoccurring payments, receive accounts,
multisig addresses, etc can be used with this proposal, but instead
you must use a different purpose code and another BIP and are not
compatible with the draft here.

Am I misunderstanding it?   Will Electrum be limiting itself in this
way?  I'd consider it a unfortunate loss of functionality if wallets
couldn't implement reoccurring payment chains without making users
generate entirely different wallets (which they couldn't share funds
across) since addresses for recurring payments was one of the main
motivations in BIP32.

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 24/04/2014 09:21, Gregory Maxwell a écrit :
> 
> It doesn't appear to me that reoccurring payments, receive accounts,
> multisig addresses, etc can be used with this proposal, but instead
> you must use a different purpose code and another BIP and are not
> compatible with the draft here.
> 
> Am I misunderstanding it?   Will Electrum be limiting itself in this
> way?  I'd consider it a unfortunate loss of functionality if wallets
> couldn't implement reoccurring payment chains without making users
> generate entirely different wallets (which they couldn't share funds
> across) since addresses for recurring payments was one of the main
> motivations in BIP32.
> 
> 

No, Electrum will not be limiting itself in this way. I believe that we
are only at the beginning of exploring the different possibilities
opened by HD wallets. It will probably take years until we have clear
ideas on what users need, what choices they make, and how to organize
everything. Therefore it is too early to take decisions that might limit
future functionality.

I can see that it is very difficult today to find a consensus on wallet
structure between wallet developers. In addition, I changed my mind
several times on these questions, so I guess I will probably need to
change things again in the future.

This is why I decided to include a version number in Electrum seeds. The
version number will be updated everytime the wallet structure changes. I
know many developers do not follow me on this, but that is something I
am quite sure Electrum needs, despite all the other things I am not sure
about :)

I think it is too early to aim for inter-wallet compatibility today. I
guess we should postpone this goal, and move on with software releases.
As Andreas pointed out, we should just make sure that we do not import
an incompatible seed in another wallet, because not recovering all your
bitcoins would be a terrible user experience; the version number built
in the seed will ensure that for Electrum.

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 24/04/2014 09:10, Pieter Wuille a écrit :
> To clarify:
> BIP64 has a much stricter definition for accounts than BIP32.
> 
> In BIP32, it is not well specified what accounts are used for. They
> can be used for "subwallets", "receive accounts" (as in bitcoind's
> account feature), "recurring payments", part of a chain used as
> multisig addresses, ... determined individually for each index.
> 
> In BIP64, they are strictly used for subwallets, and can't be used by
> anything else.
> 

yes, I saw that.

In particular, bip64 stipulates that the wallet "never mixes coins
across different accounts". This is not what Electrum does currently.
The UI allows you to chose between two modes: activate a single account
(and the wallet will only use UTXOs from that acccount), or activate all
accounts (and spend from all of them simultaneously).

Concerning multisig addresses, I have changed my mind: Electrum will use
parallel BIP32 trees. A wallet will not mix standard and multisig
accounts. I think that is better in terms of UX.

I agree with Mike Hearn's view that wallets with multiple accounts are
probably too difficult to deal with for most users. If a user feels the
need to have different "accounting identities", they will probably
create different wallet files, instead of creating bip32 subwallets.

However, since multiple subwallets have been asked by many users,
Electrum will propose them. But this should not be the default. More
important, multiple accounts should never be required (in my previous
implementation, they were required for multisig, because the wallet was
creating multisig addresses in dedicated multisig accounts)

Thomas

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 4585 bytes --]

Right. So part of this is my fault, I'm afraid, because I do not intend to
implement any kind of subwallet/account support in bitcoinj. My reasons are:

   1. The bitcoinj API already lets you create and use multiple wallets.
   What's more, because of the desire to do key rotation (think rotating a
   previously unencrypted wallet to an encrypted one that is stored on SSD's
   that cannot reliably erase data), a bitcoinj wallet can actually contain
   multiple BIP32 seeds and hierarchies at once, although only the last one
   will be used for vending addresses. So adding subwallet support onto this
   makes it even more complicated.

   2. If there was a much better user experience to be enabled by this, it
   may be worth it, but I believe many people will find subwallets rather
   confusing. They don't match the analogy of bank accounts in several ways.
   For instance, transferring money across them leaks private data and costs
   miners fees, neither of which are true with banks.

   Also it differs in a more important way. People have different bank
   accounts because those accounts implement different policies. Current
   accounts may pay a lower interest rate than savings accounts, but have
   different features, and accounts can be used as security boundaries i.e. no
   card withdrawals from savings. But "subwallets" are not like this. The only
   justification for their existence is to avoid outputs being merged together
   to make payments - a subtle technical detail of the protocol that users are
   ill equipped to understand. If someone asked me "why should I create a
   second account" I would be unable to give them a satisfying answer without
   first teaching them about how the Bitcoin protocol works and the privacy
   implications of that, which is practically a lecture sized topic.

   3. MultiBit did support multiple wallets for a long time (just by
   creating multiple wallet files and using the support in bitcoinj for
   running them in parallel), but they decided to remove this feature in
   MultiBit HD because it caused support headaches. People would stash money
   in one wallet or the other, close the wallet and then forget and think they
   had lost it, etc. It may be that TREZOR type subwallets don't suffer this
   confusion because they can't be moved around or "closed" in the same way a
   file can be, but still, this is a data point against multiple simultaneous
   wallets. At least for products targeting entry level consumers.

Whilst I can well believe there are TREZOR users who are asking for this
feature today, currently the costs feel a bit higher than the benefits.

It would be rather nice to be able to type in a mnemonic code that myTREZOR
was initialised with and duplicate that wallet into a bitcoinj based wallet
app. But if I have to implement subwallets and expose this in the API, and
if all wallet authors that want to be able to share a wallet with myTREZOR
have to expose subwallets in their GUIs too, even though the concept may
prove confusing and hard to explain, then it might be more tempting to just
tell users that want to switch wallet apps to send the money via the block
chain instead.




On Thu, Apr 24, 2014 at 9:10 AM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> On Thu, Apr 24, 2014 at 8:54 AM, Thomas Voegtlin <thomasv1@gmx.de> wrote:
> >> Why do clients need to use the features in BIP 64? If Electrum doesn't
> want to
> >> use accounts, [...]
> >
> > To clarify:
> > Electrum plans to have bip32 accounts; Multibit will not, afaik.
>
> To clarify:
> BIP64 has a much stricter definition for accounts than BIP32.
>
> In BIP32, it is not well specified what accounts are used for. They
> can be used for "subwallets", "receive accounts" (as in bitcoind's
> account feature), "recurring payments", part of a chain used as
> multisig addresses, ... determined individually for each index.
>
> In BIP64, they are strictly used for subwallets, and can't be used by
> anything else.
>
> --
> Pieter
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 5519 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 596 bytes --]

The most useful meta data to optimize chain scan is the key birth date, then the allowed gap size. 

Tamas Blummer
http://bitsofproof.com

On 23.04.2014, at 20:39, Tier Nolan <tier.nolan@gmail.com> wrote:

> Different users could have different gap limit requirements.  20 seems very low as the default.
> 
> A merchant could easily send 20 addresses in a row to customers and none of them bother to actually buy anything.
> 
> Setting the gap limit to high is just a small extra cost in that case.
> 
> Bip-32 serialization doesn't have a way of adding meta data though.
> 
> 


[-- Attachment #1.2: Type: text/html, Size: 2616 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Andreas Schildbach]

On 04/08/2014 02:43 PM, slush wrote:

> After some off-list discussion about details with wallet developers, it
> seems that structure
> 
> m/<cointype>'/<account>'/<change>/<n> 
> 
> fulfill requirements of all wallet developers around, including
> myTrezor, Electrum, Multibit, Wallet32 and other software is willing to
> adapt once anything will be standardized (i.e. they don't care).
> 
> Because I think that everybody told their comments to the topic already
> and because it seems that there's quite wide agreement on that, I would
> like to close the discussion and finally implement these paths into our
> software.

While there is an agreement that a standard would be useful for sharing
wallets, we certainly didn't agree on every aspect of a standard. At
least not on this thread, and also not at the Berlin meeting.

I understand each client will implement things a little bit different,
for example the current plan is bitcoinj will hold all keys in memory
and start reusing keys on low resources. Electrum uses a chain for their
private purpose. Etc.

If we cannot 100% agree on a standard and also agree it will not be
extended in future, I think we should not even try. It's dangerous for
the money of users.

I propose we keep our chains deliberately separate and only try
standardizing after each of us has a feel for the specific requirements.

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 1750 bytes --]

On Tue, Apr 8, 2014 at 4:49 PM, Andreas Schildbach <andreas@schildbach.de>wrote:

> While there is an agreement that a standard would be useful for sharing
> wallets, we certainly didn't agree on every aspect of a standard. At
> least not on this thread, and also not at the Berlin meeting.
>
>
We're going to write down BIP describing such structure. If any wallet want
to be BIP XX compatible, then it has chance to. Of course if any wallet
want to use another format, then it cannot call itself BIP XX compatible,
thus nobody will expect that such software will see/recover all keys
generated by BIP XX wallet.


> I understand each client will implement things a little bit different,
> for example the current plan is bitcoinj will hold all keys in memory
> and start reusing keys on low resources. Electrum uses a chain for their
> private purpose. Etc.
>
>
It still doesn't mean that bitcoinj or Electrum cannot share the bare
minimum of BIP XX. Of course if somebody will use Electrum for 2to3
transactions and then move wallet to client which does not offer such
feature, it won't work. But I don't see that as a problem.


> If we cannot 100% agree on a standard and also agree it will not be
> extended in future, I think we should not even try. It's dangerous for
> the money of users.


If some developers agree on some specific BIP, then it should be cross
compatible.  Of course if somebody implements BIP XX in different way, then
it isn't BIP XX compatible.


>
>
I propose we keep our chains deliberately separate and only try
> standardizing after each of us has a feel for the specific requirements.
>

Of course, if somebody don't want to generate compatible bip32 paths, no
problem. It will be the same situation as now.

Marek

[-- Attachment #2: Type: text/html, Size: 2847 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Andreas Schildbach]

On 04/08/2014 05:46 PM, slush wrote:

>     I understand each client will implement things a little bit different,
>     for example the current plan is bitcoinj will hold all keys in memory
>     and start reusing keys on low resources. Electrum uses a chain for their
>     private purpose. Etc.
> 
> It still doesn't mean that bitcoinj or Electrum cannot share the bare
> minimum of BIP XX. Of course if somebody will use Electrum for 2to3
> transactions and then move wallet to client which does not offer such
> feature, it won't work. But I don't see that as a problem.

There is no "bare minimum". Either you implement the "BIP" fully or not.
There is no inbetween. Likewise, the standard cannot be extended unless
you create a new standard that is based on the old (without re-using the
path, of course).

We're lightyears away from a BIP. Lets first create implementations and
see if they are compatible in all possible combinations and situations.
The moment any two apps have a different view on their wallets generated
from the same seed, they're incompatible. In this case they should
either fix the issue or intentionally choose incompatible paths, so that
they don't see and spend "subsets" of each other.

Re: [Bitcoin-development] New BIP32 structure

[slush]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On Tue, Apr 8, 2014 at 5:58 PM, Andreas Schildbach <andreas@schildbach.de>wrote:

> On 04/08/2014 05:46 PM, slush wrote:
>
> > It still doesn't mean that bitcoinj or Electrum cannot share the bare
> > minimum of BIP XX. Of course if somebody will use Electrum for 2to3
> > transactions and then move wallet to client which does not offer such
> > feature, it won't work. But I don't see that as a problem.
>
> There is no "bare minimum". Either you implement the "BIP" fully or not.
>

I didn't mean "Bare minimum of some spec", but "spec describing bare
minimum", which is different. I'm sorry for confusion.

Marek

[-- Attachment #2: Type: text/html, Size: 1088 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 26/03/2014 21:49, Mike Hearn a écrit :
>
>   * reserved is for "other stuff". I actually don't recall why we ended
>     up with this. It may have been intended to split out multisig
>     outputs etc from cointype. Marek, Thomas?
>


yes, this was intended to create multisig addresses from the same seed.
cointype was proposed after that.

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 27/03/2014 12:30, Marek Palatinus a écrit :
> Ah, I forget to two things, which should be into the BIP as well:
>
> a) Gap factor for addresses; as Thomas mentioned, although some software
> can watch almost unlimited amount of unused addresses, this is serious
> concern for lightweight or server-based wallets like Electrum or
> myTREZOR. myTREZOR currently uses gap factor 10, which is (from my
> experience so far) quite sane for most of users.


Yes, I was planning to increase the number of available unused addresses 
to 10 or 20 in the bip32 version of Electrum.

Related to this, here is another idea I would like to submit:

Instead of using a "gap limit" (maximal number of consecutive unused 
addresses), I think we should get rid of the topology, and simply count 
the number of unused addresses since the beginning of the sequence. 
Indeed, the topology of the sequence of addresses is of no interest to 
the user. Users often misinterpret "gap limit" as the "number of unused 
addresses available", so I think we should just give them what they want 
:) This is easier to understand, and it makes things more predictable, 
because the wallet will always display the same number of unused 
addresses (except when it is waiting for confirmations).

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]

By the way, I just noticed that greenaddress.it is creating seeds that have
24 words instead of 12. Does anyone know what's up with that? They claim to
be using BIP32 wallets so I wanted to see if they were using the default
structure and if so, whether bitcoinj was compatible with it (before I
switch to the one discussed here). But it seems we fall at the first hurdle
...


On Thu, Mar 27, 2014 at 1:06 PM, Thomas Voegtlin <thomasv1@gmx.de> wrote:

>
>
> Le 27/03/2014 12:30, Marek Palatinus a écrit :
> > Ah, I forget to two things, which should be into the BIP as well:
> >
> > a) Gap factor for addresses; as Thomas mentioned, although some software
> > can watch almost unlimited amount of unused addresses, this is serious
> > concern for lightweight or server-based wallets like Electrum or
> > myTREZOR. myTREZOR currently uses gap factor 10, which is (from my
> > experience so far) quite sane for most of users.
>
>
> Yes, I was planning to increase the number of available unused addresses
> to 10 or 20 in the bip32 version of Electrum.
>
> Related to this, here is another idea I would like to submit:
>
> Instead of using a "gap limit" (maximal number of consecutive unused
> addresses), I think we should get rid of the topology, and simply count
> the number of unused addresses since the beginning of the sequence.
> Indeed, the topology of the sequence of addresses is of no interest to
> the user. Users often misinterpret "gap limit" as the "number of unused
> addresses available", so I think we should just give them what they want
> :) This is easier to understand, and it makes things more predictable,
> because the wallet will always display the same number of unused
> addresses (except when it is waiting for confirmations).
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 2740 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2560 bytes --]

Ah, BIP32 allows for a range of entropy sizes and it so happens that they
picked 256 bits instead of 128 bits.

I'd have thought that there is a right answer for this. 2^128 should not be
brute forceable, and longer sizes have a cost in terms of making the seeds
harder to write down on paper. So should this be a degree of freedom?


On Thu, Mar 27, 2014 at 1:28 PM, Mike Hearn <mike@plan99.net> wrote:

> By the way, I just noticed that greenaddress.it is creating seeds that
> have 24 words instead of 12. Does anyone know what's up with that? They
> claim to be using BIP32 wallets so I wanted to see if they were using the
> default structure and if so, whether bitcoinj was compatible with it
> (before I switch to the one discussed here). But it seems we fall at the
> first hurdle ...
>
>
> On Thu, Mar 27, 2014 at 1:06 PM, Thomas Voegtlin <thomasv1@gmx.de> wrote:
>
>>
>>
>> Le 27/03/2014 12:30, Marek Palatinus a écrit :
>> > Ah, I forget to two things, which should be into the BIP as well:
>> >
>> > a) Gap factor for addresses; as Thomas mentioned, although some software
>> > can watch almost unlimited amount of unused addresses, this is serious
>> > concern for lightweight or server-based wallets like Electrum or
>> > myTREZOR. myTREZOR currently uses gap factor 10, which is (from my
>> > experience so far) quite sane for most of users.
>>
>>
>> Yes, I was planning to increase the number of available unused addresses
>> to 10 or 20 in the bip32 version of Electrum.
>>
>> Related to this, here is another idea I would like to submit:
>>
>> Instead of using a "gap limit" (maximal number of consecutive unused
>> addresses), I think we should get rid of the topology, and simply count
>> the number of unused addresses since the beginning of the sequence.
>> Indeed, the topology of the sequence of addresses is of no interest to
>> the user. Users often misinterpret "gap limit" as the "number of unused
>> addresses available", so I think we should just give them what they want
>> :) This is easier to understand, and it makes things more predictable,
>> because the wallet will always display the same number of unused
>> addresses (except when it is waiting for confirmations).
>>
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>

[-- Attachment #2: Type: text/html, Size: 3528 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Kerin]

[-- Attachment #1: Type: text/plain, Size: 3275 bytes --]

Isn't the length of the seed arbitrary anyway? Once decoded using whatever
mnemonic implementation (electrums, or BIP0039) the bytestream is
immediately passed to HMAC-SHA256 to generate the master key. No matter
what your initial entropy is, it would be hashed anyway.


On Thu, Mar 27, 2014 at 12:49 PM, Mike Hearn <mike@plan99.net> wrote:

> Ah, BIP32 allows for a range of entropy sizes and it so happens that they
> picked 256 bits instead of 128 bits.
>
> I'd have thought that there is a right answer for this. 2^128 should not
> be brute forceable, and longer sizes have a cost in terms of making the
> seeds harder to write down on paper. So should this be a degree of freedom?
>
>
> On Thu, Mar 27, 2014 at 1:28 PM, Mike Hearn <mike@plan99.net> wrote:
>
>> By the way, I just noticed that greenaddress.it is creating seeds that
>> have 24 words instead of 12. Does anyone know what's up with that? They
>> claim to be using BIP32 wallets so I wanted to see if they were using the
>> default structure and if so, whether bitcoinj was compatible with it
>> (before I switch to the one discussed here). But it seems we fall at the
>> first hurdle ...
>>
>>
>> On Thu, Mar 27, 2014 at 1:06 PM, Thomas Voegtlin <thomasv1@gmx.de> wrote:
>>
>>>
>>>
>>> Le 27/03/2014 12:30, Marek Palatinus a écrit :
>>> > Ah, I forget to two things, which should be into the BIP as well:
>>> >
>>> > a) Gap factor for addresses; as Thomas mentioned, although some
>>> software
>>> > can watch almost unlimited amount of unused addresses, this is serious
>>> > concern for lightweight or server-based wallets like Electrum or
>>> > myTREZOR. myTREZOR currently uses gap factor 10, which is (from my
>>> > experience so far) quite sane for most of users.
>>>
>>>
>>> Yes, I was planning to increase the number of available unused addresses
>>> to 10 or 20 in the bip32 version of Electrum.
>>>
>>> Related to this, here is another idea I would like to submit:
>>>
>>> Instead of using a "gap limit" (maximal number of consecutive unused
>>> addresses), I think we should get rid of the topology, and simply count
>>> the number of unused addresses since the beginning of the sequence.
>>> Indeed, the topology of the sequence of addresses is of no interest to
>>> the user. Users often misinterpret "gap limit" as the "number of unused
>>> addresses available", so I think we should just give them what they want
>>> :) This is easier to understand, and it makes things more predictable,
>>> because the wallet will always display the same number of unused
>>> addresses (except when it is waiting for confirmations).
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>
>>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 4680 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3685 bytes --]

Obviously, SHA256 can't magically generate more entropy out of nothing, it
just stretches whatever is put in. If your seed was only 32 bits then
hashing wouldn't save you: every possible private key could easily be
calculated in advance.


On Thu, Mar 27, 2014 at 2:12 PM, Thomas Kerin <thomas.kerin@gmail.com>wrote:

> Isn't the length of the seed arbitrary anyway? Once decoded using whatever
> mnemonic implementation (electrums, or BIP0039) the bytestream is
> immediately passed to HMAC-SHA256 to generate the master key. No matter
> what your initial entropy is, it would be hashed anyway.
>
>
> On Thu, Mar 27, 2014 at 12:49 PM, Mike Hearn <mike@plan99.net> wrote:
>
>> Ah, BIP32 allows for a range of entropy sizes and it so happens that they
>> picked 256 bits instead of 128 bits.
>>
>> I'd have thought that there is a right answer for this. 2^128 should not
>> be brute forceable, and longer sizes have a cost in terms of making the
>> seeds harder to write down on paper. So should this be a degree of freedom?
>>
>>
>> On Thu, Mar 27, 2014 at 1:28 PM, Mike Hearn <mike@plan99.net> wrote:
>>
>>> By the way, I just noticed that greenaddress.it is creating seeds that
>>> have 24 words instead of 12. Does anyone know what's up with that? They
>>> claim to be using BIP32 wallets so I wanted to see if they were using the
>>> default structure and if so, whether bitcoinj was compatible with it
>>> (before I switch to the one discussed here). But it seems we fall at the
>>> first hurdle ...
>>>
>>>
>>> On Thu, Mar 27, 2014 at 1:06 PM, Thomas Voegtlin <thomasv1@gmx.de>wrote:
>>>
>>>>
>>>>
>>>> Le 27/03/2014 12:30, Marek Palatinus a écrit :
>>>> > Ah, I forget to two things, which should be into the BIP as well:
>>>> >
>>>> > a) Gap factor for addresses; as Thomas mentioned, although some
>>>> software
>>>> > can watch almost unlimited amount of unused addresses, this is serious
>>>> > concern for lightweight or server-based wallets like Electrum or
>>>> > myTREZOR. myTREZOR currently uses gap factor 10, which is (from my
>>>> > experience so far) quite sane for most of users.
>>>>
>>>>
>>>> Yes, I was planning to increase the number of available unused addresses
>>>> to 10 or 20 in the bip32 version of Electrum.
>>>>
>>>> Related to this, here is another idea I would like to submit:
>>>>
>>>> Instead of using a "gap limit" (maximal number of consecutive unused
>>>> addresses), I think we should get rid of the topology, and simply count
>>>> the number of unused addresses since the beginning of the sequence.
>>>> Indeed, the topology of the sequence of addresses is of no interest to
>>>> the user. Users often misinterpret "gap limit" as the "number of unused
>>>> addresses available", so I think we should just give them what they want
>>>> :) This is easier to understand, and it makes things more predictable,
>>>> because the wallet will always display the same number of unused
>>>> addresses (except when it is waiting for confirmations).
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> Bitcoin-development mailing list
>>>> Bitcoin-development@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 5364 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 27/03/2014 13:49, Mike Hearn a écrit :
> Ah, BIP32 allows for a range of entropy sizes and it so happens that
> they picked 256 bits instead of 128 bits.
>
> I'd have thought that there is a right answer for this. 2^128 should not
> be brute forceable, and longer sizes have a cost in terms of making the
> seeds harder to write down on paper. So should this be a degree of freedom?
>


Here is what I understand:

2^128 iterations is not brute forcable today, and will not be for the 
foreseeable future.

An EC pubkey of length n can be forced in approximately 2^(n/2) 
iterations (see http://ecc-challenge.info/) Thus, Bitcoin pubkeys, which 
are 256 bits, would require 2^128 iterations. This is why unused 
addresses (160 bits hash) are better protected than already used ones.

However, people tend to believe that a public key of size n requires 2^n 
iterations. This belief might have been spread by this popular image:
https://bitcointalk.org/index.php?topic=508880.msg5616146#msg5616146

Re: [Bitcoin-development] New BIP32 structure

[Jim]

Good to hear the bip32 wallet structure is _so_ close to being standardised.
For MultiBit HD, we have put in support for 12/18/24 words but have the UI 'suggest' to use 12.
You can see this on the wallet creation wizard Gary recently blogged about:
https://multibit.org/blog/2014/03/26/multibit-hd-welcome-wizard.html

There's a little combo for the seed length, with 12 as the default.


@Thomas. You mention gaps. We are creating new addresses on the UI in a panel marked 'Request' where the user also types in a QR code label and a note to themselves. This gets stored away as a first class 'PaymentRequest'. The UI 'suggests' that each address is used once. There will be some gaps (where the payment request is never paid) but we aren't bulk creating addresses. I am hoping this shouldn't cause Electrum a problem.

We are also storing a timestamp (the number of days since the genesis block) to help wallet restore but that is SPV specific.


On Thu, Mar 27, 2014, at 01:49 PM, Thomas Voegtlin wrote:
> 
> 
> Le 27/03/2014 13:49, Mike Hearn a écrit :
IP32 allows for a range of entropy sizes and it so happens that
> > they picked 256 bits instead of 128 bits.
> >
> > I'd have thought that there is a right answer for this. 2^128 should not
> > be brute forceable, and longer sizes have a cost in terms of making the
> > seeds harder to write down on paper. So should this be a degree of freedom?
> >
> 
> 
> Here is what I understand:
> 
> 2^128 iterations is not brute forcable today, and will not be for the 
> foreseeable future.
> 
> An EC pubkey of length n can be forced in approximately 2^(n/2) 
> iterations (see http://ecc-challenge.info/) Thus, Bitcoin pubkeys, which 
> are 256 bits, would require 2^128 iterations. This is why unused 
> addresses (160 bits hash) are better protected than already used ones.
> 
> However, people tend to believe that a public key of size n requires 2^n 
> iterations. This belief might have been spread by this popular image:
> https://bitcointalk.org/index.php?topic=508880.msg5616146#msg5616146
> 
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


-- 
http://bitcoin-solutions.co.uk

Re: [Bitcoin-development] New BIP32 structure

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3465 bytes --]

For SPV wallets it's more complicated. There must always be a large
lookahead window for latency reasons. We can't query the entire database
because we don't know how far ahead the user is. So we have to assume there
might be a lot of transaction traffic and create a large window, to reduce
the chances that we run out whilst syncing and have to abort/restart the
sync after resetting the Bloom filter.

If you have a full db index then you can calculate some addresses, query,
if they all get hits, calculate some more, requery, etc. It's a bit simpler.


On Thu, Mar 27, 2014 at 2:58 PM, Jim <jim618@fastmail.co.uk> wrote:

> Good to hear the bip32 wallet structure is _so_ close to being
> standardised.
> For MultiBit HD, we have put in support for 12/18/24 words but have the UI
> 'suggest' to use 12.
> You can see this on the wallet creation wizard Gary recently blogged about:
> https://multibit.org/blog/2014/03/26/multibit-hd-welcome-wizard.html
>
> There's a little combo for the seed length, with 12 as the default.
>
>
> @Thomas. You mention gaps. We are creating new addresses on the UI in a
> panel marked 'Request' where the user also types in a QR code label and a
> note to themselves. This gets stored away as a first class
> 'PaymentRequest'. The UI 'suggests' that each address is used once. There
> will be some gaps (where the payment request is never paid) but we aren't
> bulk creating addresses. I am hoping this shouldn't cause Electrum a
> problem.
>
> We are also storing a timestamp (the number of days since the genesis
> block) to help wallet restore but that is SPV specific.
>
>
> On Thu, Mar 27, 2014, at 01:49 PM, Thomas Voegtlin wrote:
> >
> >
> > Le 27/03/2014 13:49, Mike Hearn a écrit :
> IP32 allows for a range of entropy sizes and it so happens that
> > > they picked 256 bits instead of 128 bits.
> > >
> > > I'd have thought that there is a right answer for this. 2^128 should
> not
> > > be brute forceable, and longer sizes have a cost in terms of making the
> > > seeds harder to write down on paper. So should this be a degree of
> freedom?
> > >
> >
> >
> > Here is what I understand:
> >
> > 2^128 iterations is not brute forcable today, and will not be for the
> > foreseeable future.
> >
> > An EC pubkey of length n can be forced in approximately 2^(n/2)
> > iterations (see http://ecc-challenge.info/) Thus, Bitcoin pubkeys, which
> > are 256 bits, would require 2^128 iterations. This is why unused
> > addresses (160 bits hash) are better protected than already used ones.
> >
> > However, people tend to believe that a public key of size n requires 2^n
> > iterations. This belief might have been spread by this popular image:
> > https://bitcointalk.org/index.php?topic=508880.msg5616146#msg5616146
> >
> >
> >
> ------------------------------------------------------------------------------
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
> --
> http://bitcoin-solutions.co.uk
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4851 bytes --]

Re: [Bitcoin-development] New BIP32 structure

[Troy Benjegerdes]

On Thu, Mar 27, 2014 at 02:49:32PM +0100, Thomas Voegtlin wrote:
> 
> 
> Le 27/03/2014 13:49, Mike Hearn a écrit :
> > Ah, BIP32 allows for a range of entropy sizes and it so happens that
> > they picked 256 bits instead of 128 bits.
> >
> > I'd have thought that there is a right answer for this. 2^128 should not
> > be brute forceable, and longer sizes have a cost in terms of making the
> > seeds harder to write down on paper. So should this be a degree of freedom?
> >
> 
> 
> Here is what I understand:
> 
> 2^128 iterations is not brute forcable today, and will not be for the 
> foreseeable future.

I foresee 2^128 being brute forceable in 20-25 years. See below.
 
> An EC pubkey of length n can be forced in approximately 2^(n/2) 
> iterations (see http://ecc-challenge.info/) Thus, Bitcoin pubkeys, which 
> are 256 bits, would require 2^128 iterations. This is why unused 
> addresses (160 bits hash) are better protected than already used ones.
> 
> However, people tend to believe that a public key of size n requires 2^n 
> iterations. This belief might have been spread by this popular image:
> https://bitcointalk.org/index.php?topic=508880.msg5616146#msg5616146

So I assume this image is using the Landauer principle for minimum 
energy ( http://en.wikipedia.org/wiki/Landauer%27s_principle ), however
it is unknown (to me at least) if a reversible computing ecdsa forcing
algorithm could be implemented. (this may or may not be a quantum
computing device)

Let's suppose for a moment you could, and get a million times better 
than the Landauer pinciple limit of 2.85 zJ per bit, so we have total
energy to cycle through 128 bits of address space of:

units "2**128 * 2.85zJ / 1e6" "megawatt*hours"
        * 269.39021

An attacker with a sub-Landauer limit/1e6  cracker would need a lot of
silicon area, and a couple of hours energy from a large wind farm, and
could siphon that energy out in a rural area without anyone noticing 
anything other than a few shipping containers and that the wind turbines
are running more on windy days.

If we go back to just Landauer limit, and assume a 10MW system that 
runs 24x7 (much like the NCSA Blue Waters Cray machine), we need:
(please check my math, or point out any stupid assumptions I've made)

units "2**128 * 2.85zJ / 10 megawatts" " years"
        * 3073.1914

Or 3000 years. Well that still sounds pretty safe. How about 250MW?
units "2**128 * 2.85zJ / 250 megawatts" " years"
        * 122.92766

Now I'm starting to get worried, because when I started computing, it
was on an 8-bit CPU that was measured in thousand operations-per-second.
In 1996 the largest supercomputer in the world was ASCII Red, with an
amazing 1 trillion floating-point operations per second. This morning
I have a 1-2 Teraflop water-cooled graphics processor sitting next to
me warming the room.

I expect in 5-10 years we'll have silicon with 256 bit registers that
may be able to do thousands or millions of ECDSA calculations per
second per computation unit.

So if you stop hearing from me here, it's because I found a better 
mailing list, or a got a contract to develop and ECDSA cracker, in 
which case you probably won't hear from me again until I have a talk
at DEFCON showing it off.


-- Troy Benjegerdes

Re: [Bitcoin-development] New BIP32 structure

[Matias Alejo Garcia]

On Thu, Mar 27, 2014 at 9:28 AM, Mike Hearn <mike@plan99.net> wrote:
> By the way, I just noticed that greenaddress.it is creating seeds that have 24
> words instead of 12. Does anyone know what's up with that? They claim to be
> using BIP32 wallets so I wanted to see if they were using the default structure
> and if so, whether bitcoinj was compatible with it (before I switch to the one
> discussed here). But it seems we fall at the first hurdle ...
>

They replied this on reddit: "Good question! We use 256-bit BIP39
seeds for our 512-bit extended HD wallet keypairs, as advised by
BIP32. We are aware that 128 is considered ok in Electrum but we
wanted to follow BIP32 spec, in particular because we are working with
hardware wallets that have been design with BIP32 in mind and wanted
to avoid any incompatibility issue."

http://www.reddit.com/r/Bitcoin/comments/20puhg/while_blockchaininfo_is_down_what_about_testing/cg5mr54


BTW, thanks for starting this thread. I am with the bitcore team,
working on a new multisig only wallet (Cosign).

The schema originally proposed be Mike (/m/cointype/reserved'/account'/change/n
) seems flexible enough for our needs. I wonder, tough, if we can
agree also on a format to exchange the tips (n) of each branch on the
tree when the user export/backup the wallet, as  Allen suggested.


On Wed, Mar 26, 2014 at 8:37 PM, Andreas Schildbach
<andreas@schildbach.de> wrote:
> For me, the most important thing is either we're 100% interoperable or
> 0%. There should not be anything inbetween, as users will delete seeds
> without knowing there is still money in them on another implementation.

Is the ideal schema that the user should be able to use two different
wallets at the same time? Does a wallet need to assume that derivate
keys could had been used already? That seems complicated. Maybe we
could add a  /(hash of the wallet name)/ on the BIP32 path.


--
matias



-- 
Matías Alejo Garcia
Skype/Twitter: @ematiu
Roads? Where we're going, we don't need roads!

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/27/2014 01:06 PM, Thomas Voegtlin wrote:
> Yes, I was planning to increase the number of available unused addresses 
> to 10 or 20 in the bip32 version of Electrum.

Also, we'd need to specify a "gap limit" for accounts as well. In TREZOR
we currently use 0, which means that the scan will stop as soon as we
hit first account with no transaction history (meaning that its first
X=10 addresses are unused).

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

Le 27/03/2014 14:44, Pavol Rusnak a écrit :
> On 03/27/2014 01:06 PM, Thomas Voegtlin wrote:
>> Yes, I was planning to increase the number of available unused addresses
>> to 10 or 20 in the bip32 version of Electrum.
>
> Also, we'd need to specify a "gap limit" for accounts as well. In TREZOR
> we currently use 0, which means that the scan will stop as soon as we
> hit first account with no transaction history (meaning that its first
> X=10 addresses are unused).
>

I agree with that. I was planning to do the same (no gap)

Note: Maybe we could just look at the first address of each account, 
instead of the first 10 ?

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/27/2014 02:53 PM, Thomas Voegtlin wrote:
> Note: Maybe we could just look at the first address of each account, 
> instead of the first 10 ?

This is a possible optimization, but it adds unnecessary logic. Also it
does not decrease the number of required requests between a client and a
server (e.g. when backend sends responses in "bulks" of 10 addresses or
more).


-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] New BIP32 structure

[Thomas Voegtlin]

>
>     Related to this, here is another idea I would like to submit:
>
>     Instead of using a "gap limit" (maximal number of consecutive unused
>     addresses), I think we should get rid of the topology, and simply count
>     the number of unused addresses since the beginning of the sequence.
>     Indeed, the topology of the sequence of addresses is of no interest to
>     the user. Users often misinterpret "gap limit" as the "number of unused
>     addresses available", so I think we should just give them what they want
>     :) This is easier to understand, and it makes things more predictable,
>     because the wallet will always display the same number of unused
>     addresses (except when it is waiting for confirmations).
>
>
> I'm not convinced that it is either simpler to implement nor simpler to
> understand. I considering every "account" as completely independent
> structure (except that it has been derived from master seed). Because of
> this, I don't see a reason why number of generated addresses in one
> account should affect available addresses of other account.


Sorry if I was not clear, but no, I was not proposing that... I propose 
to count the number of unused addresses in each account, independently.

Here is an explanation with pictures:

Currently, Electrum does this:

x = used address
o = unused address

x x x o o x x x x o o o o o
       |           └───────┘ 5 unused addresses at the end (gap limit =5)
      existing gap, size 2

Case 1:
Suppose the user decides to use the 4th address starting from the end:
x x x o o x x x x x o o o o

After 3 confirmations, Electrum will create one extra address:
x x x o o x x x x x o o o o o
                     └───────┘ 5 final address (one new)

Case 2:
The user, who understands nothing about topology, decides to use the 
final address:
x x x o o x x x x o o o o x

After 3 confirmations, Electrum will create 5 new addresses:
x x x o o x x x x o o o o x o o o o o
                     |        └───────┘ 5 new addresses
                  new gap (size 4)


You see that this leads to a number of newly created addresses that 
depends on the address picked by the user. This is the behaviour that I 
want to fix. I think it is bad for two reasons: First, because it is 
unpredictable (I had a user who set his gap limit to 100, and then was 
systematically using the last address of the list, ending up with a lot 
of wasted addresses). Second, because the worst case scenario (case 2) 
creates large gaps, which are less robust to blockchain reorgs than in 
case 1. A reorg in case 1, where the last tx is cancelled will create no 
gap. A reorg in case 2 can result in a gap that is larger than the limit 
(5), if another transaction uses the final addresses. (this is why 
Electrum has to wait for a few confirmations before adding addresses)


So, what I propose is to always have a constant number of unused 
addresses in each account, instead of something that depends on the 
topology. (this number can be 10 or 20, I don't mind, but I guess Mike 
might prefer 20...)

Re: [Bitcoin-development] New BIP32 structure

[Pavol Rusnak]

On 03/26/2014 09:49 PM, Mike Hearn wrote:
>    - cointype:  This is zero for Bitcoin. This is here to support two
>    things, one is supporting alt coins based off the same root seed. Right now
>    nobody seemed very bothered about alt coins but sometimes feature requests

There is one "altcoin" that is pretty important even today and it is
Testnet.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>