[Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Andreas Schildbach]

Does anyone use or plan to use the "wallet structure" part of the BIP32
document?

I suggest removing it from the document and maybe instead point at
BIP43. That new specification proposal just defines a "purpose" and
leaves everything else to the inventor of that purpose. The idea it that
over time a standard will evolve naturally rather than top-down.

https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki

I'd volunteer to submit a pull request if I can see some level of
consent here.

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Jim]

Oh dear.

For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets.

In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin.

Can we not agree on a lowest common denominator that we agree to support ?
An "HD Basic" if you like. 
For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable.

Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it.

I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice.






On Fri, Apr 25, 2014, at 11:23 AM, Andreas Schildbach wrote:
> Does anyone use or plan to use the "wallet structure" part of the BIP32
> document?
> 
> I suggest removing it from the document and maybe instead point at
> BIP43. That new specification proposal just defines a "purpose" and
> leaves everything else to the inventor of that purpose. The idea it that
> over time a standard will evolve naturally rather than top-down.
> 
> https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki
> 
> I'd volunteer to submit a pull request if I can see some level of
> consent here.
> 
> 
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


-- 
http://bitcoin-solutions.co.uk

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Gregory Maxwell]

On Fri, Apr 25, 2014 at 7:53 AM, Jim <jim618@fastmail.co.uk> wrote:
> Oh dear.
>
> For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets.
>
> In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin.
>
> Can we not agree on a lowest common denominator that we agree to support ?
> An "HD Basic" if you like.
> For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable.
>
> Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it.
>
> I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice.

I don't believe that wallet interoperability at this level is possible
in general except as an explicit compatibility feature. I also don't
believe that it is a huge loss that it is so.

The structure of the derivation defines and constrains functionality.
You cannot be structure compatible unless you have the same features
and behavior with respect to key management.  To that extent that
wallets have the same features, I agree its better if they are
compatible— but unless they are dead software they likely won't keep
the same features for long.

Even if their key management were compatible there are many other
things that go into making a wallet portable between systems; the
handling of private keys is just one part:  a complete wallet will
have other (again, functionality specific) metadata.

I agree that it would be it would be possible to support a
compatibility mode where a wallet has just a subset of features which
works when loaded into different systems, but I'm somewhat doubtful
that it would be widely used. The decision to use that mode comes at
the wrong time— when you start, not when you need the features you
chose to disable or when you want to switch programs. But the obvious
thing to do there is to just specify that a linear chain with no
further branching is that mode: then that will be the same mode you
use when someone gives you a master public key and asks you to use it
for reoccurring changes— so at least the software will get used.

Compatibility for something like a recovery tool is another matter,
and BIP32 probably defines enough there that with a bit of extra data
about how the real wallet worked that recovery can be successful.

Calling it "vendor lock in" sounds overblown to me.  If someone wants
to change wallets they can transfer the funds— manual handling of
private keys is seldom advisable, and as is they're going to lose
their metadata in any case.  No one expects to switch banks and to
keep their account records at the new bank. And while less than
perfect, the price of heavily constraining functionality in order to
get another result is just too high.

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

I generally agree, but I wonder how popular cloning wallets between devices
will be in future. Right now if someone wants to have a wallet shared
between Hive, blockchain.info and Bitcoin Wallet for Android, we just tell
them they're out of luck and they need to pick one, or split their funds up
manually.

But probably a lot of people would like to use different UI's to access the
same wallets. Sharing key trees is a part of that, though full blown wallet
metadata sync would also be needed.

So I guess we're going to end up with some kind of fairly complex
compatibility matrix. But I agree it may be unavoidable.

[-- Attachment #2: Type: text/html, Size: 743 bytes --]

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Aaron Voisine]

On github I commented on the BIP43 pull request about adding a
"purpose" of 0' which would correspond to the BIP32 recommended tree
structure for a single account wallet. (m/0'/chain) This will allow
for backwards compatibility and interoperability at least for existing
single account BIP32 implementations like my own. The only issue is
that it would involve a new BIP, and it wouldn't follow the BIP43
recommendation of using the BIP number as the "purpose number", unless
I can get BIP0.  :)

Aaron

There's no trick to being a humorist when you have the whole
government working for you -- Will Rodgers


On Fri, Apr 25, 2014 at 8:49 AM, Mike Hearn <mike@plan99.net> wrote:
> I generally agree, but I wonder how popular cloning wallets between devices
> will be in future. Right now if someone wants to have a wallet shared
> between Hive, blockchain.info and Bitcoin Wallet for Android, we just tell
> them they're out of luck and they need to pick one, or split their funds up
> manually.
>
> But probably a lot of people would like to use different UI's to access the
> same wallets. Sharing key trees is a part of that, though full blown wallet
> metadata sync would also be needed.
>
> So I guess we're going to end up with some kind of fairly complex
> compatibility matrix. But I agree it may be unavoidable.
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Thomas Voegtlin]

I totally agree with gmaxwell here. The cost of interoperability is too
high. It would force us to freeze all features, and to require a broad
consensus everytime we want to add something new.

In addition, some partial level of compatibility would probably lead to
users not able to recover all their funds when they enter their seed in
another wallet. That is not acceptable, and should be avoided.




Le 25/04/2014 17:46, Gregory Maxwell a écrit :
> 
> I don't believe that wallet interoperability at this level is possible
> in general except as an explicit compatibility feature. I also don't
> believe that it is a huge loss that it is so.
> 
> The structure of the derivation defines and constrains functionality.
> You cannot be structure compatible unless you have the same features
> and behavior with respect to key management.  To that extent that
> wallets have the same features, I agree its better if they are
> compatible— but unless they are dead software they likely won't keep
> the same features for long.
> 
> Even if their key management were compatible there are many other
> things that go into making a wallet portable between systems; the
> handling of private keys is just one part:  a complete wallet will
> have other (again, functionality specific) metadata.
> 
> I agree that it would be it would be possible to support a
> compatibility mode where a wallet has just a subset of features which
> works when loaded into different systems, but I'm somewhat doubtful
> that it would be widely used. The decision to use that mode comes at
> the wrong time— when you start, not when you need the features you
> chose to disable or when you want to switch programs. But the obvious
> thing to do there is to just specify that a linear chain with no
> further branching is that mode: then that will be the same mode you
> use when someone gives you a master public key and asks you to use it
> for reoccurring changes— so at least the software will get used.
> 
> Compatibility for something like a recovery tool is another matter,
> and BIP32 probably defines enough there that with a bit of extra data
> about how the real wallet worked that recovery can be successful.
> 
> Calling it "vendor lock in" sounds overblown to me.  If someone wants
> to change wallets they can transfer the funds— manual handling of
> private keys is seldom advisable, and as is they're going to lose
> their metadata in any case.  No one expects to switch banks and to
> keep their account records at the new bank. And while less than
> perfect, the price of heavily constraining functionality in order to
> get another result is just too high.
> 
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 4408 bytes --]

Maybe the solution is to have a defined way to import an unknown wallet?

This means that the gap space and a search ordering needs to be defined.

Given a blockchain and a root seed, it should be possible to find all the
addresses for that root seed.

The hierarchy that the wallet actually uses could be anything.


On Sat, Apr 26, 2014 at 11:36 AM, Thomas Voegtlin <thomasv1@gmx.de> wrote:

> I totally agree with gmaxwell here. The cost of interoperability is too
> high. It would force us to freeze all features, and to require a broad
> consensus everytime we want to add something new.
>
> In addition, some partial level of compatibility would probably lead to
> users not able to recover all their funds when they enter their seed in
> another wallet. That is not acceptable, and should be avoided.
>
>
>
>
> Le 25/04/2014 17:46, Gregory Maxwell a écrit :
> >
> > I don't believe that wallet interoperability at this level is possible
> > in general except as an explicit compatibility feature. I also don't
> > believe that it is a huge loss that it is so.
> >
> > The structure of the derivation defines and constrains functionality.
> > You cannot be structure compatible unless you have the same features
> > and behavior with respect to key management.  To that extent that
> > wallets have the same features, I agree its better if they are
> > compatible— but unless they are dead software they likely won't keep
> > the same features for long.
> >
> > Even if their key management were compatible there are many other
> > things that go into making a wallet portable between systems; the
> > handling of private keys is just one part:  a complete wallet will
> > have other (again, functionality specific) metadata.
> >
> > I agree that it would be it would be possible to support a
> > compatibility mode where a wallet has just a subset of features which
> > works when loaded into different systems, but I'm somewhat doubtful
> > that it would be widely used. The decision to use that mode comes at
> > the wrong time— when you start, not when you need the features you
> > chose to disable or when you want to switch programs. But the obvious
> > thing to do there is to just specify that a linear chain with no
> > further branching is that mode: then that will be the same mode you
> > use when someone gives you a master public key and asks you to use it
> > for reoccurring changes— so at least the software will get used.
> >
> > Compatibility for something like a recovery tool is another matter,
> > and BIP32 probably defines enough there that with a bit of extra data
> > about how the real wallet worked that recovery can be successful.
> >
> > Calling it "vendor lock in" sounds overblown to me.  If someone wants
> > to change wallets they can transfer the funds— manual handling of
> > private keys is seldom advisable, and as is they're going to lose
> > their metadata in any case.  No one expects to switch banks and to
> > keep their account records at the new bank. And while less than
> > perfect, the price of heavily constraining functionality in order to
> > get another result is just too high.
> >
> >
> ------------------------------------------------------------------------------
> > Start Your Social Network Today - Download eXo Platform
> > Build your Enterprise Intranet with eXo Platform Software
> > Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> > Get Started Now And Turn Your Intranet Into A Collaboration Platform
> > http://p.sf.net/sfu/ExoPlatform
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> >
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 5557 bytes --]

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 738 bytes --]

Yes, it is expensive but possible to discover any funds associated with a seed, provided there are set limits to:

1. gap of address use (e.g. 20)
2. depth of hierarchy (e.g. 6)
3. gap in use of parallel branches (e.g. 0) 

I would pick the limits in brackets above. 

Regards,

Tamas Blummer
http://bitsofproof.com

On 26.04.2014, at 12:48, Tier Nolan <tier.nolan@gmail.com> wrote:

> Maybe the solution is to have a defined way to import an unknown wallet?
> 
> This means that the gap space and a search ordering needs to be defined.
> 
> Given a blockchain and a root seed, it should be possible to find all the addresses for that root seed.
> 
> The hierarchy that the wallet actually uses could be anything.


[-- Attachment #1.2: Type: text/html, Size: 4196 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Tamas Blummer]

[-- Attachment #1.1: Type: text/plain, Size: 1015 bytes --]

Actually gap in parallel branches already fails with BIP64 as it starts with m/64'/…. without having m/63'

Regards,

Tamas Blummer
http://bitsofproof.com

On 26.04.2014, at 12:59, Tamas Blummer <tamas@bitsofproof.com> wrote:

> Yes, it is expensive but possible to discover any funds associated with a seed, provided there are set limits to:
> 
> 1. gap of address use (e.g. 20)
> 2. depth of hierarchy (e.g. 6)
> 3. gap in use of parallel branches (e.g. 0) 
> 
> I would pick the limits in brackets above. 
> 
> Regards,
> 
> Tamas Blummer
> http://bitsofproof.com
> 
> On 26.04.2014, at 12:48, Tier Nolan <tier.nolan@gmail.com> wrote:
> 
>> Maybe the solution is to have a defined way to import an unknown wallet?
>> 
>> This means that the gap space and a search ordering needs to be defined.
>> 
>> Given a blockchain and a root seed, it should be possible to find all the addresses for that root seed.
>> 
>> The hierarchy that the wallet actually uses could be anything.
> 


[-- Attachment #1.2: Type: text/html, Size: 7552 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Pavol Rusnak]

On 04/26/2014 12:48 PM, Tier Nolan wrote:
> Maybe the solution is to have a defined way to import an unknown wallet?

That is nonsense. There is no way how to import the wallet if you don't
know its structure.

> Given a blockchain and a root seed, it should be possible to find all the
> addresses for that root seed.

Unless the keyspace is almost infinite because:

> The hierarchy that the wallet actually uses could be anything.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Pieter Wuille]

On Sat, Apr 26, 2014 at 2:24 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> On 04/26/2014 12:48 PM, Tier Nolan wrote:
>> Maybe the solution is to have a defined way to import an unknown wallet?
>
> That is nonsense. There is no way how to import the wallet if you don't
> know its structure.

I agree. Especially when multiple chains are combined (multisig) for
P2SH usage, defining things like a gap limit becomes impossible
without knowing some metadata.

However, perhaps it is possible to define something like "BIP44
import-compatible", meaning that the application doesn't actually
support all of BIP44 features, but does guarantee not losing any funds
when imported? Similar things could be done for other purpose types.

-- 
Pieter

Re: [Bitcoin-development] BIP32 "wallet structure" in use? Remove it?

[Jeff Garzik]

It is very young in bitcoin's life.  We don't know what features will
work out best, or need to be radically changed after initial
deployment in the field.

Loose coordination is good.  Good ideas will spread on their own.
Users will demand compatibility with certain features, and fail to
care incompatibilities in other features.

Tight interoperability at this stage is too confining.




On Fri, Apr 25, 2014 at 11:46 AM, Gregory Maxwell <gmaxwell@gmail.com> wrote:
> On Fri, Apr 25, 2014 at 7:53 AM, Jim <jim618@fastmail.co.uk> wrote:
>> Oh dear.
>>
>> For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets.
>>
>> In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin.
>>
>> Can we not agree on a lowest common denominator that we agree to support ?
>> An "HD Basic" if you like.
>> For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable.
>>
>> Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it.
>>
>> I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice.
>
> I don't believe that wallet interoperability at this level is possible
> in general except as an explicit compatibility feature. I also don't
> believe that it is a huge loss that it is so.
>
> The structure of the derivation defines and constrains functionality.
> You cannot be structure compatible unless you have the same features
> and behavior with respect to key management.  To that extent that
> wallets have the same features, I agree its better if they are
> compatible-- but unless they are dead software they likely won't keep
> the same features for long.
>
> Even if their key management were compatible there are many other
> things that go into making a wallet portable between systems; the
> handling of private keys is just one part:  a complete wallet will
> have other (again, functionality specific) metadata.
>
> I agree that it would be it would be possible to support a
> compatibility mode where a wallet has just a subset of features which
> works when loaded into different systems, but I'm somewhat doubtful
> that it would be widely used. The decision to use that mode comes at
> the wrong time-- when you start, not when you need the features you
> chose to disable or when you want to switch programs. But the obvious
> thing to do there is to just specify that a linear chain with no
> further branching is that mode: then that will be the same mode you
> use when someone gives you a master public key and asks you to use it
> for reoccurring changes-- so at least the software will get used.
>
> Compatibility for something like a recovery tool is another matter,
> and BIP32 probably defines enough there that with a bit of extra data
> about how the real wallet worked that recovery can be successful.
>
> Calling it "vendor lock in" sounds overblown to me.  If someone wants
> to change wallets they can transfer the funds-- manual handling of
> private keys is seldom advisable, and as is they're going to lose
> their metadata in any case.  No one expects to switch banks and to
> keep their account records at the new bank. And while less than
> perfect, the price of heavily constraining functionality in order to
> get another result is just too high.
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development



-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/