[Bitcoin-development] Making the H in HD keychains useful

[Eric Lombrozo <elombrozo@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3832 bytes --]

I've been trying to find ways to make HD keychain wallets (BIP0032) really usable from an application development perspective. I think we all know a number of solid use cases and possible applications for the D in HD, but nobody seems to have really found a way to make use of the H in a way that is actually manageable from a usability standpoint.

After pondering it a bit more, I think I've stumbled upon at least a couple issues that seem to give hints as to how we can change this.

Hierarchical organizations do not generally tend to be designed up front, cast in stone. In the real world, hierarchies tend to evolve organically, growing new branches as entities differentiate themselves to different purposes. Organizations grow over time. Sometimes branches merge, sometimes branches die. This means that for HD keychains to be truly useful, they too need to be sufficiently flexible to adapt to the needs of a growing and evolving organization. It needs to be simple to create and move branches around as the need for them arises without having to plan the structure a priori.

A significant problem I'm runnign into in trying to build applications around the BIP0032 standard is the lack of a clear separation between signing keys and hierarchical nodes. That's to say, a child of a node can either be used as a signing key or as a parent for new branches to the tree. From a usability standpoint, what this means is that one must be very careful in how one allocates keys from the very beginning - if one mixes signing keys with new branching nodes in the same generation, the whole thing becomes a horrendous mess. Moreover, it is impossible to generally distinguish these two fundamentally different types of objects (at least from a use model perspective) just from the extended key representation, something that is certain to create significant confusion as we try to design applications that can share these types of objects.

An organization might begin as a single individual who just wants to generate signing keys for him/herself. Later on, this individual might bring on another individual or two and create new branches for them. With the current HD keychain structure, unless this individual made sure to set aside these new branches from the start, the individual is now forced to mix the new branches in at the same level of the hierarchy as the signing keys. Instead, it should be possible to branch off any node without having to worry at all about whether or not that node has been used to generate signing keys at all.

A possible workaround to this issue is to always allocate a specific child for hierarchical derivation and the rest of the children for signing keys. Then to create subbranches, the specific child would be used as the new parent, effectively alternating generations between signing keys and organizational nodes. However, this solution seems pretty ugly.

A better solution, IMO, is to only use BIP0032 for organizational hierarchy and have a different mechanism for generating a sequence of signing keys from a given node. This different mechanism could be used standalone by those not needing the full set of hierarchical features. For those who do want to use the hierarchical features, it could be seeded by the keys in the BIP0032 hierarchy. These individual signing keys would NEVER be represented in the same format as the organizational hierarchy nodes, thus ensuring applications can share these structures without risk of confusion.

Until we make this clear distinction between organizational hierarchy (which parallels real-world organizations) and signing keys (which are merely cryptographic primitives, preferably never even shown directly to most endusers), I think we'll fail to find good ways to make the H in HD keychains useful.

-Eric Lombrozo

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 801 bytes --]