Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[SomberNight]

See some replies inline. (quoted text from BIP draft)

> Date: Sun, 14 Mar 2021 01:51:15 +0000
> From: Robert Spigler <RobertSpigler@protonmail.ch>
> Subject: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

> There are many issues with the current standards. As background, BIP 44/49/84 specifies:
> `m / purpose' / coin_type' / account' / change / address_index`
> where the BIP43 `purpose'` path is separate for each script (P2PKH, P2WPKH-in-P2SH, and P2WPKH respectively).  However, these per-script derivations are made redundant with descriptors

> We should not be mixing keys and scripts in the same layer. The wallet should create extended private/public keys independent of the script or signature type

You say that keys and scripts should not be mixed in the same layer, and imply that this was solely done due to these standards predating output script descriptors. Even if this was the case, it is not the only reason for doing it. BIP44/49/84 mixing scripts and keys in the same layer makes recovery from seed/mnemonic much easier.
Note the significant overlap between the authors of BIP39 and BIP44. I am fairly certain BIP44 was designed with recovering from a BIP39 seed (and no additional information backed up) in mind. Note the "Account discovery" section of BIP44.
(Electrum seeds go even further, as such seeds contain a version number that encodes both the script type and the key derivation path to use.)

> We define the following 5 levels in the BIP32 path:
> `m / purpose' / coin_type' / account' / change / address_index`

> [Account]
> It is crucial that this level is increased for each new wallet joined or private/public keys created; for both privacy and cryptographic purposes.
> For example, in multisignature wallets, before sending a new key record to a coordinator, the wallet must increment the `account'` level.  Before creating it's own single signature wallet, the `account'` level must again be incremented.

Imagine a user who has a BIP39 (or similar) seed. Even today, recovering most non-singlesig scripts from that is obviously infeasible. However, all singlesig scripts at least can be discovered if the keys are using the suggested derivation paths.
By trying to create a standard that mixes discoverable and non-discoverable scripts in the same derivation scheme and incrementing a single index, you are turning all scripts into being non-discoverable.
Note that even if a user only used singlesig scripts and followed this proposal, during recovery from seed the wallet would have to check all script types for all account indices (which is only ever going to get more expensive as new script types come).
The workaround and I imagine your suggested solution is clearly to backup both seed words and output script descriptors; and to keep appending new output script descriptors to existing backups when the account index is incremented. While much less user-friendly than backing up just a seed, it is more generic and extendable.

My point is simply that your proposal is making a tradeoff here. The tradeoff itself seems easy to miss on first read of the text, so I just wanted to explicitly point it out for the record.

ghost43 / SomberNight

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Robert Spigler]

No, wallets don't and shouldn't have to check all script types on recovery.  Descriptor Wallets solve all of this.

To back up a multisignature wallet, each cosigner stores their xprv (how you do this; BIP39, WIF, etc, is out of scope). and the wallet descriptor.  This is done, for example, in Yeti. To recover, simply combine `M` private keys, and check the script designated in 1 of the descriptor copies.

For single signature wallets, it is the same, except only one signature is needed.  Store xprv and descriptor.

It is not fair nor accurate to say that currently, in order to recover, wallets need "just the seed words".  They also need all public keys, and derivation paths.

Descriptors (and this BIP), is a much cleaner way to handle wallet creation and backup, by separating the two layers (keys and scripts) and getting rid of redundant information.


Personal Fingerprint:  BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F



‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, March 14, 2021 11:13 AM, SomberNight <somber.night@protonmail.com> wrote:

> See some replies inline. (quoted text from BIP draft)
>
> > Date: Sun, 14 Mar 2021 01:51:15 +0000
> > From: Robert Spigler RobertSpigler@protonmail.ch
> > Subject: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.
>
> > There are many issues with the current standards. As background, BIP 44/49/84 specifies:
> > `m / purpose' / coin_type' / account' / change / address_index`
> > where the BIP43 `purpose'` path is separate for each script (P2PKH, P2WPKH-in-P2SH, and P2WPKH respectively). However, these per-script derivations are made redundant with descriptors
>
> > We should not be mixing keys and scripts in the same layer. The wallet should create extended private/public keys independent of the script or signature type
>
> You say that keys and scripts should not be mixed in the same layer, and imply that this was solely done due to these standards predating output script descriptors. Even if this was the case, it is not the only reason for doing it. BIP44/49/84 mixing scripts and keys in the same layer makes recovery from seed/mnemonic much easier.
> Note the significant overlap between the authors of BIP39 and BIP44. I am fairly certain BIP44 was designed with recovering from a BIP39 seed (and no additional information backed up) in mind. Note the "Account discovery" section of BIP44.
> (Electrum seeds go even further, as such seeds contain a version number that encodes both the script type and the key derivation path to use.)
>
> > We define the following 5 levels in the BIP32 path:
> > `m / purpose' / coin_type' / account' / change / address_index`
>
> > [Account]
> > It is crucial that this level is increased for each new wallet joined or private/public keys created; for both privacy and cryptographic purposes.
> > For example, in multisignature wallets, before sending a new key record to a coordinator, the wallet must increment the `account'` level. Before creating it's own single signature wallet, the `account'` level must again be incremented.
>
> Imagine a user who has a BIP39 (or similar) seed. Even today, recovering most non-singlesig scripts from that is obviously infeasible. However, all singlesig scripts at least can be discovered if the keys are using the suggested derivation paths.
> By trying to create a standard that mixes discoverable and non-discoverable scripts in the same derivation scheme and incrementing a single index, you are turning all scripts into being non-discoverable.
> Note that even if a user only used singlesig scripts and followed this proposal, during recovery from seed the wallet would have to check all script types for all account indices (which is only ever going to get more expensive as new script types come).
> The workaround and I imagine your suggested solution is clearly to backup both seed words and output script descriptors; and to keep appending new output script descriptors to existing backups when the account index is incremented. While much less user-friendly than backing up just a seed, it is more generic and extendable.
>
> My point is simply that your proposal is making a tradeoff here. The tradeoff itself seems easy to miss on first read of the text, so I just wanted to explicitly point it out for the record.
>
> ghost43 / SomberNight

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Craig Raw]

[-- Attachment #1: Type: text/plain, Size: 6259 bytes --]

The motivation for this BIP proposal states that encoding the script type
in the derivation is redundant because output descriptors are becoming
commonplace. However, I think it's important to note that this shifts more
burden of wallet backup onto the user, who now has an additional item (the
output descriptor) to backup correctly. This is particularly important when
we consider that performing this backup is often laborious, including
stamping onto metal etc.

There are two ways to go here:
1. Use derivation paths that do not encode the script type, and conduct a
global education effort to encourage users to backup output descriptors in
addition to their seed words, without which their funds may be lost.
2. Use a standard set of derivation paths that encode the script type,
ensuring that even if users do not back this up, they can check against a
handful of common derivations and recover the funds.

Option 1 is obviously cleaner and more precise from a technical POV, but I
feel it is less practical. The approach of using common words to encode the
seed was chosen because these words are short and familiar - an output
descriptor on the other hand is technical and longer form, and thus more
difficult to record correctly, particularly on damage-resistant media.

Craig

On Tue, Mar 16, 2021 at 2:19 PM Robert Spigler via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> No, wallets don't and shouldn't have to check all script types on
> recovery.  Descriptor Wallets solve all of this.
>
> To back up a multisignature wallet, each cosigner stores their xprv (how
> you do this; BIP39, WIF, etc, is out of scope). and the wallet descriptor.
> This is done, for example, in Yeti. To recover, simply combine `M` private
> keys, and check the script designated in 1 of the descriptor copies.
>
> For single signature wallets, it is the same, except only one signature is
> needed.  Store xprv and descriptor.
>
> It is not fair nor accurate to say that currently, in order to recover,
> wallets need "just the seed words".  They also need all public keys, and
> derivation paths.
>
> Descriptors (and this BIP), is a much cleaner way to handle wallet
> creation and backup, by separating the two layers (keys and scripts) and
> getting rid of redundant information.
>
>
> Personal Fingerprint:  BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F
>
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Sunday, March 14, 2021 11:13 AM, SomberNight <
> somber.night@protonmail.com> wrote:
>
> > See some replies inline. (quoted text from BIP draft)
> >
> > > Date: Sun, 14 Mar 2021 01:51:15 +0000
> > > From: Robert Spigler RobertSpigler@protonmail.ch
> > > Subject: [bitcoin-dev] Signature and Script Independent Hierarchy for
> Deterministic Wallets.
> >
> > > There are many issues with the current standards. As background, BIP
> 44/49/84 specifies:
> > > `m / purpose' / coin_type' / account' / change / address_index`
> > > where the BIP43 `purpose'` path is separate for each script (P2PKH,
> P2WPKH-in-P2SH, and P2WPKH respectively). However, these per-script
> derivations are made redundant with descriptors
> >
> > > We should not be mixing keys and scripts in the same layer. The wallet
> should create extended private/public keys independent of the script or
> signature type
> >
> > You say that keys and scripts should not be mixed in the same layer, and
> imply that this was solely done due to these standards predating output
> script descriptors. Even if this was the case, it is not the only reason
> for doing it. BIP44/49/84 mixing scripts and keys in the same layer makes
> recovery from seed/mnemonic much easier.
> > Note the significant overlap between the authors of BIP39 and BIP44. I
> am fairly certain BIP44 was designed with recovering from a BIP39 seed (and
> no additional information backed up) in mind. Note the "Account discovery"
> section of BIP44.
> > (Electrum seeds go even further, as such seeds contain a version number
> that encodes both the script type and the key derivation path to use.)
> >
> > > We define the following 5 levels in the BIP32 path:
> > > `m / purpose' / coin_type' / account' / change / address_index`
> >
> > > [Account]
> > > It is crucial that this level is increased for each new wallet joined
> or private/public keys created; for both privacy and cryptographic purposes.
> > > For example, in multisignature wallets, before sending a new key
> record to a coordinator, the wallet must increment the `account'` level.
> Before creating it's own single signature wallet, the `account'` level must
> again be incremented.
> >
> > Imagine a user who has a BIP39 (or similar) seed. Even today, recovering
> most non-singlesig scripts from that is obviously infeasible. However, all
> singlesig scripts at least can be discovered if the keys are using the
> suggested derivation paths.
> > By trying to create a standard that mixes discoverable and
> non-discoverable scripts in the same derivation scheme and incrementing a
> single index, you are turning all scripts into being non-discoverable.
> > Note that even if a user only used singlesig scripts and followed this
> proposal, during recovery from seed the wallet would have to check all
> script types for all account indices (which is only ever going to get more
> expensive as new script types come).
> > The workaround and I imagine your suggested solution is clearly to
> backup both seed words and output script descriptors; and to keep appending
> new output script descriptors to existing backups when the account index is
> incremented. While much less user-friendly than backing up just a seed, it
> is more generic and extendable.
> >
> > My point is simply that your proposal is making a tradeoff here. The
> tradeoff itself seems easy to miss on first read of the text, so I just
> wanted to explicitly point it out for the record.
> >
> > ghost43 / SomberNight
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 7147 bytes --]

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Jochen Hoenicke]

[-- Attachment #1: Type: text/plain, Size: 6093 bytes --]

Hello Robert,

you wrote:
> It is not fair nor accurate to say that currently, in order to recover,
wallets need "just the seed words".  They also need all public keys, and
derivation paths.

No a BIP39/BIP44 wallet can recover all BIP44 accounts using just the seed
words.  Read the section account recovery closely to see how it's done.  No
public keys or derivation paths needed, as the derivation path can just be
enumerated and for single signature wallets you don't need external public
keys. Similarly a BIP39/BIP44/BIP49/BIP84 can recover all these accounts
using just the seed words and for example the Trezor Suite does this.  Most
hardware wallets currently rely on this feature, as they don't store any
additional information like output descriptors (which would be possible,
but would make regular backups necessary).  Of course, autodiscover doesn't
work for multisig wallets, unless you would store some output descriptor in
an OP_RETURN on the blockchain.

It's okay to make a new standard that requires regular backups, but you
should at least keep in mind that this complicates some use cases.

  Jochen


On Tue, 16 Mar 2021 at 13:19, Robert Spigler via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> No, wallets don't and shouldn't have to check all script types on
> recovery.  Descriptor Wallets solve all of this.
>
> To back up a multisignature wallet, each cosigner stores their xprv (how
> you do this; BIP39, WIF, etc, is out of scope). and the wallet descriptor.
> This is done, for example, in Yeti. To recover, simply combine `M` private
> keys, and check the script designated in 1 of the descriptor copies.
>
> For single signature wallets, it is the same, except only one signature is
> needed.  Store xprv and descriptor.
>
> It is not fair nor accurate to say that currently, in order to recover,
> wallets need "just the seed words".  They also need all public keys, and
> derivation paths.
>
> Descriptors (and this BIP), is a much cleaner way to handle wallet
> creation and backup, by separating the two layers (keys and scripts) and
> getting rid of redundant information.
>
>
> Personal Fingerprint:  BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F
>
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Sunday, March 14, 2021 11:13 AM, SomberNight <
> somber.night@protonmail.com> wrote:
>
> > See some replies inline. (quoted text from BIP draft)
> >
> > > Date: Sun, 14 Mar 2021 01:51:15 +0000
> > > From: Robert Spigler RobertSpigler@protonmail.ch
> > > Subject: [bitcoin-dev] Signature and Script Independent Hierarchy for
> Deterministic Wallets.
> >
> > > There are many issues with the current standards. As background, BIP
> 44/49/84 specifies:
> > > `m / purpose' / coin_type' / account' / change / address_index`
> > > where the BIP43 `purpose'` path is separate for each script (P2PKH,
> P2WPKH-in-P2SH, and P2WPKH respectively). However, these per-script
> derivations are made redundant with descriptors
> >
> > > We should not be mixing keys and scripts in the same layer. The wallet
> should create extended private/public keys independent of the script or
> signature type
> >
> > You say that keys and scripts should not be mixed in the same layer, and
> imply that this was solely done due to these standards predating output
> script descriptors. Even if this was the case, it is not the only reason
> for doing it. BIP44/49/84 mixing scripts and keys in the same layer makes
> recovery from seed/mnemonic much easier.
> > Note the significant overlap between the authors of BIP39 and BIP44. I
> am fairly certain BIP44 was designed with recovering from a BIP39 seed (and
> no additional information backed up) in mind. Note the "Account discovery"
> section of BIP44.
> > (Electrum seeds go even further, as such seeds contain a version number
> that encodes both the script type and the key derivation path to use.)
> >
> > > We define the following 5 levels in the BIP32 path:
> > > `m / purpose' / coin_type' / account' / change / address_index`
> >
> > > [Account]
> > > It is crucial that this level is increased for each new wallet joined
> or private/public keys created; for both privacy and cryptographic purposes.
> > > For example, in multisignature wallets, before sending a new key
> record to a coordinator, the wallet must increment the `account'` level.
> Before creating it's own single signature wallet, the `account'` level must
> again be incremented.
> >
> > Imagine a user who has a BIP39 (or similar) seed. Even today, recovering
> most non-singlesig scripts from that is obviously infeasible. However, all
> singlesig scripts at least can be discovered if the keys are using the
> suggested derivation paths.
> > By trying to create a standard that mixes discoverable and
> non-discoverable scripts in the same derivation scheme and incrementing a
> single index, you are turning all scripts into being non-discoverable.
> > Note that even if a user only used singlesig scripts and followed this
> proposal, during recovery from seed the wallet would have to check all
> script types for all account indices (which is only ever going to get more
> expensive as new script types come).
> > The workaround and I imagine your suggested solution is clearly to
> backup both seed words and output script descriptors; and to keep appending
> new output script descriptors to existing backups when the account index is
> incremented. While much less user-friendly than backing up just a seed, it
> is more generic and extendable.
> >
> > My point is simply that your proposal is making a tradeoff here. The
> tradeoff itself seems easy to miss on first read of the text, so I just
> wanted to explicitly point it out for the record.
> >
> > ghost43 / SomberNight
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 7063 bytes --]

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Robert Spigler]

[-- Attachment #1: Type: text/plain, Size: 12696 bytes --]

Please keep comments in the ML. That way it can be archived, seen be anyone interested in the proposal, and have as much review and improvements as necessary/possible. The ML isn't taproot activation discussion only :)

For summary:

David Harding commented to Craig's post:

[Replying off list as I'm not sure this is an important point and the
list is already noisy these days.]

On Wed, Mar 17, 2021 at 09:26:50AM +0200, Craig Raw via bitcoin-dev wrote:
> I think it's important to note that [not encoding the script type in
> the derivation path] shifts more burden of wallet backup onto the
> user, who now has an additional item (the output descriptor) to backup
> correctly.

I don't think that's correct. Most wallets today that allow you to
restore with just a seed by deriving the keys for that seed in the BIP32
paths the wallet supports (i.e, including the purpose field) and then
scanning for outputs paying addresses derived from those keys.

If you try entering your seed into a different wallet (or one that's not
maintaining backwards compatibility with earlier versions of itself), it
won't necessarily support the same addresses you used earlier and so may
not work even if it fully supports BIP44 in principle. You'll need to
use different software and may need to tell that software the addresses
and paths your previous software used.

Under this proposed standard, I don't think any of that changes. If you
try to restore a seed into the same wallet that created it, it can scan
for the address types it supports. A different or
backwards-incompatible wallet may not work and you may need to find
software that can handle your old address type and specific path
information, which descriptors (and miniscript) makes easier.

The way I see it, there's not really a change to the information the
user needs to record for the normal case (seed phrase + name of software
used) or the unusual case (seed phrase + address types + paths).

Where I think this new proposal excels is in handling multisigs, where
the user needs to record the other signers' keys anyway. In those
cases, the fairly small overhead of descriptors compared to a minimalist
encoding is worth descriptor's benefits of providing an error detection
code and BIP32 fingerprints/paths that allow key location.

> 2. Use a standard set of derivation paths that encode the script type,
> ensuring that even if users do not back this up, they can check against a
> handful of common derivations and recover the funds.

I think the scanning work in this case is the same whether you use
descriptors or not.

- Under the current BIP44-style paths, a wallet that supports P2PKH and
P2WPKH with a single account and a gap limit of 50 needs to scan:

pkh([deadbeef/44'/0'/0'/0/0])
pkh([deadbeef/44'/0'/0'/0/1])
pkh([deadbeef/44'/0'/0'/0/...])
pkh([deadbeef/44'/0'/0'/0/49])

wpkh([deadbeef/84'/0'/0'/0/0])
wpkh([deadbeef/84'/0'/0'/0/1])
wpkh([deadbeef/84'/0'/0'/0/...])
wpkh([deadbeef/84'/0'/0'/0/49])

- Under the proposed standard, a wallet that suports P2PKH and P2WPKH
with a single account and a gap limit of 50 needs to scan:

pkh([deadbeef/XY'/0'/0'/0/0])
pkh([deadbeef/XY'/0'/0'/0/1])
pkh([deadbeef/XY'/0'/0'/0/...])
pkh([deadbeef/XY'/0'/0'/0/49])

wpkh([deadbeef/XY'/0'/0'/0/0])
wpkh([deadbeef/XY'/0'/0'/0/1])
wpkh([deadbeef/XY'/0'/0'/0/...])
wpkh([deadbeef/XY'/0'/0'/0/49])

It's essentially the same amount of work in either case. (It's
technically less work under the new standard as you only need to derive
the set of keys once---but since the actual scanning process typically
takes many orders of magnitude longer than key derivation, this doesn't
matter.)

There is a one-time cost of transaction, where wallets that previously
supported BIP44-style purposes and switch to the new proposal need to
scan both sets of paths. That can be avoided if the new paths are only
implemented for new address types (e.g. wallet previously supported
P2PKH with 44', P2WPKH with 84', but now the wallet adds P2TR support
with XY').

-Dave

***********

To which ghost43 replied:

Hi Dave,

>> 2. Use a standard set of derivation paths that encode the script type,
>> ensuring that even if users do not back this up, they can check against a
>> handful of common derivations and recover the funds.
>
> I think the scanning work in this case is the same whether you use
> descriptors or not.
>
> - Under the current BIP44-style paths, a wallet that supports P2PKH and
> P2WPKH with a single account and a gap limit of 50 needs to scan:
>
> pkh([deadbeef/44'/0'/0'/0/0])
> pkh([deadbeef/44'/0'/0'/0/...])
> pkh([deadbeef/44'/0'/0'/0/49])
>
> wpkh([deadbeef/84'/0'/0'/0/0])
> wpkh([deadbeef/84'/0'/0'/0/...])
> wpkh([deadbeef/84'/0'/0'/0/49])
>
> - Under the proposed standard, a wallet that suports P2PKH and P2WPKH
> with a single account and a gap limit of 50 needs to scan:
>
> pkh([deadbeef/XY'/0'/0'/0/0])
> pkh([deadbeef/XY'/0'/0'/0/1])
> pkh([deadbeef/XY'/0'/0'/0/...])
> pkh([deadbeef/XY'/0'/0'/0/49])
>
> wpkh([deadbeef/XY'/0'/0'/0/0])
> wpkh([deadbeef/XY'/0'/0'/0/1])
> wpkh([deadbeef/XY'/0'/0'/0/...])
> wpkh([deadbeef/XY'/0'/0'/0/49])

That is not how I understand the BIP draft. I think the text specifically
mentions avoiding key reuse between different scripts (by increasing the
account index); unlike your example list which reuses keys between pkh and
wpkh.

From the draft BIP's text [0]:

> It is crucial that this level is increased for each new wallet joined or
> private/public keys created; for both privacy and cryptographic purposes.
> For example, in multisignature wallets, before sending a new key record
> to a coordinator, the wallet must increment the account' level. Before
> creating it's own single signature wallet, the account' level must again
> be incremented. This prevents key reuse - across single signature and
> multisignature wallets, across ECDSA and Schnorr signatures, and inbetween
> the same wallet types.

It is good not to reuse keys. However this raises a new issue, which is what
I was alluding to in my previous email: when restoring from seed the wallet
will have to try all scripts for all account indices. Continuing your example,
it could be that account 0 is pkh and account 1 is wpkh or it could be the
other way around. Or the user might have created two wpkh accounts -- after all,
BIP44 allows that, and this new BIP is trying to be even more generic so why not?

> It's essentially the same amount of work in either case.

No, it is more work; and it will be even more in the future as new script types
come. It is the same amount of work only in the best case if the user backed up
descriptors; it is a lot more if they did not.

ghost43

*****************

David replied back:

On Wed, Mar 17, 2021 at 11:29:26PM +0000, SomberNight wrote:
> That is not how I understand the BIP draft. I think the text specifically
> mentions avoiding key reuse between different scripts (by increasing the
> account index); unlike your example list which reuses keys between pkh and
> wpkh.

There's no actual reuse there. Let's say the Foo Wallet v1 used P2PKH
and v2 defaulted to P2WPKH (but still allowed spending from P2PKH).
User Alice started with v1, so Foo Wallet gave her addresses,

pkh([a11ce000/XY'/0'/0'/0/0])
pkh([a11ce000/XY'/0'/0'/0/1])
pkh([a11ce000/XY'/0'/0'/0/...])

When she upgraded to v2, it gave her the following new addresses (note
the antepenultimate path level (the account level) is incremented):

wpkh([a11ce000/XY'/0'/1'/0/0])
wpkh([a11ce000/XY'/0'/1'/0/1])
wpkh([a11ce000/XY'/0'/1'/0/...])

Now we consider Bob who started with the v2 version of the wallet that
defaulted to P2WPKH. His addresses are:

wpkh([b0bb10c5/XY'/0'/0'/0/0])
wpkh([b0bb10c5/XY'/0'/0'/0/1])
wpkh([b0bb10c5/XY'/0'/0'/0/...])

This means that, when Alice restores from backup, the wallet looks
through the zeroeth account to see if it finds any P2PKH outputs; when it
does, it also checks the first account to see if there are any P2WPKH
outputs. When Bob restores from backup, it scans for P2PKH, doesn't find
any, scans fro P2WPH and finds them.

> It is good not to reuse keys. However this raises a new issue, which is what
> I was alluding to in my previous email: when restoring from seed the wallet
> will have to try all scripts for all account indices. Continuing your example,
> it could be that account 0 is pkh and account 1 is wpkh or it could be the
> other way around. Or the user might have created two wpkh accounts -- after all,
> BIP44 allows that, and this new BIP is trying to be even more generic so why not?

So to make sure I understand your corncern, following the example abole
there is an edge case there where Alice creates a v1 wallet, hands out
a P2PKH address that doesn't get used, and then upgrades to v2 and
starts using P2WPKH addresses. Is that your concern?

I agree, that's a problem. I'll have to think about that (and maybe Robert
already has).

-Dave

******************

ghost43 clarifies concerns:

Consider Carol, who creates 5 pkh accounts, then creates 1 wpkh account, then 1 wpkh-sh account, then 2 more wpkh (9 accounts overall).

If Carol is using BIP44/49/84, she will then have used:
- BIP44 account indices 0-4 inclusive,
- BIP49 account index 0,
- BIP84 account indices 0-2

If using new BIP XY, she will then have used:
- BIPXY account indices 0-4 for p2pkh
- BIPXY account index 5 for p2wpkh
- BIPXY account index 6 for p2wpkh-p2sh
- BIPXY account indices 7-8 for p2wpkh

The issue is that when restoring, with BIPXY, the wallet software will have to try all script types for all account indices: if it does not find an account, it might stop there, and miss all subsequent accounts too.

If using BIP44/49/84, when restoring, the wallet will scan:
- for p2pkh, BIP44, indices 0-5, stopping at 5 as it has not been used: scanned 5 accounts worth of addresses (+1 extra address for account idx 5)
- for p2wpkh-p2sh, BIP49, indices 0 and 1, stopping at 1 as it has not been used: scanned 1 account worth of addresses (+1 extra address for account idx 1)
- for p2wpkh, BIP84, indices 0-3, stopping at idx 3 as it has not been used: scanned 3 accoutns worth of addresses (+1 extra address for account idx 3)
All together the wallet needs to scan ~9 accounts worth of addresses which matches the 9 accounts the user actually created.

If using new BIP XY, when restoring, the wallet will scan:
- address index 0, hmmm wonder what script type the user might have used here, let's try p2wpkh as that is the cool thing today... --- ends up trying all script types (in worst case) until it finds p2pkh
- address index 1, hmmm surely the user used the new script type by then? tries all script types in worst case again
- ...
All together the wallet might end up scanning 9 * n (maybe three for now; but what about p2pk; or p2tr?) accounts worth of addresses.

ghost43

***********

ghost43 clarifies further:

> If using new BIP XY, she will then have used:
> - BIPXY account indices 0-4 for p2pkh
> - BIPXY account index 5 for p2wpkh
> - BIPXY account index 6 for p2wpkh-p2sh
> - BIPXY account indices 7-8 for p2wpkh

> The issue is that when restoring, with BIPXY, the wallet software will have to try all script types for all account indices: if it does not find an account, it might stop there, and miss all subsequent accounts too.

In fact note that if an older wallet does not support p2wpkh; only p2pkh and p2wpkh-p2sh, it only find accounts 0-4, and miss account 6.
This is a regression compared to BIP44/49.

**********

David Replies:

On Thu, Mar 18, 2021 at 12:22:48AM +0000, SomberNight wrote:
> > If using new BIP XY, she will then have used:
> > - BIPXY account indices 0-4 for p2pkh
> > - BIPXY account index 5 for p2wpkh
> > - BIPXY account index 6 for p2wpkh-p2sh
> > - BIPXY account indices 7-8 for p2wpkh
>
> > The issue is that when restoring, with BIPXY, the wallet software will have to try all script types for all account indices: if it does not find an account, it might stop there, and miss all subsequent accounts too.
>
> In fact note that if an older wallet does not support p2wpkh; only p2pkh and p2wpkh-p2sh, it only find accounts 0-4, and miss account 6.
> This is a regression compared to BIP44/49.

Agreed. The risk that older wallet software will reuse keys because it's
unaware of what a newer version of itself is doing negates my whole
premise of being able to use seeds without explicit descriptors.

Thank you for your insight and for arguing with me!

-Dave

*******

Craig response:

Great explanation SomberNight.

A pity now this isn't on-list, would like to share it in future!

Craig

*********

I (Robert Spigler) will respond in a next post.

Thanks for your comments!

[-- Attachment #2: Type: text/html, Size: 18131 bytes --]

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Robert Spigler]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

(IMO), where this proposal really excels, is mutlisig. And if you check my commits, it actually originally was for multisig only, and I extended it for multisig and single sig wallets.

ghost43 and Jochen Hoenicke brought up important issues with this proposed BIP re: single sig wallets, so I will be reverting this back for multisig derivations. I believe that should cover all concerns.

Please view the updated BIP here: https://github.com/Rspigler/bips-1/blob/Sane_Mulitisg_deriv/Modern%20Hierarchy%20for%20Deterministic%20Multisignature%20Wallets.mediawiki

And the updated PR here: https://github.com/Rspigler/bips-1/pull/1

Thank you,

Robert

Personal Fingerprint: BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, March 18, 2021 4:44 PM, Robert Spigler via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> I (Robert Spigler) will respond in a next post.
>
> Thanks for your comments!

[-- Attachment #2: Type: text/html, Size: 1823 bytes --]

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Craig Raw]

[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]

Hi Robert,

I believe many of the same concerns apply in multisig. If one has the
threshold and all the cosigner seeds, the funds in a multisig wallet can
very likely be discovered by checking one of a handful of standard
derivation paths/script types. Is the motivation of this BIP proposal
strong enough to lose this capability?

Craig

On Thu, Mar 18, 2021 at 11:43 PM Robert Spigler <RobertSpigler@protonmail.ch>
wrote:

> (IMO), where this proposal really excels, is mutlisig.  And if you check
> my commits, it actually originally was for multisig only, and I extended it
> for multisig and single sig wallets.
>
> ghost43 and Jochen Hoenicke brought up important issues with this proposed
> BIP re: single sig wallets, so I will be reverting this back for multisig
> derivations.  I believe that should cover all concerns.
>
> Please view the updated BIP here:
> https://github.com/Rspigler/bips-1/blob/Sane_Mulitisg_deriv/Modern%20Hierarchy%20for%20Deterministic%20Multisignature%20Wallets.mediawiki
>
> And the updated PR here:  https://github.com/Rspigler/bips-1/pull/1
>
> Thank you,
>
> Robert
>
> Personal Fingerprint:  BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, March 18, 2021 4:44 PM, Robert Spigler via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>
> I (Robert Spigler) will respond in a next post.
>
> Thanks for your comments!
>
>
>

[-- Attachment #2: Type: text/html, Size: 2585 bytes --]

Re: [bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Robert Spigler]

[-- Attachment #1: Type: text/plain, Size: 1936 bytes --]

Hi Craig,

For multisignature wallets, you need to back up the descriptor anyway, especially for secure multisignature schemes (https://github.com/nunchuk-io/bips/pull/1).

To restore, the user needs M private key data, and all public keys. The descriptor provides the required public key data, with the benefit of key origin information and error detection, and tells the wallet which script to watch.

So I see this proposal as a clear gain for multisignature wallets, as it (1) gets rid of redundant data in the key record, (2) separates keys/scripts, (3) makes full use of descriptors, and (4) prevents key reuse.

>>If one has the threshold and all the cosigner seeds, the funds in a multisig wallet can very likely be discovered by checking one of a handful of standard derivation paths/script types

Old scenario: You have BIP39 words (I assume). You try BIP 45 - search 20 addresses on the first 5 accounts. Then try BIP 48 - search 20 addresses on the first 5 accounts with script type 1 (P2SH-P2WSH); then search 20 addresses on the first 5 accounts with script type 2 (P2WSH).

New scenario: Should have a descriptor, and you just do what it says.

If you don't, you have to search the `address_index` for each script. The moment funds are found, you know that script type is the only script type used for that `account'` level. Increment `account'`, and repeat.

Robert

Personal Fingerprint: BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, March 19, 2021 3:54 AM, Craig Raw <craigraw@gmail.com> wrote:

> Hi Robert,
>
> I believe many of the same concerns apply in multisig. If one has the threshold and all the cosigner seeds, the funds in a multisig wallet can very likely be discovered by checking one of a handful of standard derivation paths/script types. Is the motivation of this BIP proposal strong enough to lose this capability?
>
> Craig

[-- Attachment #2: Type: text/html, Size: 2715 bytes --]

[bitcoin-dev] Signature and Script Independent Hierarchy for Deterministic Wallets.

[Robert Spigler]

[-- Attachment #1: Type: text/plain, Size: 1867 bytes --]

Hello,

I am working on a draft BIP for a signature and script independent hierarchy for deterministic wallets.

I believe with the implementation of descriptor wallets, the typical use case of of a BIP43 `purpose’` level per script type is redundant. The differentiation of separate BIPs for multisignature derivation paths, with BIP45 and “BIP” 48, is also redundant – with path levels such as `cosigner_index` and `script_type`. Descriptors can set the order of the public keys with `multi` or have them sorted lexicographically with `sortedmulti`.

I don’t believe we should be mixing keys and scripts in the same layer. The wallet should create extended private/public keys independent of the script or signature type, whereas the descriptor language tells wallets to watch (single or multi-sig) outputs with the specified public keys.

The BIP defines the following 5 levels in the BIP32 path:

m / purpose' / coin_type' / account' / change / address_index

It is crucial that the `account’` level is increased for each new wallet joined or private/public keys created; for both privacy and cryptographic purposes. For example, in multisignature wallets, before sending a new key record to a coordinator, the wallet must increment the `account’` level. Before creating it's own single signature wallet, the `account’` level must again be incremented. This prevents key reuse - across single signature and multisignature wallets, across ECDSA and Schnorr signatures, and inbetween the same wallet types.

For full details, please see the BIP here: https://github.com/Rspigler/bips-1/blob/Sane_Mulitisg_deriv/Modern%20Derivation%20Standard.mediawiki

Please see the PR here: https://github.com/Rspigler/bips-1/pull/1

Looking forward to comments.

Thank you,

Robert Spigler

Personal Fingerprint: BF0D 3C08 A439 5AC6 11C1 5395 B70B 4A77 F850 548F

[-- Attachment #2: Type: text/html, Size: 3251 bytes --]