[bitcoin-dev] BIP Proposals for Output Script Descriptors

[bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow]

Hi All,

I've been working on formalizing the Output Script Descriptors that have
been available in Bitcoin Core for a while into BIPs. Since descriptors
are modular and have optional components, I've decided to split it into
7 BIPs, rather than a single one. The first describes descriptors in
general and does not specify any particular descriptor. However it does
describe the general operation, key expressions (including derivation
paths and key origin info), and the descriptor checksum. The following 6
BIPs specify the actual descriptors themselves. These are non-segwit
descriptor (pk, pkh, sh), segwit descriptors (wpkh, wsh), multisig
descriptors (multi, sortedmulti), the taproot descriptor (tr), the combo
descriptor, and opaque descriptors (raw, addr). This separation is so
that implementors can choose to not implement some descriptors and still
say which descriptors they support without being too difficult to
understand.

The text of all of the documents are below, and they can also be found
on github:https://github.com/achow101/bips/tree/descriptors/

Thanks,
Andrew Chow

---

<pre>
   BIP: bip-descriptors-general
   Layer: Applications
   Title: Output Script Descriptors General Operation
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-general
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

Output Script Descriptors are a simple language which can be used to
describe collections ofoutput scripts.
There can be many different descriptor fragments and functions.
This document describes the general syntax for descriptors, descriptor
checksums, and common expressions.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Bitcoin wallets traditionally have stored a set of keys which are later
serialized and mutated to produce the output scripts that the wallet
watches and the addresses it provides to users.
Typically backups have consisted of solely the private keys, nowadays
primarily in the form of BIP 39 mnemonics.
However this backup solution is insuffient, especially since the
introduction of Segregated Witness which added new output types.
Given just the private keys, it is not possible for restored wallets to
know which kinds of output scripts and addresses to produce.
This has lead to incompatibilities between wallets when restoring a
backup or exporting data for a watch only wallet.

Further complicating matters are BIP 32 derivation paths.
Although BIPs 44, 49, and 84 have specified standard BIP 32 derivation
paths for different output scripts and addresses, not all wallets
support them nor use those derivation paths.
The lack of derivation path information in these backups and exports
leads to further incompatibilities between wallets.

Current solutions to these issues have not been generic and can be
viewed as being layer violations.
Solutions such as introducing different version bytes for extended key
serialization both are a layer violation (key derivation should be
separate from script type meaning) and specific only to a particular
derivation path and script type.

Output Script Descriptors introduces a generic solution to these issues.
Script types are specified explicitly through the use of Script Expressions.
Key derivation paths are specified explicitly in Key Expressions.
These allow for creating wallet backups and exports which specify the
exact scripts, subscripts (redeemScript, witnessScript, etc.), and keys
to produce.
With the general structure specified in this BIP, new Script Expressions
can be introduced as new script types are added.
Lastly, the use of common terminology and existing standards allow for
Output Script Descriptors to be engineer readable so that the results
can be understood at a glance.

==Specification==

Descriptors consist of several types of expressions.
The top level expression is a <tt>SCRIPT</tt>.
This expression may be followed by <tt>#CHECKSUM</tt>, where
<tt>CHECKSUM</tt> is an 8 character alphanumeric descriptor checksum.

===Script Expressions===

Script Expressions (denoted <tt>SCRIPT</tt>) are expressions which
correspond directly with a Bitcoin script.
These expressions are written as functions and take arguments.
Such expressions have a script template which is filled with the
arguments correspondingly.
Expressions are written with a human readable identifier string with the
arguments enclosed with parentheses.
The identifier string should be alphanumeric and may include underscores.

The arguments to a script expression are defined by that expression itself.
They could be a script expression, a key expression, or some other
expression entirely.

===Key Expressions===

A common expression used as an argument to script expressions are key
expressions (denoted <tt>KEY</tt>).
These represent a public or private key and, optionally, information
about the origin of that key.
Key expressions can only be used as arguments to script expressions.

Key expressions consist of:
* Optionally, key origin information, consisting of:
** An open bracket <tt>[</tt>
** Exactly 8 hex characters for the fingerprint of the key where the
derivation starts (see BIP 32 for details)
** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt>  path
elements to indicate the unhardened or hardened derivation steps between
the fingerprint and the key that follows.
** A closing bracket <tt>]</tt>
* Followed by the actual key, which is either:
** A hex encoded public key, which depending the script expression, may
be either:
*** 66 hex character string beginning with <tt>02</tt> or <tt>03</tt>
representing a compressed public key
*** 130 hex character string beginning with <tt>04</tt> representing an
uncompressed public key
*** 64 hex character string representing an x-only public key
** A [[https://en.bitcoin.it/wiki/Wallet_import_format|WIF]] encoded
private key
** <tt>xpub</tt> encoded extended public key or <tt>xprv</tt> encoded
extended private key (as defined in BIP 32)
*** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt> path
elements indicating BIP 32 derivation steps to be taken after the given
extended key.
*** Optionally followed by a single <tt>/*</tt> or <tt>/*'</tt> final
step to denote all direct unhardened or hardened children.

If the <tt>KEY</tt> is a BIP 32 extended key, before output scripts can
be created, child keys must be derived using the derivation information
that follows the extended key.
When the final step is <tt>/*</tt> or <tt>/*'</tt>, an output script
will be produced for every child key index.
The derived key must be serialized as a compressed public key.

In the above specification, the hardened indicator <tt>'</tt> may be
replaced with alternative hardnened indicators of <tt>h</tt> or <tt>H</tt>.

===Character Set===

The expressions used in descriptors must only contain characters within
this character set so that the descriptor checksum will work.

The allowed characters are:
<pre>
0123456789()[],'/*abcdefgh@:$%{}
IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~
ijklmnopqrstuvwxyzABCDEFGH`#"\<space>
</pre>
Note that <tt><space></tt> on the last line is a space character.

This character set is written as 3 groups of 32 characters in this
specific order so that the checksum below can identify more errors.
The first group are the most common "unprotected" characters (i.e.
things such as hex and keypaths that do not already have their own
checksums).
Case errors cause an offset that is a multiple of 32 while as many
alphabetic characters are in the same group while following the previous
restrictions.

===Checksum===

Follwing the top level script expression is a single octothorpe
(<tt>#</tt>) followed by the 8 character checksum.
The checksum is an error correcting checksum similar to bech32.

The checksum has the following properties:
* Mistakes in a descriptor string are measured in "symbol errors". The
higher the number of symbol errors, the harder it is to detect:
** An error substituting a character from
<tt>0123456789()[],'/*abcdefgh@:$%{}</tt> for another in that set always
counts as 1 symbol error.
*** Note that hex encoded keys are covered by these characters. Extended
keys (<tt>xpub</tt> and <tt>xprv</tt>) use other characters too, but
also have their own checksum mechansim.
*** <tt>SCRIPT</tt> expression function names use other characters, but
mistakes in these would generally result in an unparsable descriptor.
** A case error always counts as 1 symbol error.
** Any other 1 character substitution error counts as 1 or 2 symbol errors.
* Any 1 symbol error is always detected.
* Any 2 or 3 symbol error in a descriptor of up to 49154 characters is
always detected.
* Any 4 symbol error in a descriptor of up to 507 characters is always
detected.
* Any 5 symbol error in a descriptor of up to 77 characters is always
detected.
* Is optimized to minimize the chance of a 5 symbol error in a
descriptor up to 387 characters is undetected
* Random errors have a chance of 1 in 2<super>40</super> of being
undetected.

The checksum itself uses the same character set as bech32:
<tt>qpzry9x8gf2tvdw0s3jn54khce6mua7l</tt>

Valid descriptor strings with a checksum must pass the criteria for
validity specified by the Python3 code snippet below.
The function <tt>descsum_check</tt> must return true when its argument
<tt>s</tt> is a descriptor consisting in the form <tt>SCRIPT#CHECKSUM</tt>.

<pre>
INPUT_CHARSET =
"0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\
"
CHECKSUM_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"
GENERATOR = [0xf5dee51989, 0xa9fdca3312, 0x1bab10e32d, 0x3706b1677a,
0x644d626ffd]

def descsum_polymod(symbols):
     """Internal function that computes the descriptor checksum."""
     chk = 1
     for value in symbols:
         top = chk >> 35
         chk = (chk & 0x7ffffffff) << 5 ^ value
         for i in range(5):
             chk ^= GENERATOR[i] if ((top >> i) & 1) else 0
     return chk

def descsum_expand(s):
     """Internal function that does the character to symbol expansion"""
     groups = []
     symbols = []
     for c in s:
         if not c in INPUT_CHARSET:
             return None
         v = INPUT_CHARSET.find(c)
         symbols.append(v & 31)
         groups.append(v >> 5)
         if len(groups) == 3:
             symbols.append(groups[0] * 9 + groups[1] * 3 + groups[2])
             groups = []
     if len(groups) == 1:
         symbols.append(groups[0])
     elif len(groups) == 2:
         symbols.append(groups[0] * 3 + groups[1])
     return symbols

def descsum_check(s):
     """Verify that the checksum is correct in a descriptor"""
     if s[-9] != '#':
         return False
     if not all(x in CHECKSUM_CHARSET for x in s[-8:]):
         return False
     symbols = descsum_expand(s[:-9]) + [CHECKSUM_CHARSET.find(x) for x
in s[-8:]]
     return descsum_polymod(symbols) == 1
</pre>

This implements a BCH code that has the properties described above.
The entire descriptor string is first processed into an array of symbols.
The symbol for each character is its position within its group.
After every 3rd symbol, a 4th symbol is inserted which represents the
group numbers combined together.
This means that a change that only affects the position within a group,
or only a group number change, will only affect a single symbol.

To construct a valid checksum given a script expression, the code below
can be used:

<pre>
def descsum_create(s):
     """Add a checksum to a descriptor without"""
     symbols = descsum_expand(s) + [0, 0, 0, 0, 0, 0, 0, 0]
     checksum = descsum_polymod(symbols) ^ 1
     return s + '#' + ''.join(CHECKSUM_CHARSET[(checksum >> (5 * (7 -
i))) & 31] for i in range(8))

</pre>

==Backwards Compatibility==

Output script descriptors are an entirely new language which is not
compatible with any existing software.
However many components of the expressions reuse encodings and
serializations defined by previous BIPs.

Output script descriptors are designed for future extension with further
fragment types and new script expressions.
These will be specified in additional BIPs.

==Reference Implemntation==

Descriptors have been implemented in Bitcoin Core since version 0.17.

---

<pre>
   BIP: bip-descriptors-segwit
   Layer: Applications
   Title: segwit Output Script Descriptors
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-segwit
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>wpkh()</tt>, and <tt>wsh()</tt> output
script descriptors.
<tt>wpkh()</tt> descriptors take a key and produces a P2WPKH output script.
<tt>wsh()</tt> descriptors take a script and produces a P2WSH output script.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Segregated Witness added 2 additional standard output script formats:
P2WPKH and P2WSH.
These expressions allow specifying those formats as a descriptor.

==Specification==

Two new script expressions are defined: <tt>wpkh()</tt>, and <tt>wsh()</tt>.

===<tt>wpkh()</tt>===

The <tt>wpkh(KEY)</tt> expression can be used as a top level expression,
or inside of a <tt>sh()</tt> descriptor.
It takes a single key expression as an argument and produces a P2WPKH
output script.
Only keys which are/has compressed public keys can be contained in a
<tt>wpkh()</tt> expression.

The output script produced is:
<pre>
OP_0 <KEY_hash160>
</pre>

===<tt>wsh()</tt>===

The <tt>wsh(SCRIPT)</tt> expression can be used as a top level
expression, or inside of a <tt>sh()</tt> descriptor.
It takes a single script expression as an argument and produces a P2WSH
output script.
<tt>wsh()</tt> expressions also create a witnessScript which is required
in order to spend outputs which use its output script.
This redeemScript is the output script produced by the <tt>SCRIPT</tt>
argument to <tt>wsh()</tt>.
Any key expression found in any script expression contained by a
<tt>wsh()</tt> expression must only produce compresed public keys.

The output script produced is:
<pre>
OP_0 <SCRIPT_sha256>
</pre>

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors use the format and
general operation specified in
[[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As these are a wholly new descriptors, they are not compatible with any
implementation.
However the scripts produced are standard scripts so existing software
are likely to be familiar with them.

==Reference Implemntation==

<tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors have been implemented in
Bitcoin Core since version 0.17.

---

<pre>
   BIP: bip-descriptors-non-segwit
   Layer: Applications
   Title: Non-segwit Output Script Descriptors
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-non-segwit
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt>
output script descriptors.
<tt>pk()</tt> descriptors take a key and produces a P2PK output script.
<tt>pkh()</tt> descriptors take a key and produces a P2PKH output script.
<tt>sh()</tt> descriptors take a script and produces a P2SH output script.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Prior to the activation of Segregated Witness, there were 3 main
standard output script formats: P2PK, P2PKH, and P2SH.
These expressions allow specifying those formats as a descriptor.

==Specification==

Three new script expressions are defined: <tt>pk()</tt>, <tt>pkh()</tt>,
and <tt>sh()</tt>.

===<tt>pk()</tt>===

The <tt>pk(KEY)</tt> expression can be used in any context or level of a
descriptor.
It takes a single key expression as an argument and produces a P2PK
output script.
Depending on the higher level descriptors, there may be restrictions on
the type of public keys that can be included.
Such restrictions will be specified by those descriptors.

The output script produced is:
<pre>
<KEY> OP_CHECKSIG
</pre>

===<tt>pkh()</tt>===

The <tt>pkh(KEY)</tt> expression can be used as a top level expression,
or inside of either a <tt>sh()</tt> or <tt>wsh()</tt> descriptor.
It takes a single key expression as an argument and produces a P2PKH
output script.
Depending on the higher level descriptors, there may be restrictions on
the type of public keys that can be included.
Such restrictions will be specified by those descriptors.

The output script produced is:
<pre>
OP_DUP OP_HASH160 <KEY_hash160> OP_EQUALVERIFY OP_CHECKSIG
</pre>

===<tt>sh()</tt>===

The <tt>sh(SCRIPT)</tt> expression can only be used as a top level
expression.
It takes a single script expression as an argument and produces a P2SH
output script.
<tt>sh()</tt> expressions also create a redeemScript which is required
in order to spend outputs which use its output script.
This redeemScript is the output script produced by the <tt>SCRIPT</tt>
argument to <tt>sh()</tt>.

The output script produced is:
<pre>
OP_HASH160 <SCRIPT_hash160> OP_EQUAL
</pre>

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors use the
format and general operation specified in
[[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As these are a wholly new descriptors, they are not compatible with any
implementation.
However the scripts produced are standard scripts so existing software
are likely to be familiar with them.

==Reference Implemntation==

<tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors have been
implemented in Bitcoin Core since version 0.17.

---

<pre>
   BIP: bip-descriptors-tr
   Layer: Applications
   Title: tr() Output Script Descriptors
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-tr
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>tr()</tt> output script descriptors.
<tt>tr()</tt> descriptors take a key and optionally a tree of scripts
and produces a P2TR output script.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Taproot added one additional standard output script format: P2TR.
These expressions allow specifying those formats as a descriptor.

==Specification==

A new script expressions are defined: <tt>tr()</tt>.
A new expression is defined: Tree Expressions

===Tree Expression===

A Tree Expression (denoted <tt>TREE</tt>) is an expression which
represents a tree of scripts.
The way the tree is represented in an output script is dependent on the
higher level expressions.

A Tree Expression is:
* Any Script Expression that is allowed at the level this Tree
Expression is in.
* A pair of Tree Expressions consisting of:
** An open brace <tt>{</tt>
** A Tree Expression
** A comma <tt>,</tt>
** A Tree Expression
** A closing brance <tt>}</tt>

===<tt>tr()</tt>===

The <tt>tr(KEY)</tt> or <tt>tr(KEY, TREE)</tt> expression can only be
used as a top level expression.
All key expressions under any <tt>tr()</tt> expression must create
x-only public keys.

<tt>tr(KEY</tt> takes a single key expression as an argument and
produces a P2TR output script which does not have a script path.
The keys produced by the key expression are used as the internal key as
specified by [[bip-0341.mediawiki#cite_ref-22-0|BIP 341]].
Specifically, "If the spending conditions do not require a script path,
the output key should commit to an unspendable script path instead of
having no script path.
This can be achieved by computing the output key point as ''Q = P +
int(hash<sub>TapTweak</sub>(bytes(P)))G''."

<pre>
internal_key:       lift_x(KEY)
32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)))G
scriptPubKey:       OP_1 <32_byte_output_key>
</pre>

<tt>tr(KEY, TREE)</tt> takes a key expression as the first argument, and
a tree expression as the second argument and produces a P2TR output
script which has a script path.
The keys produced by the first key expression are used as the internal
key as specified by
[[bip-0341.mediawiki#Constructing_and_spending_Taproot_outputs|BIP 341]].
The Tree expression becomes the Taproot script tree as described in BIP 341.
A merkle root is computed from this tree and combined with the internal
key to create the Taproot output key.

<pre>
internal_key:       lift_x(KEY)
merkle_root:        HashTapBranch(TREE)
32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)
|| merkle_root))G
scriptPubKey:       OP_1 <32_byte_output_key>
</pre>

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>tr()</tt> descriptors use the format and general operation specified
in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As these are a wholly new descriptors, they are not compatible with any
implementation.
However the scripts produced are standard scripts so existing software
are likely to be familiar with them.

Tree Expressions are largely incompatible with existing script
expressions due to the restrictions in those expressions.
As of 2021-06-27, the only allowed script expression that can be used in
a tree expression is <tt>pk()</tt>.
However there will be future BIPs that specify script expressions that
can be used in tree expressions.

==Reference Implemntation==

<tt>tr()</tt> descriptors have been implemented in Bitcoin Core since
version 22.0.

---

<pre>
   BIP: bip-descriptors-multi
   Layer: Applications
   Title: Multisig Output Script Descriptors
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-multi
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>multi()</tt>, and <tt>sortedmulti()</tt>
output script descriptors.
Both functions take a threshold and one or more public keys and produce
a multisig output script.
<tt>multi()</tt> specifies the public keys in the output script in the
order given in the descriptor while <tt>sortedmulti()</tt> sorts the
public keys lexicographically when the output script is produced.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

The most common complex script used in Bitcoin is a threshold multisig.
These expressions allow specifying multisig scripts as a descriptor.

==Specification==

Two new script expressions are defined: <tt>multi()</tt>, and
<tt>sortedmulti()</tt>.
Both expressions produce the scripts of the same template and take the
same arguments.
They are written as <tt>multi(k,KEY_1,KEY_2,...,KEY_n)</tt>.
<tt>k</tt> is the threshold - the number of keys that must sign the
input for the script to be valid.
<tt>KEY_1,KEY_2,...,KEY_n</tt> are the key expressions for the multisig.
<tt>k</tt> must be less than or equal to <tt>n<tt>.

<tt>multi()</tt> and <tt>sortedmulti()</tt> expressions can be used as a
top level expression, or inside of either a <tt>sh()</tt> or
<tt>wsh()</tt> descriptor.
Depending on the higher level descriptors, there may be restrictions on
the type of public keys that can be included.

Depending on the higher level descriptors, there are also restrictions
on the number of keys that can be present, i.e. the maximum value of
<tt>n</tt>.
When used at the top level, there can only be at most 3 keys.
When used inside of a <tt>sh()</tt> expression, there can only be most
15 compressed public keys (this is limited by the P2SH script limit).
Otherwise the maximum number of keys is 20.

The output script produced also depends on the value of <tt>k</tt>. If
<tt>k</tt> is less than or equal to 16:
<pre>
OP_k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
</pre>

if <tt>k</tt> is greater than 16:
<pre>
k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
</pre>

===<tt>sortedmulti()</tt>===

The only change for <tt>sortedmulti()</tt> is that the keys are sorted
lexicographically prior to the creation of the output script.
This sorting is on the keys that are to be put into the output script,
i.e. after all extended keys are derived.

===Multiple Extended Keys</tt>===

When one or more the key expressions in a <tt>multi()</tt> or
<tt>sortedmulti()</tt> expression are extended keys, the derived keys
use the same child index.
This changes the keys in lockstep and allows for output scripts to be
indexed in the same way that the derived keys are indexed.

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>multi()</tt>, and <tt>sortedmulti()</tt> descriptors use the format
and general operation specified in
[[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As these are a wholly new descriptors, they are not compatible with any
implementation.
However the scripts produced are standard scripts so existing software
are likely to be familiar with them.

==Reference Implemntation==

<tt>multi()</tt>, and <tt>multi()</tt> descriptors have been implemented
in Bitcoin Core since version 0.17.

---

<pre>
   BIP: bip-descriptors-combo
   Layer: Applications
   Title: combo() Output Script Descriptors
   Author: Pieter Wuille <pieter@wuille.net>
           Andrew Chow <andrew@achow101.com>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-combo
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>combo()</tt> output script descriptors.
These take a key and produce P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH output
scripts if applicable to the key.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

In order to make the transition from traditional key based wallets to
descriptor based wallets easier, it is useful to be able to take a key
and produce the scripts which have traditionally been produced by wallet
software.

==Specification==

A new top level script expression is defined: <tt>combo(KEY)</tt>.
This expression can only be used as a top level expression.
It takes a single key expression as an argument and produces either 2 or
4 output scripts, depending on the key.
A <tt>combo()</tt> expression always produces a P2PK and P2PKH script,
the same as putting the key in both a <tt>pk()</tt> and a <tt>pkh()</tt>
expression.
If the key is/has a compressed public key, then P2WPKH and P2SH-P2WPKH
scripts are also produced, the same as putting the key in both a
<tt>wpkh()</tt> and <tt>sh(wpkh())</tt> expression.

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>combo()</tt> descriptors use the format and general operation
specified in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As this is a wholly new descriptor, it is not compatible with any
implementation.
However the scripts produced are standard scripts so existing software
are likely to be familiar with them.

==Reference Implemntation==

<tt>combo</tt> descriptors have been implemented in Bitcoin Core since
version 0.17.

---

<pre>
   BIP: bip-descriptors-encap
   Layer: Applications
   Title: raw() and addr() Output Script Descriptors
   Author: Andrew Chow <andrew@achow101.com>
           Pieter Wuille <pieter@wuille.net>
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-raw
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause
</pre>

==Abstract==

This document specifies <tt>raw()</tt> and <tt>addr()</tt> output script
descriptors.
<tt>raw()</tt> encapsulates a raw script as a descriptor.
<tt>addr()</tt> encapsulates an address as a descriptor.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

In order to make descriptors maximally compatible with scripts in use
today, it is useful to be able to wrap any arbitrary output script or an
address into a descriptor.

==Specification==

Two new script expressions are defined: <tt>raw()</tt> and <tt>addr()</tt>.

===<tt>raw()</tt>===

The <tt>raw(HEX)</tt> expression can only be used as a top level descriptor.
As the argument, it takes a hex string representing a Bitcoin script.
The output script produced by this descriptor is the script represented
by <tt>HEX</tt>.

===<tt>addr()</tt>===

The <tt>addr(ADDR)</tt> expression can only be used as a top level
descriptor.
It takes an address as its single argument.
The output script produced by this descriptor is the output script
produced by the address <tt>ADDR</tt>.

==Test Vectors==

TBD

==Backwards Compatibility==

<tt>raw()</tt> and <tt>addr()</tt> descriptors use the format and
general operation specified in
[[bip-descriptor-general.mediawiki|bip-descriptor-general]].
As this is a wholly new descriptor, it is not compatible with any
implementation.
The reuse of existing Bitcoin addresses allows for this to be more
easily implemented.

==Reference Implemntation==

<tt>raw()</tt> and <tt>addr</tt> descriptors have been implemented in
Bitcoin Core since version 0.17.

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Christopher Allen]

[-- Attachment #1: Type: text/plain, Size: 467 bytes --]

Are there any plans other than `raw` to support time locks in descriptors?

Any plans for descriptors offering closer integration with miniscript?

All of Blockchain Commons libraries and tools are multisig descriptor
centric, and there are many scenarios that require describing time locks:

   - [Designing Multisig for Independence & Resilience](
   https://github.com/BlockchainCommons/Gordian/blob/master/Docs/Multisig.md
   )

— Christopher Allen

[-- Attachment #2: Type: text/html, Size: 878 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

On 6/29/21 6:22 PM, Christopher Allen wrote:

> Are there any plans other than `raw` to support time locks in descriptors?
>
> Any plans for descriptors offering closer integration with miniscript?

I expect miniscript to be a followup BIP that extends these descriptors. Miniscript has timelock functionality.

Andrew

> All of Blockchain Commons libraries and tools are multisig descriptor centric, and there are many scenarios that require describing time locks:
>
> - [Designing Multisig for Independence & Resilience](https://github.com/BlockchainCommons/Gordian/blob/master/Docs/Multisig.md)
>
> — Christopher Allen

[-- Attachment #2: Type: text/html, Size: 1819 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow]

[-- Attachment #1: Type: text/plain, Size: 32273 bytes --]

I've opened a PR against my own BIPs fork for review: https://github.com/achow101/bips/pull/3

Andrew

On 6/29/21 11:41 PM, Jeremy wrote:

> Kudos, this is fantastic!
>
> It might be easier, since there is a ton of content here, for you to open up some WIP PRs to collect feedback?
> --
> [@JeremyRubin](https://twitter.com/JeremyRubin)
>
> On Tue, Jun 29, 2021 at 2:15 PM Andrew Chow via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Hi All,
>>
>> I've been working on formalizing the Output Script Descriptors that have
>> been available in Bitcoin Core for a while into BIPs. Since descriptors
>> are modular and have optional components, I've decided to split it into
>> 7 BIPs, rather than a single one. The first describes descriptors in
>> general and does not specify any particular descriptor. However it does
>> describe the general operation, key expressions (including derivation
>> paths and key origin info), and the descriptor checksum. The following 6
>> BIPs specify the actual descriptors themselves. These are non-segwit
>> descriptor (pk, pkh, sh), segwit descriptors (wpkh, wsh), multisig
>> descriptors (multi, sortedmulti), the taproot descriptor (tr), the combo
>> descriptor, and opaque descriptors (raw, addr). This separation is so
>> that implementors can choose to not implement some descriptors and still
>> say which descriptors they support without being too difficult to
>> understand.
>>
>> The text of all of the documents are below, and they can also be found
>> on github:https://github.com/achow101/bips/tree/descriptors/
>>
>> Thanks,
>> Andrew Chow
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-general
>> Layer: Applications
>> Title: Output Script Descriptors General Operation
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-general
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> Output Script Descriptors are a simple language which can be used to
>> describe collections ofoutput scripts.
>> There can be many different descriptor fragments and functions.
>> This document describes the general syntax for descriptors, descriptor
>> checksums, and common expressions.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Bitcoin wallets traditionally have stored a set of keys which are later
>> serialized and mutated to produce the output scripts that the wallet
>> watches and the addresses it provides to users.
>> Typically backups have consisted of solely the private keys, nowadays
>> primarily in the form of BIP 39 mnemonics.
>> However this backup solution is insuffient, especially since the
>> introduction of Segregated Witness which added new output types.
>> Given just the private keys, it is not possible for restored wallets to
>> know which kinds of output scripts and addresses to produce.
>> This has lead to incompatibilities between wallets when restoring a
>> backup or exporting data for a watch only wallet.
>>
>> Further complicating matters are BIP 32 derivation paths.
>> Although BIPs 44, 49, and 84 have specified standard BIP 32 derivation
>> paths for different output scripts and addresses, not all wallets
>> support them nor use those derivation paths.
>> The lack of derivation path information in these backups and exports
>> leads to further incompatibilities between wallets.
>>
>> Current solutions to these issues have not been generic and can be
>> viewed as being layer violations.
>> Solutions such as introducing different version bytes for extended key
>> serialization both are a layer violation (key derivation should be
>> separate from script type meaning) and specific only to a particular
>> derivation path and script type.
>>
>> Output Script Descriptors introduces a generic solution to these issues.
>> Script types are specified explicitly through the use of Script Expressions.
>> Key derivation paths are specified explicitly in Key Expressions.
>> These allow for creating wallet backups and exports which specify the
>> exact scripts, subscripts (redeemScript, witnessScript, etc.), and keys
>> to produce.
>> With the general structure specified in this BIP, new Script Expressions
>> can be introduced as new script types are added.
>> Lastly, the use of common terminology and existing standards allow for
>> Output Script Descriptors to be engineer readable so that the results
>> can be understood at a glance.
>>
>> ==Specification==
>>
>> Descriptors consist of several types of expressions.
>> The top level expression is a <tt>SCRIPT</tt>.
>> This expression may be followed by <tt>#CHECKSUM</tt>, where
>> <tt>CHECKSUM</tt> is an 8 character alphanumeric descriptor checksum.
>>
>> ===Script Expressions===
>>
>> Script Expressions (denoted <tt>SCRIPT</tt>) are expressions which
>> correspond directly with a Bitcoin script.
>> These expressions are written as functions and take arguments.
>> Such expressions have a script template which is filled with the
>> arguments correspondingly.
>> Expressions are written with a human readable identifier string with the
>> arguments enclosed with parentheses.
>> The identifier string should be alphanumeric and may include underscores.
>>
>> The arguments to a script expression are defined by that expression itself.
>> They could be a script expression, a key expression, or some other
>> expression entirely.
>>
>> ===Key Expressions===
>>
>> A common expression used as an argument to script expressions are key
>> expressions (denoted <tt>KEY</tt>).
>> These represent a public or private key and, optionally, information
>> about the origin of that key.
>> Key expressions can only be used as arguments to script expressions.
>>
>> Key expressions consist of:
>> * Optionally, key origin information, consisting of:
>> ** An open bracket <tt>[</tt>
>> ** Exactly 8 hex characters for the fingerprint of the key where the
>> derivation starts (see BIP 32 for details)
>> ** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt> path
>> elements to indicate the unhardened or hardened derivation steps between
>> the fingerprint and the key that follows.
>> ** A closing bracket <tt>]</tt>
>> * Followed by the actual key, which is either:
>> ** A hex encoded public key, which depending the script expression, may
>> be either:
>> *** 66 hex character string beginning with <tt>02</tt> or <tt>03</tt>
>> representing a compressed public key
>> *** 130 hex character string beginning with <tt>04</tt> representing an
>> uncompressed public key
>> *** 64 hex character string representing an x-only public key
>> ** A [[[https://en.bitcoin.it/wiki/Wallet_import_format|WIF](https://en.bitcoin.it/wiki/Wallet_import_format%7CWIF)]] encoded
>> private key
>> ** <tt>xpub</tt> encoded extended public key or <tt>xprv</tt> encoded
>> extended private key (as defined in BIP 32)
>> *** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt> path
>> elements indicating BIP 32 derivation steps to be taken after the given
>> extended key.
>> *** Optionally followed by a single <tt>/*</tt> or <tt>/*'</tt> final
>> step to denote all direct unhardened or hardened children.
>>
>> If the <tt>KEY</tt> is a BIP 32 extended key, before output scripts can
>> be created, child keys must be derived using the derivation information
>> that follows the extended key.
>> When the final step is <tt>/*</tt> or <tt>/*'</tt>, an output script
>> will be produced for every child key index.
>> The derived key must be serialized as a compressed public key.
>>
>> In the above specification, the hardened indicator <tt>'</tt> may be
>> replaced with alternative hardnened indicators of <tt>h</tt> or <tt>H</tt>.
>>
>> ===Character Set===
>>
>> The expressions used in descriptors must only contain characters within
>> this character set so that the descriptor checksum will work.
>>
>> The allowed characters are:
>> <pre>
>> 0123456789()[],'/*abcdefgh@:$%{}
>> IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~
>> ijklmnopqrstuvwxyzABCDEFGH`#"\<space>
>> </pre>
>> Note that <tt><space></tt> on the last line is a space character.
>>
>> This character set is written as 3 groups of 32 characters in this
>> specific order so that the checksum below can identify more errors.
>> The first group are the most common "unprotected" characters (i.e.
>> things such as hex and keypaths that do not already have their own
>> checksums).
>> Case errors cause an offset that is a multiple of 32 while as many
>> alphabetic characters are in the same group while following the previous
>> restrictions.
>>
>> ===Checksum===
>>
>> Follwing the top level script expression is a single octothorpe
>> (<tt>#</tt>) followed by the 8 character checksum.
>> The checksum is an error correcting checksum similar to bech32.
>>
>> The checksum has the following properties:
>> * Mistakes in a descriptor string are measured in "symbol errors". The
>> higher the number of symbol errors, the harder it is to detect:
>> ** An error substituting a character from
>> <tt>0123456789()[],'/*abcdefgh@:$%{}</tt> for another in that set always
>> counts as 1 symbol error.
>> *** Note that hex encoded keys are covered by these characters. Extended
>> keys (<tt>xpub</tt> and <tt>xprv</tt>) use other characters too, but
>> also have their own checksum mechansim.
>> *** <tt>SCRIPT</tt> expression function names use other characters, but
>> mistakes in these would generally result in an unparsable descriptor.
>> ** A case error always counts as 1 symbol error.
>> ** Any other 1 character substitution error counts as 1 or 2 symbol errors.
>> * Any 1 symbol error is always detected.
>> * Any 2 or 3 symbol error in a descriptor of up to 49154 characters is
>> always detected.
>> * Any 4 symbol error in a descriptor of up to 507 characters is always
>> detected.
>> * Any 5 symbol error in a descriptor of up to 77 characters is always
>> detected.
>> * Is optimized to minimize the chance of a 5 symbol error in a
>> descriptor up to 387 characters is undetected
>> * Random errors have a chance of 1 in 2<super>40</super> of being
>> undetected.
>>
>> The checksum itself uses the same character set as bech32:
>> <tt>qpzry9x8gf2tvdw0s3jn54khce6mua7l</tt>
>>
>> Valid descriptor strings with a checksum must pass the criteria for
>> validity specified by the Python3 code snippet below.
>> The function <tt>descsum_check</tt> must return true when its argument
>> <tt>s</tt> is a descriptor consisting in the form <tt>SCRIPT#CHECKSUM</tt>.
>>
>> <pre>
>> INPUT_CHARSET =
>> "0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\
>> "
>> CHECKSUM_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"
>> GENERATOR = [0xf5dee51989, 0xa9fdca3312, 0x1bab10e32d, 0x3706b1677a,
>> 0x644d626ffd]
>>
>> def descsum_polymod(symbols):
>> """Internal function that computes the descriptor checksum."""
>> chk = 1
>> for value in symbols:
>> top = chk >> 35
>> chk = (chk & 0x7ffffffff) << 5 ^ value
>> for i in range(5):
>> chk ^= GENERATOR[i] if ((top >> i) & 1) else 0
>> return chk
>>
>> def descsum_expand(s):
>> """Internal function that does the character to symbol expansion"""
>> groups = []
>> symbols = []
>> for c in s:
>> if not c in INPUT_CHARSET:
>> return None
>> v = INPUT_CHARSET.find(c)
>> symbols.append(v & 31)
>> groups.append(v >> 5)
>> if len(groups) == 3:
>> symbols.append(groups[0] * 9 + groups[1] * 3 + groups[2])
>> groups = []
>> if len(groups) == 1:
>> symbols.append(groups[0])
>> elif len(groups) == 2:
>> symbols.append(groups[0] * 3 + groups[1])
>> return symbols
>>
>> def descsum_check(s):
>> """Verify that the checksum is correct in a descriptor"""
>> if s[-9] != '#':
>> return False
>> if not all(x in CHECKSUM_CHARSET for x in s[-8:]):
>> return False
>> symbols = descsum_expand(s[:-9]) + [CHECKSUM_CHARSET.find(x) for x
>> in s[-8:]]
>> return descsum_polymod(symbols) == 1
>> </pre>
>>
>> This implements a BCH code that has the properties described above.
>> The entire descriptor string is first processed into an array of symbols.
>> The symbol for each character is its position within its group.
>> After every 3rd symbol, a 4th symbol is inserted which represents the
>> group numbers combined together.
>> This means that a change that only affects the position within a group,
>> or only a group number change, will only affect a single symbol.
>>
>> To construct a valid checksum given a script expression, the code below
>> can be used:
>>
>> <pre>
>> def descsum_create(s):
>> """Add a checksum to a descriptor without"""
>> symbols = descsum_expand(s) + [0, 0, 0, 0, 0, 0, 0, 0]
>> checksum = descsum_polymod(symbols) ^ 1
>> return s + '#' + ''.join(CHECKSUM_CHARSET[(checksum >> (5 * (7 -
>> i))) & 31] for i in range(8))
>>
>> </pre>
>>
>> ==Backwards Compatibility==
>>
>> Output script descriptors are an entirely new language which is not
>> compatible with any existing software.
>> However many components of the expressions reuse encodings and
>> serializations defined by previous BIPs.
>>
>> Output script descriptors are designed for future extension with further
>> fragment types and new script expressions.
>> These will be specified in additional BIPs.
>>
>> ==Reference Implemntation==
>>
>> Descriptors have been implemented in Bitcoin Core since version 0.17.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-segwit
>> Layer: Applications
>> Title: segwit Output Script Descriptors
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-segwit
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>wpkh()</tt>, and <tt>wsh()</tt> output
>> script descriptors.
>> <tt>wpkh()</tt> descriptors take a key and produces a P2WPKH output script.
>> <tt>wsh()</tt> descriptors take a script and produces a P2WSH output script.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Segregated Witness added 2 additional standard output script formats:
>> P2WPKH and P2WSH.
>> These expressions allow specifying those formats as a descriptor.
>>
>> ==Specification==
>>
>> Two new script expressions are defined: <tt>wpkh()</tt>, and <tt>wsh()</tt>.
>>
>> ===<tt>wpkh()</tt>===
>>
>> The <tt>wpkh(KEY)</tt> expression can be used as a top level expression,
>> or inside of a <tt>sh()</tt> descriptor.
>> It takes a single key expression as an argument and produces a P2WPKH
>> output script.
>> Only keys which are/has compressed public keys can be contained in a
>> <tt>wpkh()</tt> expression.
>>
>> The output script produced is:
>> <pre>
>> OP_0 <KEY_hash160>
>> </pre>
>>
>> ===<tt>wsh()</tt>===
>>
>> The <tt>wsh(SCRIPT)</tt> expression can be used as a top level
>> expression, or inside of a <tt>sh()</tt> descriptor.
>> It takes a single script expression as an argument and produces a P2WSH
>> output script.
>> <tt>wsh()</tt> expressions also create a witnessScript which is required
>> in order to spend outputs which use its output script.
>> This redeemScript is the output script produced by the <tt>SCRIPT</tt>
>> argument to <tt>wsh()</tt>.
>> Any key expression found in any script expression contained by a
>> <tt>wsh()</tt> expression must only produce compresed public keys.
>>
>> The output script produced is:
>> <pre>
>> OP_0 <SCRIPT_sha256>
>> </pre>
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors use the format and
>> general operation specified in
>> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As these are a wholly new descriptors, they are not compatible with any
>> implementation.
>> However the scripts produced are standard scripts so existing software
>> are likely to be familiar with them.
>>
>> ==Reference Implemntation==
>>
>> <tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors have been implemented in
>> Bitcoin Core since version 0.17.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-non-segwit
>> Layer: Applications
>> Title: Non-segwit Output Script Descriptors
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-non-segwit
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt>
>> output script descriptors.
>> <tt>pk()</tt> descriptors take a key and produces a P2PK output script.
>> <tt>pkh()</tt> descriptors take a key and produces a P2PKH output script.
>> <tt>sh()</tt> descriptors take a script and produces a P2SH output script.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Prior to the activation of Segregated Witness, there were 3 main
>> standard output script formats: P2PK, P2PKH, and P2SH.
>> These expressions allow specifying those formats as a descriptor.
>>
>> ==Specification==
>>
>> Three new script expressions are defined: <tt>pk()</tt>, <tt>pkh()</tt>,
>> and <tt>sh()</tt>.
>>
>> ===<tt>pk()</tt>===
>>
>> The <tt>pk(KEY)</tt> expression can be used in any context or level of a
>> descriptor.
>> It takes a single key expression as an argument and produces a P2PK
>> output script.
>> Depending on the higher level descriptors, there may be restrictions on
>> the type of public keys that can be included.
>> Such restrictions will be specified by those descriptors.
>>
>> The output script produced is:
>> <pre>
>> <KEY> OP_CHECKSIG
>> </pre>
>>
>> ===<tt>pkh()</tt>===
>>
>> The <tt>pkh(KEY)</tt> expression can be used as a top level expression,
>> or inside of either a <tt>sh()</tt> or <tt>wsh()</tt> descriptor.
>> It takes a single key expression as an argument and produces a P2PKH
>> output script.
>> Depending on the higher level descriptors, there may be restrictions on
>> the type of public keys that can be included.
>> Such restrictions will be specified by those descriptors.
>>
>> The output script produced is:
>> <pre>
>> OP_DUP OP_HASH160 <KEY_hash160> OP_EQUALVERIFY OP_CHECKSIG
>> </pre>
>>
>> ===<tt>sh()</tt>===
>>
>> The <tt>sh(SCRIPT)</tt> expression can only be used as a top level
>> expression.
>> It takes a single script expression as an argument and produces a P2SH
>> output script.
>> <tt>sh()</tt> expressions also create a redeemScript which is required
>> in order to spend outputs which use its output script.
>> This redeemScript is the output script produced by the <tt>SCRIPT</tt>
>> argument to <tt>sh()</tt>.
>>
>> The output script produced is:
>> <pre>
>> OP_HASH160 <SCRIPT_hash160> OP_EQUAL
>> </pre>
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors use the
>> format and general operation specified in
>> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As these are a wholly new descriptors, they are not compatible with any
>> implementation.
>> However the scripts produced are standard scripts so existing software
>> are likely to be familiar with them.
>>
>> ==Reference Implemntation==
>>
>> <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors have been
>> implemented in Bitcoin Core since version 0.17.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-tr
>> Layer: Applications
>> Title: tr() Output Script Descriptors
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-tr
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>tr()</tt> output script descriptors.
>> <tt>tr()</tt> descriptors take a key and optionally a tree of scripts
>> and produces a P2TR output script.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Taproot added one additional standard output script format: P2TR.
>> These expressions allow specifying those formats as a descriptor.
>>
>> ==Specification==
>>
>> A new script expressions are defined: <tt>tr()</tt>.
>> A new expression is defined: Tree Expressions
>>
>> ===Tree Expression===
>>
>> A Tree Expression (denoted <tt>TREE</tt>) is an expression which
>> represents a tree of scripts.
>> The way the tree is represented in an output script is dependent on the
>> higher level expressions.
>>
>> A Tree Expression is:
>> * Any Script Expression that is allowed at the level this Tree
>> Expression is in.
>> * A pair of Tree Expressions consisting of:
>> ** An open brace <tt>{</tt>
>> ** A Tree Expression
>> ** A comma <tt>,</tt>
>> ** A Tree Expression
>> ** A closing brance <tt>}</tt>
>>
>> ===<tt>tr()</tt>===
>>
>> The <tt>tr(KEY)</tt> or <tt>tr(KEY, TREE)</tt> expression can only be
>> used as a top level expression.
>> All key expressions under any <tt>tr()</tt> expression must create
>> x-only public keys.
>>
>> <tt>tr(KEY</tt> takes a single key expression as an argument and
>> produces a P2TR output script which does not have a script path.
>> The keys produced by the key expression are used as the internal key as
>> specified by [[bip-0341.mediawiki#cite_ref-22-0|BIP 341]].
>> Specifically, "If the spending conditions do not require a script path,
>> the output key should commit to an unspendable script path instead of
>> having no script path.
>> This can be achieved by computing the output key point as ''Q = P +
>> int(hash<sub>TapTweak</sub>(bytes(P)))G''."
>>
>> <pre>
>> internal_key: lift_x(KEY)
>> 32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)))G
>> scriptPubKey: OP_1 <32_byte_output_key>
>> </pre>
>>
>> <tt>tr(KEY, TREE)</tt> takes a key expression as the first argument, and
>> a tree expression as the second argument and produces a P2TR output
>> script which has a script path.
>> The keys produced by the first key expression are used as the internal
>> key as specified by
>> [[bip-0341.mediawiki#Constructing_and_spending_Taproot_outputs|BIP 341]].
>> The Tree expression becomes the Taproot script tree as described in BIP 341.
>> A merkle root is computed from this tree and combined with the internal
>> key to create the Taproot output key.
>>
>> <pre>
>> internal_key: lift_x(KEY)
>> merkle_root: HashTapBranch(TREE)
>> 32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)
>> || merkle_root))G
>> scriptPubKey: OP_1 <32_byte_output_key>
>> </pre>
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>tr()</tt> descriptors use the format and general operation specified
>> in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As these are a wholly new descriptors, they are not compatible with any
>> implementation.
>> However the scripts produced are standard scripts so existing software
>> are likely to be familiar with them.
>>
>> Tree Expressions are largely incompatible with existing script
>> expressions due to the restrictions in those expressions.
>> As of 2021-06-27, the only allowed script expression that can be used in
>> a tree expression is <tt>pk()</tt>.
>> However there will be future BIPs that specify script expressions that
>> can be used in tree expressions.
>>
>> ==Reference Implemntation==
>>
>> <tt>tr()</tt> descriptors have been implemented in Bitcoin Core since
>> version 22.0.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-multi
>> Layer: Applications
>> Title: Multisig Output Script Descriptors
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-multi
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>multi()</tt>, and <tt>sortedmulti()</tt>
>> output script descriptors.
>> Both functions take a threshold and one or more public keys and produce
>> a multisig output script.
>> <tt>multi()</tt> specifies the public keys in the output script in the
>> order given in the descriptor while <tt>sortedmulti()</tt> sorts the
>> public keys lexicographically when the output script is produced.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> The most common complex script used in Bitcoin is a threshold multisig.
>> These expressions allow specifying multisig scripts as a descriptor.
>>
>> ==Specification==
>>
>> Two new script expressions are defined: <tt>multi()</tt>, and
>> <tt>sortedmulti()</tt>.
>> Both expressions produce the scripts of the same template and take the
>> same arguments.
>> They are written as <tt>multi(k,KEY_1,KEY_2,...,KEY_n)</tt>.
>> <tt>k</tt> is the threshold - the number of keys that must sign the
>> input for the script to be valid.
>> <tt>KEY_1,KEY_2,...,KEY_n</tt> are the key expressions for the multisig.
>> <tt>k</tt> must be less than or equal to <tt>n<tt>.
>>
>> <tt>multi()</tt> and <tt>sortedmulti()</tt> expressions can be used as a
>> top level expression, or inside of either a <tt>sh()</tt> or
>> <tt>wsh()</tt> descriptor.
>> Depending on the higher level descriptors, there may be restrictions on
>> the type of public keys that can be included.
>>
>> Depending on the higher level descriptors, there are also restrictions
>> on the number of keys that can be present, i.e. the maximum value of
>> <tt>n</tt>.
>> When used at the top level, there can only be at most 3 keys.
>> When used inside of a <tt>sh()</tt> expression, there can only be most
>> 15 compressed public keys (this is limited by the P2SH script limit).
>> Otherwise the maximum number of keys is 20.
>>
>> The output script produced also depends on the value of <tt>k</tt>. If
>> <tt>k</tt> is less than or equal to 16:
>> <pre>
>> OP_k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
>> </pre>
>>
>> if <tt>k</tt> is greater than 16:
>> <pre>
>> k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
>> </pre>
>>
>> ===<tt>sortedmulti()</tt>===
>>
>> The only change for <tt>sortedmulti()</tt> is that the keys are sorted
>> lexicographically prior to the creation of the output script.
>> This sorting is on the keys that are to be put into the output script,
>> i.e. after all extended keys are derived.
>>
>> ===Multiple Extended Keys</tt>===
>>
>> When one or more the key expressions in a <tt>multi()</tt> or
>> <tt>sortedmulti()</tt> expression are extended keys, the derived keys
>> use the same child index.
>> This changes the keys in lockstep and allows for output scripts to be
>> indexed in the same way that the derived keys are indexed.
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>multi()</tt>, and <tt>sortedmulti()</tt> descriptors use the format
>> and general operation specified in
>> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As these are a wholly new descriptors, they are not compatible with any
>> implementation.
>> However the scripts produced are standard scripts so existing software
>> are likely to be familiar with them.
>>
>> ==Reference Implemntation==
>>
>> <tt>multi()</tt>, and <tt>multi()</tt> descriptors have been implemented
>> in Bitcoin Core since version 0.17.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-combo
>> Layer: Applications
>> Title: combo() Output Script Descriptors
>> Author: Pieter Wuille <pieter@wuille.net>
>> Andrew Chow <andrew@achow101.com>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-combo
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>combo()</tt> output script descriptors.
>> These take a key and produce P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH output
>> scripts if applicable to the key.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> In order to make the transition from traditional key based wallets to
>> descriptor based wallets easier, it is useful to be able to take a key
>> and produce the scripts which have traditionally been produced by wallet
>> software.
>>
>> ==Specification==
>>
>> A new top level script expression is defined: <tt>combo(KEY)</tt>.
>> This expression can only be used as a top level expression.
>> It takes a single key expression as an argument and produces either 2 or
>> 4 output scripts, depending on the key.
>> A <tt>combo()</tt> expression always produces a P2PK and P2PKH script,
>> the same as putting the key in both a <tt>pk()</tt> and a <tt>pkh()</tt>
>> expression.
>> If the key is/has a compressed public key, then P2WPKH and P2SH-P2WPKH
>> scripts are also produced, the same as putting the key in both a
>> <tt>wpkh()</tt> and <tt>sh(wpkh())</tt> expression.
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>combo()</tt> descriptors use the format and general operation
>> specified in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As this is a wholly new descriptor, it is not compatible with any
>> implementation.
>> However the scripts produced are standard scripts so existing software
>> are likely to be familiar with them.
>>
>> ==Reference Implemntation==
>>
>> <tt>combo</tt> descriptors have been implemented in Bitcoin Core since
>> version 0.17.
>>
>> ---
>>
>> <pre>
>> BIP: bip-descriptors-encap
>> Layer: Applications
>> Title: raw() and addr() Output Script Descriptors
>> Author: Andrew Chow <andrew@achow101.com>
>> Pieter Wuille <pieter@wuille.net>
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-raw
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> </pre>
>>
>> ==Abstract==
>>
>> This document specifies <tt>raw()</tt> and <tt>addr()</tt> output script
>> descriptors.
>> <tt>raw()</tt> encapsulates a raw script as a descriptor.
>> <tt>addr()</tt> encapsulates an address as a descriptor.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> In order to make descriptors maximally compatible with scripts in use
>> today, it is useful to be able to wrap any arbitrary output script or an
>> address into a descriptor.
>>
>> ==Specification==
>>
>> Two new script expressions are defined: <tt>raw()</tt> and <tt>addr()</tt>.
>>
>> ===<tt>raw()</tt>===
>>
>> The <tt>raw(HEX)</tt> expression can only be used as a top level descriptor.
>> As the argument, it takes a hex string representing a Bitcoin script.
>> The output script produced by this descriptor is the script represented
>> by <tt>HEX</tt>.
>>
>> ===<tt>addr()</tt>===
>>
>> The <tt>addr(ADDR)</tt> expression can only be used as a top level
>> descriptor.
>> It takes an address as its single argument.
>> The output script produced by this descriptor is the output script
>> produced by the address <tt>ADDR</tt>.
>>
>> ==Test Vectors==
>>
>> TBD
>>
>> ==Backwards Compatibility==
>>
>> <tt>raw()</tt> and <tt>addr()</tt> descriptors use the format and
>> general operation specified in
>> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
>> As this is a wholly new descriptor, it is not compatible with any
>> implementation.
>> The reuse of existing Bitcoin addresses allows for this to be more
>> easily implemented.
>>
>> ==Reference Implemntation==
>>
>> <tt>raw()</tt> and <tt>addr</tt> descriptors have been implemented in
>> Bitcoin Core since version 0.17.
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 54067 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[David A. Harding]

[-- Attachment #1: Type: text/plain, Size: 1197 bytes --]

On Tue, Jun 29, 2021 at 09:14:39PM +0000, Andrew Chow via bitcoin-dev wrote:
> *** Optionally followed by a single <tt>/*</tt> or <tt>/*'</tt> final
> step to denote all direct unhardened or hardened children.
> 
> [...]
> 
> In the above specification, the hardened indicator <tt>'</tt> may be
> replaced with alternative hardnened indicators of <tt>h</tt> or <tt>H</tt>.

Is there any chance we can take this opportunity to make "h"/"H" the
preferred aliases?  Using "'" in bourne-style shells is very
annoying[1], and I suspect it's also creating unnecessary complications
elsewhere.

Alternatives:

- Completely kill "'" (I'd prefer this, but I realize it's complicated
  with descriptors already being used widely).  If "h"/"H" are made the
  preferred aliases, maybe it'd be enough to make implementing "'" a
  SHOULD rather than a MUST; this would push implementations towards
  displaying descriptors using the h versions for maximum compatibility.

- Calculate the checksum over s/(h|H)/'/ (again, I know that's
  complicated with descriptors already widely used)

Thanks,

-Dave

[1] https://github.com/bitcoin/bitcoin/issues/15740#issuecomment-695815432


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow]

On 7/2/21 11:24 PM, David A. Harding wrote:
> Is there any chance we can take this opportunity to make "h"/"H" the
> preferred aliases?  Using "'" in bourne-style shells is very
> annoying[1], and I suspect it's also creating unnecessary complications
> elsewhere.
I've updated the text to use "h".
> Alternatives:
>
> - Completely kill "'" (I'd prefer this, but I realize it's complicated
>    with descriptors already being used widely).  If "h"/"H" are made the
>    preferred aliases, maybe it'd be enough to make implementing "'" a
>    SHOULD rather than a MUST; this would push implementations towards
>    displaying descriptors using the h versions for maximum compatibility.
Since there already are software implementing descriptors, I don't think
we can do this. I'm not sure about making "'" a SHOULD either.
> - Calculate the checksum over s/(h|H)/'/ (again, I know that's
>    complicated with descriptors already widely used)
This has been discussed in the past and the conclusion was that the
checksum should be strictly over the string itself. This would allow for
dumb checksum checkers which don't have to be able to parse descriptors
in order to check the checksum.

Thanks,
Andrew

>
> Thanks,
>
> -Dave
>
> [1] https://github.com/bitcoin/bitcoin/issues/15740#issuecomment-695815432

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Craig Raw]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

There is a downside to using "h"/"H" from a UX perspective - taking up more
space and appearing as alphanumeric characters similar to the path numbers,
they make derivation paths and descriptors more difficult to read. Also,
although not as important, less efficient when making metal backups.

On Sat, Jul 3, 2021 at 7:13 AM Andrew Chow via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On 7/2/21 11:24 PM, David A. Harding wrote:
> > Is there any chance we can take this opportunity to make "h"/"H" the
> > preferred aliases?  Using "'" in bourne-style shells is very
> > annoying[1], and I suspect it's also creating unnecessary complications
> > elsewhere.
> I've updated the text to use "h".
> > Alternatives:
> >
> > - Completely kill "'" (I'd prefer this, but I realize it's complicated
> >    with descriptors already being used widely).  If "h"/"H" are made the
> >    preferred aliases, maybe it'd be enough to make implementing "'" a
> >    SHOULD rather than a MUST; this would push implementations towards
> >    displaying descriptors using the h versions for maximum compatibility.
> Since there already are software implementing descriptors, I don't think
> we can do this. I'm not sure about making "'" a SHOULD either.
> > - Calculate the checksum over s/(h|H)/'/ (again, I know that's
> >    complicated with descriptors already widely used)
> This has been discussed in the past and the conclusion was that the
> checksum should be strictly over the string itself. This would allow for
> dumb checksum checkers which don't have to be able to parse descriptors
> in order to check the checksum.
>
> Thanks,
> Andrew
>
> >
> > Thanks,
> >
> > -Dave
> >
> > [1]
> https://github.com/bitcoin/bitcoin/issues/15740#issuecomment-695815432
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2902 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[David A. Harding]

[-- Attachment #1: Type: text/plain, Size: 2338 bytes --]

On Sat, Jul 03, 2021 at 10:35:48AM +0200, Craig Raw wrote:
> There is a downside to using "h"/"H" from a UX perspective - taking up more
> space 

Is this a serious concern of yours?  An apostrophe is 1/2 en; an "h" is
1 en; the following descriptor contains three hardened derivations in 149
characters; assuming the average non-'/h character width is 1.5 en, the
difference between 207 en and 208.5 en is barely more than half a
percent.

    pkh([d34db33f/44h/0h/0h]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/1/*)#ml40v0wf

Here's a direct visual comparison: https://gist.github.com/harding/2fbbf2bfdce04c3e4110082f03ae3c80

> appearing as alphanumeric characters similar to the path numbers

First, I think you'd have to be using an awful font to confuse "h" with
any arabic numeral.  Second, avoiding transcription errors is exactly
why descriptors now have checksums.

> they make derivation paths and descriptors more difficult to read.

The example descriptor pasted above looks equally (un)readable to me
whether it uses ' or h.

> Also, although not as important, less efficient when making metal
> backups.

I think many metal backup schemes are using stamps or punch grids that
are fixed-width in nature, so there's no difference either way.  (And
you can argue that h is better since it's part of both the base58check
and bech32 character sets, so you already need a stamp or a grid row for
it---but ' is otherwise unused, so a stamp or grid row for it would be
special).

But even if people are manually etching descriptors into metal, we're
back to the original point where we're looking at something like a 0.7%
difference in "efficiency".

By comparison, the Bitcoin Core issue I cited in my earlier post
contains several examples of actual users needing technical support
because they tried to use '-containing descriptors in a bourne-style
shell.  (And I've personally lost time to that class of problems.)  In
the worst case, a shell-quoting accident can cause loss of money by
sending bitcoins to the descriptor for a key your hardware signing
device won't sign for.  I think these problems are much more serious
than using a tiny bit of extra space in a GUI or on a physical backup
medium.

-Dave

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Craig Raw]

[-- Attachment #1: Type: text/plain, Size: 3240 bytes --]

It's a consideration, not a serious concern.

When I made the point around alphanumeric characters being similar to the
path numbers, I was actually thinking of the output descriptor appearing in
a fixed character width font, which I prefer as more appropriate for
displaying hexidecimal values. In this case, the apostrophe provides more
whitespace which makes the path easier to parse visually. It's difficult to
reduce this to a mathematical argument, as is true for many UX
considerations. Your example in fixed width here:
https://gist.github.com/craigraw/fc98b9031a7e01e1bc5d75a77bdb72e5

That said you make good arguments around the shell quoting and stamps for
metal backups, and therefore I agree it is preferable to use the lowercase
"h". Thanks for the detailed reply.

Craig

On Sat, Jul 3, 2021 at 12:11 PM David A. Harding <dave@dtrt.org> wrote:

> On Sat, Jul 03, 2021 at 10:35:48AM +0200, Craig Raw wrote:
> > There is a downside to using "h"/"H" from a UX perspective - taking up
> more
> > space
>
> Is this a serious concern of yours?  An apostrophe is 1/2 en; an "h" is
> 1 en; the following descriptor contains three hardened derivations in 149
> characters; assuming the average non-'/h character width is 1.5 en, the
> difference between 207 en and 208.5 en is barely more than half a
> percent.
>
>
> pkh([d34db33f/44h/0h/0h]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/1/*)#ml40v0wf
>
> Here's a direct visual comparison:
> https://gist.github.com/harding/2fbbf2bfdce04c3e4110082f03ae3c80
>
> > appearing as alphanumeric characters similar to the path numbers
>
> First, I think you'd have to be using an awful font to confuse "h" with
> any arabic numeral.  Second, avoiding transcription errors is exactly
> why descriptors now have checksums.
>
> > they make derivation paths and descriptors more difficult to read.
>
> The example descriptor pasted above looks equally (un)readable to me
> whether it uses ' or h.
>
> > Also, although not as important, less efficient when making metal
> > backups.
>
> I think many metal backup schemes are using stamps or punch grids that
> are fixed-width in nature, so there's no difference either way.  (And
> you can argue that h is better since it's part of both the base58check
> and bech32 character sets, so you already need a stamp or a grid row for
> it---but ' is otherwise unused, so a stamp or grid row for it would be
> special).
>
> But even if people are manually etching descriptors into metal, we're
> back to the original point where we're looking at something like a 0.7%
> difference in "efficiency".
>
> By comparison, the Bitcoin Core issue I cited in my earlier post
> contains several examples of actual users needing technical support
> because they tried to use '-containing descriptors in a bourne-style
> shell.  (And I've personally lost time to that class of problems.)  In
> the worst case, a shell-quoting accident can cause loss of money by
> sending bitcoins to the descriptor for a key your hardware signing
> device won't sign for.  I think these problems are much more serious
> than using a tiny bit of extra space in a GUI or on a physical backup
> medium.
>
> -Dave
>

[-- Attachment #2: Type: text/html, Size: 4083 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Daniel Bayerdorffer]

[-- Attachment #1: Type: text/plain, Size: 4507 bytes --]

Hello, 

I just wanted to put my two cents in, on the metal backup aspect. We make the Bitcoin Recovery Tag for a similar purpose. We use a fixed font, so using ' (apostrophe) or H/h are both acceptable. Most metal stamping tools are fixed width fonts. 

You can see a picture here... 
[ https://cyphersafe.io/product/bitcoin-recovery-tag/ | https://cyphersafe.io/product/bitcoin-recovery-tag/ ] 

Thanks, 
Daniel 

-- 
Daniel Bayerdorffer, VP danielb@numberall.com 
Numberall Stamp & Tool Co., Inc. www.numberall.com 
Reuleaux Models www.reuleauxmodels.com 
CypherSafe www.cyphersafe.io 
PO BOX 187, Sangerville, ME 04479 USA 
TEL: 207-876-3541 FAX: 207-876-3566 


From: "Craig Raw via bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org> 
To: "David A. Harding" <dave@dtrt.org> 
Cc: "Bitcoin Protocol Discussion" <bitcoin-dev@lists.linuxfoundation.org> 
Sent: Saturday, July 3, 2021 10:00:51 AM 
Subject: Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors 

It's a consideration, not a serious concern. 
When I made the point around alphanumeric characters being similar to the path numbers, I was actually thinking of the output descriptor appearing in a fixed character width font, which I prefer as more appropriate for displaying hexidecimal values. In this case, the apostrophe provides more whitespace which makes the path easier to parse visually. It's difficult to reduce this to a mathematical argument, as is true for many UX considerations. Your example in fixed width here: [ https://gist.github.com/craigraw/fc98b9031a7e01e1bc5d75a77bdb72e5 | https://gist.github.com/craigraw/fc98b9031a7e01e1bc5d75a77bdb72e5 ] 

That said you make good arguments around the shell quoting and stamps for metal backups, and therefore I agree it is preferable to use the lowercase "h". Thanks for the detailed reply. 

Craig 

On Sat, Jul 3, 2021 at 12:11 PM David A. Harding < [ mailto:dave@dtrt.org | dave@dtrt.org ] > wrote: 


On Sat, Jul 03, 2021 at 10:35:48AM +0200, Craig Raw wrote: 
> There is a downside to using "h"/"H" from a UX perspective - taking up more 
> space 

Is this a serious concern of yours? An apostrophe is 1/2 en; an "h" is 
1 en; the following descriptor contains three hardened derivations in 149 
characters; assuming the average non-'/h character width is 1.5 en, the 
difference between 207 en and 208.5 en is barely more than half a 
percent. 

pkh([d34db33f/44h/0h/0h]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/1/*)#ml40v0wf 

Here's a direct visual comparison: [ https://gist.github.com/harding/2fbbf2bfdce04c3e4110082f03ae3c80 | https://gist.github.com/harding/2fbbf2bfdce04c3e4110082f03ae3c80 ] 

> appearing as alphanumeric characters similar to the path numbers 

First, I think you'd have to be using an awful font to confuse "h" with 
any arabic numeral. Second, avoiding transcription errors is exactly 
why descriptors now have checksums. 

> they make derivation paths and descriptors more difficult to read. 

The example descriptor pasted above looks equally (un)readable to me 
whether it uses ' or h. 

> Also, although not as important, less efficient when making metal 
> backups. 

I think many metal backup schemes are using stamps or punch grids that 
are fixed-width in nature, so there's no difference either way. (And 
you can argue that h is better since it's part of both the base58check 
and bech32 character sets, so you already need a stamp or a grid row for 
it---but ' is otherwise unused, so a stamp or grid row for it would be 
special). 

But even if people are manually etching descriptors into metal, we're 
back to the original point where we're looking at something like a 0.7% 
difference in "efficiency". 

By comparison, the Bitcoin Core issue I cited in my earlier post 
contains several examples of actual users needing technical support 
because they tried to use '-containing descriptors in a bourne-style 
shell. (And I've personally lost time to that class of problems.) In 
the worst case, a shell-quoting accident can cause loss of money by 
sending bitcoins to the descriptor for a key your hardware signing 
device won't sign for. I think these problems are much more serious 
than using a tiny bit of extra space in a GUI or on a physical backup 
medium. 

-Dave 




_______________________________________________ 
bitcoin-dev mailing list 
bitcoin-dev@lists.linuxfoundation.org 
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev 

[-- Attachment #2: Type: text/html, Size: 6017 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Sjors Provoost]

[-- Attachment #1: Type: text/plain, Size: 34677 bytes --]

Thanks for writing this up!

I think your modular BIP approach makes sense. (the abstract should mention this too)

Contents look good to me, modulo missing test vectors. I also suggest dropping combo(), see below.


Regarding the use of h vs ', especially since they result in a different checksum, and equality is more tedious to verify, should we just pick one and recommend that software normalises to that?

For bip-descriptors-segwit, regardless of what Bitcoin Core does, is any hex encoded script allowed for wsh()? If so is it mandatory and/or allowed to use raw() as a sub descriptor?

Conversely, its BIP says: "The <tt>raw(HEX)</tt> expression can only be used as a top level descriptor". That answers the above, but not the why.

In the backwards compatibility section it may be worth pointing out that descriptors are also used by:
* Specter since at least v1.2.2: https://github.com/cryptoadvance/specter-desktop/releases/tag/v1.2.2
* Coldcard since 2.0.1: https://github.com/Coldcard/firmware/commit/af00f8778947664f2d74f19879b98f7925feb327
* HWI since 1.0.3: https://github.com/bitcoin-core/HWI/releases/tag/1.0.3

None of these support the tr(), raw() and addr() descriptors afaik. HWI doesn't implement (sorted_)multi.

Does anyone actually use combo? It seems useless, because even with the help of BIP 88 there's no way to compress all three in a single descriptor, since BIP 44/49/84 each have a different derivation. Afaik Bitcoin Core doesn't really use them either. And for future wallet migration, we might as well make separate descriptors for each key type.

One thing on my wish list - for this BIP, BIP 88 (Hierarchical Deterministic Path Templates) or yet another one - is to include a birth date (minimum block height). E.g. tr([m/86'/0'/0']xpub.../{0-1}/*)>709631

And then of course there's the gap limit. Perhaps we just need a "metadata" format to go along with descriptors to track the birth data, gap limit and anything else you need (nonce collection for musig2 setup?). E.g. a simple dictionary: tr([m/86'/0'/0']xpub.../{0-1}/*){dob:709631,gap:1000}

- Sjors


> Op 29 jun. 2021, om 23:14 heeft Andrew Chow via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> het volgende geschreven:
> 
> Hi All,
> 
> I've been working on formalizing the Output Script Descriptors that have
> been available in Bitcoin Core for a while into BIPs. Since descriptors
> are modular and have optional components, I've decided to split it into
> 7 BIPs, rather than a single one. The first describes descriptors in
> general and does not specify any particular descriptor. However it does
> describe the general operation, key expressions (including derivation
> paths and key origin info), and the descriptor checksum. The following 6
> BIPs specify the actual descriptors themselves. These are non-segwit
> descriptor (pk, pkh, sh), segwit descriptors (wpkh, wsh), multisig
> descriptors (multi, sortedmulti), the taproot descriptor (tr), the combo
> descriptor, and opaque descriptors (raw, addr). This separation is so
> that implementors can choose to not implement some descriptors and still
> say which descriptors they support without being too difficult to
> understand.
> 
> The text of all of the documents are below, and they can also be found
> on github:https://github.com/achow101/bips/tree/descriptors/
> 
> Thanks,
> Andrew Chow
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-general
>   Layer: Applications
>   Title: Output Script Descriptors General Operation
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-general
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> Output Script Descriptors are a simple language which can be used to
> describe collections ofoutput scripts.
> There can be many different descriptor fragments and functions.
> This document describes the general syntax for descriptors, descriptor
> checksums, and common expressions.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> Bitcoin wallets traditionally have stored a set of keys which are later
> serialized and mutated to produce the output scripts that the wallet
> watches and the addresses it provides to users.
> Typically backups have consisted of solely the private keys, nowadays
> primarily in the form of BIP 39 mnemonics.
> However this backup solution is insuffient, especially since the
> introduction of Segregated Witness which added new output types.
> Given just the private keys, it is not possible for restored wallets to
> know which kinds of output scripts and addresses to produce.
> This has lead to incompatibilities between wallets when restoring a
> backup or exporting data for a watch only wallet.
> 
> Further complicating matters are BIP 32 derivation paths.
> Although BIPs 44, 49, and 84 have specified standard BIP 32 derivation
> paths for different output scripts and addresses, not all wallets
> support them nor use those derivation paths.
> The lack of derivation path information in these backups and exports
> leads to further incompatibilities between wallets.
> 
> Current solutions to these issues have not been generic and can be
> viewed as being layer violations.
> Solutions such as introducing different version bytes for extended key
> serialization both are a layer violation (key derivation should be
> separate from script type meaning) and specific only to a particular
> derivation path and script type.
> 
> Output Script Descriptors introduces a generic solution to these issues.
> Script types are specified explicitly through the use of Script Expressions.
> Key derivation paths are specified explicitly in Key Expressions.
> These allow for creating wallet backups and exports which specify the
> exact scripts, subscripts (redeemScript, witnessScript, etc.), and keys
> to produce.
> With the general structure specified in this BIP, new Script Expressions
> can be introduced as new script types are added.
> Lastly, the use of common terminology and existing standards allow for
> Output Script Descriptors to be engineer readable so that the results
> can be understood at a glance.
> 
> ==Specification==
> 
> Descriptors consist of several types of expressions.
> The top level expression is a <tt>SCRIPT</tt>.
> This expression may be followed by <tt>#CHECKSUM</tt>, where
> <tt>CHECKSUM</tt> is an 8 character alphanumeric descriptor checksum.
> 
> ===Script Expressions===
> 
> Script Expressions (denoted <tt>SCRIPT</tt>) are expressions which
> correspond directly with a Bitcoin script.
> These expressions are written as functions and take arguments.
> Such expressions have a script template which is filled with the
> arguments correspondingly.
> Expressions are written with a human readable identifier string with the
> arguments enclosed with parentheses.
> The identifier string should be alphanumeric and may include underscores.
> 
> The arguments to a script expression are defined by that expression itself.
> They could be a script expression, a key expression, or some other
> expression entirely.
> 
> ===Key Expressions===
> 
> A common expression used as an argument to script expressions are key
> expressions (denoted <tt>KEY</tt>).
> These represent a public or private key and, optionally, information
> about the origin of that key.
> Key expressions can only be used as arguments to script expressions.
> 
> Key expressions consist of:
> * Optionally, key origin information, consisting of:
> ** An open bracket <tt>[</tt>
> ** Exactly 8 hex characters for the fingerprint of the key where the
> derivation starts (see BIP 32 for details)
> ** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt>  path
> elements to indicate the unhardened or hardened derivation steps between
> the fingerprint and the key that follows.
> ** A closing bracket <tt>]</tt>
> * Followed by the actual key, which is either:
> ** A hex encoded public key, which depending the script expression, may
> be either:
> *** 66 hex character string beginning with <tt>02</tt> or <tt>03</tt>
> representing a compressed public key
> *** 130 hex character string beginning with <tt>04</tt> representing an
> uncompressed public key
> *** 64 hex character string representing an x-only public key
> ** A [[https://en.bitcoin.it/wiki/Wallet_import_format|WIF]] encoded
> private key
> ** <tt>xpub</tt> encoded extended public key or <tt>xprv</tt> encoded
> extended private key (as defined in BIP 32)
> *** Followed by zero or more <tt>/NUM</tt> or <tt>/NUM'</tt> path
> elements indicating BIP 32 derivation steps to be taken after the given
> extended key.
> *** Optionally followed by a single <tt>/*</tt> or <tt>/*'</tt> final
> step to denote all direct unhardened or hardened children.
> 
> If the <tt>KEY</tt> is a BIP 32 extended key, before output scripts can
> be created, child keys must be derived using the derivation information
> that follows the extended key.
> When the final step is <tt>/*</tt> or <tt>/*'</tt>, an output script
> will be produced for every child key index.
> The derived key must be serialized as a compressed public key.
> 
> In the above specification, the hardened indicator <tt>'</tt> may be
> replaced with alternative hardnened indicators of <tt>h</tt> or <tt>H</tt>.
> 
> ===Character Set===
> 
> The expressions used in descriptors must only contain characters within
> this character set so that the descriptor checksum will work.
> 
> The allowed characters are:
> <pre>
> 0123456789()[],'/*abcdefgh@:$%{}
> IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~
> ijklmnopqrstuvwxyzABCDEFGH`#"\<space>
> </pre>
> Note that <tt><space></tt> on the last line is a space character.
> 
> This character set is written as 3 groups of 32 characters in this
> specific order so that the checksum below can identify more errors.
> The first group are the most common "unprotected" characters (i.e.
> things such as hex and keypaths that do not already have their own
> checksums).
> Case errors cause an offset that is a multiple of 32 while as many
> alphabetic characters are in the same group while following the previous
> restrictions.
> 
> ===Checksum===
> 
> Follwing the top level script expression is a single octothorpe
> (<tt>#</tt>) followed by the 8 character checksum.
> The checksum is an error correcting checksum similar to bech32.
> 
> The checksum has the following properties:
> * Mistakes in a descriptor string are measured in "symbol errors". The
> higher the number of symbol errors, the harder it is to detect:
> ** An error substituting a character from
> <tt>0123456789()[],'/*abcdefgh@:$%{}</tt> for another in that set always
> counts as 1 symbol error.
> *** Note that hex encoded keys are covered by these characters. Extended
> keys (<tt>xpub</tt> and <tt>xprv</tt>) use other characters too, but
> also have their own checksum mechansim.
> *** <tt>SCRIPT</tt> expression function names use other characters, but
> mistakes in these would generally result in an unparsable descriptor.
> ** A case error always counts as 1 symbol error.
> ** Any other 1 character substitution error counts as 1 or 2 symbol errors.
> * Any 1 symbol error is always detected.
> * Any 2 or 3 symbol error in a descriptor of up to 49154 characters is
> always detected.
> * Any 4 symbol error in a descriptor of up to 507 characters is always
> detected.
> * Any 5 symbol error in a descriptor of up to 77 characters is always
> detected.
> * Is optimized to minimize the chance of a 5 symbol error in a
> descriptor up to 387 characters is undetected
> * Random errors have a chance of 1 in 2<super>40</super> of being
> undetected.
> 
> The checksum itself uses the same character set as bech32:
> <tt>qpzry9x8gf2tvdw0s3jn54khce6mua7l</tt>
> 
> Valid descriptor strings with a checksum must pass the criteria for
> validity specified by the Python3 code snippet below.
> The function <tt>descsum_check</tt> must return true when its argument
> <tt>s</tt> is a descriptor consisting in the form <tt>SCRIPT#CHECKSUM</tt>.
> 
> <pre>
> INPUT_CHARSET =
> "0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\
> "
> CHECKSUM_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"
> GENERATOR = [0xf5dee51989, 0xa9fdca3312, 0x1bab10e32d, 0x3706b1677a,
> 0x644d626ffd]
> 
> def descsum_polymod(symbols):
>     """Internal function that computes the descriptor checksum."""
>     chk = 1
>     for value in symbols:
>         top = chk >> 35
>         chk = (chk & 0x7ffffffff) << 5 ^ value
>         for i in range(5):
>             chk ^= GENERATOR[i] if ((top >> i) & 1) else 0
>     return chk
> 
> def descsum_expand(s):
>     """Internal function that does the character to symbol expansion"""
>     groups = []
>     symbols = []
>     for c in s:
>         if not c in INPUT_CHARSET:
>             return None
>         v = INPUT_CHARSET.find(c)
>         symbols.append(v & 31)
>         groups.append(v >> 5)
>         if len(groups) == 3:
>             symbols.append(groups[0] * 9 + groups[1] * 3 + groups[2])
>             groups = []
>     if len(groups) == 1:
>         symbols.append(groups[0])
>     elif len(groups) == 2:
>         symbols.append(groups[0] * 3 + groups[1])
>     return symbols
> 
> def descsum_check(s):
>     """Verify that the checksum is correct in a descriptor"""
>     if s[-9] != '#':
>         return False
>     if not all(x in CHECKSUM_CHARSET for x in s[-8:]):
>         return False
>     symbols = descsum_expand(s[:-9]) + [CHECKSUM_CHARSET.find(x) for x
> in s[-8:]]
>     return descsum_polymod(symbols) == 1
> </pre>
> 
> This implements a BCH code that has the properties described above.
> The entire descriptor string is first processed into an array of symbols.
> The symbol for each character is its position within its group.
> After every 3rd symbol, a 4th symbol is inserted which represents the
> group numbers combined together.
> This means that a change that only affects the position within a group,
> or only a group number change, will only affect a single symbol.
> 
> To construct a valid checksum given a script expression, the code below
> can be used:
> 
> <pre>
> def descsum_create(s):
>     """Add a checksum to a descriptor without"""
>     symbols = descsum_expand(s) + [0, 0, 0, 0, 0, 0, 0, 0]
>     checksum = descsum_polymod(symbols) ^ 1
>     return s + '#' + ''.join(CHECKSUM_CHARSET[(checksum >> (5 * (7 -
> i))) & 31] for i in range(8))
> 
> </pre>
> 
> ==Backwards Compatibility==
> 
> Output script descriptors are an entirely new language which is not
> compatible with any existing software.
> However many components of the expressions reuse encodings and
> serializations defined by previous BIPs.
> 
> Output script descriptors are designed for future extension with further
> fragment types and new script expressions.
> These will be specified in additional BIPs.
> 
> ==Reference Implemntation==
> 
> Descriptors have been implemented in Bitcoin Core since version 0.17.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-segwit
>   Layer: Applications
>   Title: segwit Output Script Descriptors
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-segwit
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>wpkh()</tt>, and <tt>wsh()</tt> output
> script descriptors.
> <tt>wpkh()</tt> descriptors take a key and produces a P2WPKH output script.
> <tt>wsh()</tt> descriptors take a script and produces a P2WSH output script.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> Segregated Witness added 2 additional standard output script formats:
> P2WPKH and P2WSH.
> These expressions allow specifying those formats as a descriptor.
> 
> ==Specification==
> 
> Two new script expressions are defined: <tt>wpkh()</tt>, and <tt>wsh()</tt>.
> 
> ===<tt>wpkh()</tt>===
> 
> The <tt>wpkh(KEY)</tt> expression can be used as a top level expression,
> or inside of a <tt>sh()</tt> descriptor.
> It takes a single key expression as an argument and produces a P2WPKH
> output script.
> Only keys which are/has compressed public keys can be contained in a
> <tt>wpkh()</tt> expression.
> 
> The output script produced is:
> <pre>
> OP_0 <KEY_hash160>
> </pre>
> 
> ===<tt>wsh()</tt>===
> 
> The <tt>wsh(SCRIPT)</tt> expression can be used as a top level
> expression, or inside of a <tt>sh()</tt> descriptor.
> It takes a single script expression as an argument and produces a P2WSH
> output script.
> <tt>wsh()</tt> expressions also create a witnessScript which is required
> in order to spend outputs which use its output script.
> This redeemScript is the output script produced by the <tt>SCRIPT</tt>
> argument to <tt>wsh()</tt>.
> Any key expression found in any script expression contained by a
> <tt>wsh()</tt> expression must only produce compresed public keys.
> 
> The output script produced is:
> <pre>
> OP_0 <SCRIPT_sha256>
> </pre>
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors use the format and
> general operation specified in
> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As these are a wholly new descriptors, they are not compatible with any
> implementation.
> However the scripts produced are standard scripts so existing software
> are likely to be familiar with them.
> 
> ==Reference Implemntation==
> 
> <tt>wpkh()</tt>, and <tt>wsh()</tt> descriptors have been implemented in
> Bitcoin Core since version 0.17.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-non-segwit
>   Layer: Applications
>   Title: Non-segwit Output Script Descriptors
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-non-segwit
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt>
> output script descriptors.
> <tt>pk()</tt> descriptors take a key and produces a P2PK output script.
> <tt>pkh()</tt> descriptors take a key and produces a P2PKH output script.
> <tt>sh()</tt> descriptors take a script and produces a P2SH output script.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> Prior to the activation of Segregated Witness, there were 3 main
> standard output script formats: P2PK, P2PKH, and P2SH.
> These expressions allow specifying those formats as a descriptor.
> 
> ==Specification==
> 
> Three new script expressions are defined: <tt>pk()</tt>, <tt>pkh()</tt>,
> and <tt>sh()</tt>.
> 
> ===<tt>pk()</tt>===
> 
> The <tt>pk(KEY)</tt> expression can be used in any context or level of a
> descriptor.
> It takes a single key expression as an argument and produces a P2PK
> output script.
> Depending on the higher level descriptors, there may be restrictions on
> the type of public keys that can be included.
> Such restrictions will be specified by those descriptors.
> 
> The output script produced is:
> <pre>
> <KEY> OP_CHECKSIG
> </pre>
> 
> ===<tt>pkh()</tt>===
> 
> The <tt>pkh(KEY)</tt> expression can be used as a top level expression,
> or inside of either a <tt>sh()</tt> or <tt>wsh()</tt> descriptor.
> It takes a single key expression as an argument and produces a P2PKH
> output script.
> Depending on the higher level descriptors, there may be restrictions on
> the type of public keys that can be included.
> Such restrictions will be specified by those descriptors.
> 
> The output script produced is:
> <pre>
> OP_DUP OP_HASH160 <KEY_hash160> OP_EQUALVERIFY OP_CHECKSIG
> </pre>
> 
> ===<tt>sh()</tt>===
> 
> The <tt>sh(SCRIPT)</tt> expression can only be used as a top level
> expression.
> It takes a single script expression as an argument and produces a P2SH
> output script.
> <tt>sh()</tt> expressions also create a redeemScript which is required
> in order to spend outputs which use its output script.
> This redeemScript is the output script produced by the <tt>SCRIPT</tt>
> argument to <tt>sh()</tt>.
> 
> The output script produced is:
> <pre>
> OP_HASH160 <SCRIPT_hash160> OP_EQUAL
> </pre>
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors use the
> format and general operation specified in
> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As these are a wholly new descriptors, they are not compatible with any
> implementation.
> However the scripts produced are standard scripts so existing software
> are likely to be familiar with them.
> 
> ==Reference Implemntation==
> 
> <tt>pk()</tt>, <tt>pkh()</tt>, and <tt>sh()</tt> descriptors have been
> implemented in Bitcoin Core since version 0.17.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-tr
>   Layer: Applications
>   Title: tr() Output Script Descriptors
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-tr
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>tr()</tt> output script descriptors.
> <tt>tr()</tt> descriptors take a key and optionally a tree of scripts
> and produces a P2TR output script.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> Taproot added one additional standard output script format: P2TR.
> These expressions allow specifying those formats as a descriptor.
> 
> ==Specification==
> 
> A new script expressions are defined: <tt>tr()</tt>.
> A new expression is defined: Tree Expressions
> 
> ===Tree Expression===
> 
> A Tree Expression (denoted <tt>TREE</tt>) is an expression which
> represents a tree of scripts.
> The way the tree is represented in an output script is dependent on the
> higher level expressions.
> 
> A Tree Expression is:
> * Any Script Expression that is allowed at the level this Tree
> Expression is in.
> * A pair of Tree Expressions consisting of:
> ** An open brace <tt>{</tt>
> ** A Tree Expression
> ** A comma <tt>,</tt>
> ** A Tree Expression
> ** A closing brance <tt>}</tt>
> 
> ===<tt>tr()</tt>===
> 
> The <tt>tr(KEY)</tt> or <tt>tr(KEY, TREE)</tt> expression can only be
> used as a top level expression.
> All key expressions under any <tt>tr()</tt> expression must create
> x-only public keys.
> 
> <tt>tr(KEY</tt> takes a single key expression as an argument and
> produces a P2TR output script which does not have a script path.
> The keys produced by the key expression are used as the internal key as
> specified by [[bip-0341.mediawiki#cite_ref-22-0|BIP 341]].
> Specifically, "If the spending conditions do not require a script path,
> the output key should commit to an unspendable script path instead of
> having no script path.
> This can be achieved by computing the output key point as ''Q = P +
> int(hash<sub>TapTweak</sub>(bytes(P)))G''."
> 
> <pre>
> internal_key:       lift_x(KEY)
> 32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)))G
> scriptPubKey:       OP_1 <32_byte_output_key>
> </pre>
> 
> <tt>tr(KEY, TREE)</tt> takes a key expression as the first argument, and
> a tree expression as the second argument and produces a P2TR output
> script which has a script path.
> The keys produced by the first key expression are used as the internal
> key as specified by
> [[bip-0341.mediawiki#Constructing_and_spending_Taproot_outputs|BIP 341]].
> The Tree expression becomes the Taproot script tree as described in BIP 341.
> A merkle root is computed from this tree and combined with the internal
> key to create the Taproot output key.
> 
> <pre>
> internal_key:       lift_x(KEY)
> merkle_root:        HashTapBranch(TREE)
> 32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)
> || merkle_root))G
> scriptPubKey:       OP_1 <32_byte_output_key>
> </pre>
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>tr()</tt> descriptors use the format and general operation specified
> in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As these are a wholly new descriptors, they are not compatible with any
> implementation.
> However the scripts produced are standard scripts so existing software
> are likely to be familiar with them.
> 
> Tree Expressions are largely incompatible with existing script
> expressions due to the restrictions in those expressions.
> As of 2021-06-27, the only allowed script expression that can be used in
> a tree expression is <tt>pk()</tt>.
> However there will be future BIPs that specify script expressions that
> can be used in tree expressions.
> 
> ==Reference Implemntation==
> 
> <tt>tr()</tt> descriptors have been implemented in Bitcoin Core since
> version 22.0.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-multi
>   Layer: Applications
>   Title: Multisig Output Script Descriptors
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-multi
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>multi()</tt>, and <tt>sortedmulti()</tt>
> output script descriptors.
> Both functions take a threshold and one or more public keys and produce
> a multisig output script.
> <tt>multi()</tt> specifies the public keys in the output script in the
> order given in the descriptor while <tt>sortedmulti()</tt> sorts the
> public keys lexicographically when the output script is produced.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> The most common complex script used in Bitcoin is a threshold multisig.
> These expressions allow specifying multisig scripts as a descriptor.
> 
> ==Specification==
> 
> Two new script expressions are defined: <tt>multi()</tt>, and
> <tt>sortedmulti()</tt>.
> Both expressions produce the scripts of the same template and take the
> same arguments.
> They are written as <tt>multi(k,KEY_1,KEY_2,...,KEY_n)</tt>.
> <tt>k</tt> is the threshold - the number of keys that must sign the
> input for the script to be valid.
> <tt>KEY_1,KEY_2,...,KEY_n</tt> are the key expressions for the multisig.
> <tt>k</tt> must be less than or equal to <tt>n<tt>.
> 
> <tt>multi()</tt> and <tt>sortedmulti()</tt> expressions can be used as a
> top level expression, or inside of either a <tt>sh()</tt> or
> <tt>wsh()</tt> descriptor.
> Depending on the higher level descriptors, there may be restrictions on
> the type of public keys that can be included.
> 
> Depending on the higher level descriptors, there are also restrictions
> on the number of keys that can be present, i.e. the maximum value of
> <tt>n</tt>.
> When used at the top level, there can only be at most 3 keys.
> When used inside of a <tt>sh()</tt> expression, there can only be most
> 15 compressed public keys (this is limited by the P2SH script limit).
> Otherwise the maximum number of keys is 20.
> 
> The output script produced also depends on the value of <tt>k</tt>. If
> <tt>k</tt> is less than or equal to 16:
> <pre>
> OP_k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
> </pre>
> 
> if <tt>k</tt> is greater than 16:
> <pre>
> k KEY_1 KEY_2 ... KEY_n OP_CHECKMULTISIG
> </pre>
> 
> ===<tt>sortedmulti()</tt>===
> 
> The only change for <tt>sortedmulti()</tt> is that the keys are sorted
> lexicographically prior to the creation of the output script.
> This sorting is on the keys that are to be put into the output script,
> i.e. after all extended keys are derived.
> 
> ===Multiple Extended Keys</tt>===
> 
> When one or more the key expressions in a <tt>multi()</tt> or
> <tt>sortedmulti()</tt> expression are extended keys, the derived keys
> use the same child index.
> This changes the keys in lockstep and allows for output scripts to be
> indexed in the same way that the derived keys are indexed.
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>multi()</tt>, and <tt>sortedmulti()</tt> descriptors use the format
> and general operation specified in
> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As these are a wholly new descriptors, they are not compatible with any
> implementation.
> However the scripts produced are standard scripts so existing software
> are likely to be familiar with them.
> 
> ==Reference Implemntation==
> 
> <tt>multi()</tt>, and <tt>multi()</tt> descriptors have been implemented
> in Bitcoin Core since version 0.17.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-combo
>   Layer: Applications
>   Title: combo() Output Script Descriptors
>   Author: Pieter Wuille <pieter@wuille.net>
>           Andrew Chow <andrew@achow101.com>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-combo
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>combo()</tt> output script descriptors.
> These take a key and produce P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH output
> scripts if applicable to the key.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> In order to make the transition from traditional key based wallets to
> descriptor based wallets easier, it is useful to be able to take a key
> and produce the scripts which have traditionally been produced by wallet
> software.
> 
> ==Specification==
> 
> A new top level script expression is defined: <tt>combo(KEY)</tt>.
> This expression can only be used as a top level expression.
> It takes a single key expression as an argument and produces either 2 or
> 4 output scripts, depending on the key.
> A <tt>combo()</tt> expression always produces a P2PK and P2PKH script,
> the same as putting the key in both a <tt>pk()</tt> and a <tt>pkh()</tt>
> expression.
> If the key is/has a compressed public key, then P2WPKH and P2SH-P2WPKH
> scripts are also produced, the same as putting the key in both a
> <tt>wpkh()</tt> and <tt>sh(wpkh())</tt> expression.
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>combo()</tt> descriptors use the format and general operation
> specified in [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As this is a wholly new descriptor, it is not compatible with any
> implementation.
> However the scripts produced are standard scripts so existing software
> are likely to be familiar with them.
> 
> ==Reference Implemntation==
> 
> <tt>combo</tt> descriptors have been implemented in Bitcoin Core since
> version 0.17.
> 
> ---
> 
> <pre>
>   BIP: bip-descriptors-encap
>   Layer: Applications
>   Title: raw() and addr() Output Script Descriptors
>   Author: Andrew Chow <andrew@achow101.com>
>           Pieter Wuille <pieter@wuille.net>
>   Comments-Summary: No comments yet.
>   Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-raw
>   Status: Draft
>   Type: Informational
>   Created: 2021-06-27
>   License: BSD-2-Clause
> </pre>
> 
> ==Abstract==
> 
> This document specifies <tt>raw()</tt> and <tt>addr()</tt> output script
> descriptors.
> <tt>raw()</tt> encapsulates a raw script as a descriptor.
> <tt>addr()</tt> encapsulates an address as a descriptor.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> In order to make descriptors maximally compatible with scripts in use
> today, it is useful to be able to wrap any arbitrary output script or an
> address into a descriptor.
> 
> ==Specification==
> 
> Two new script expressions are defined: <tt>raw()</tt> and <tt>addr()</tt>.
> 
> ===<tt>raw()</tt>===
> 
> The <tt>raw(HEX)</tt> expression can only be used as a top level descriptor.
> As the argument, it takes a hex string representing a Bitcoin script.
> The output script produced by this descriptor is the script represented
> by <tt>HEX</tt>.
> 
> ===<tt>addr()</tt>===
> 
> The <tt>addr(ADDR)</tt> expression can only be used as a top level
> descriptor.
> It takes an address as its single argument.
> The output script produced by this descriptor is the output script
> produced by the address <tt>ADDR</tt>.
> 
> ==Test Vectors==
> 
> TBD
> 
> ==Backwards Compatibility==
> 
> <tt>raw()</tt> and <tt>addr()</tt> descriptors use the format and
> general operation specified in
> [[bip-descriptor-general.mediawiki|bip-descriptor-general]].
> As this is a wholly new descriptor, it is not compatible with any
> implementation.
> The reuse of existing Bitcoin addresses allows for this to be more
> easily implemented.
> 
> ==Reference Implemntation==
> 
> <tt>raw()</tt> and <tt>addr</tt> descriptors have been implemented in
> Bitcoin Core since version 0.17.
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Christopher Allen]

[-- Attachment #1: Type: text/plain, Size: 2280 bytes --]

On Thu, Aug 5, 2021 at 8:07 AM Sjors Provoost via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> One thing on my wish list - for this BIP, BIP 88 (Hierarchical
> Deterministic Path Templates) or yet another one - is to include a birth
> date (minimum block height). E.g. tr([m/86'/0'/0']xpub.../{0-1}/*)>709631
>
> And then of course there's the gap limit. Perhaps we just need a
> "metadata" format to go along with descriptors to track the birth data, gap
> limit and anything else you need (nonce collection for musig2 setup?). E.g.
> a simple dictionary: tr([m/86'/0'/0']xpub.../{0-1}/*){dob:709631,gap:1000}
>

The UR standards we use in Airgap Wallet Community for interoperability
(currently used by a number of recent wallets for airgap PSBT via animated
QR) leverages CBOR's tagging capability, and thus explicitly supports
metadata. In particular the spec reference code support optional seed
birthdate as some wallet vendors really wanted that metadata.

It would be trivial to support it for hd-keys, and if gap is important, we
could also easily add this to the hd-keys spec as well. That is part of the
reason why use CBOR for the underlying binary encoding is tagging lets
us add important metadata. The UR standards themselves adds to binary CBOR
encoding a very efficient transport via QRs or URLs that leverage native QR
compression.

See:
* Video: [Blockchain Commons Technology Overview](
https://www.youtube.com/watch?v=RYgOFSdUqWY)
* Articles: [URs: An Overview](Docs/ur-1-overview.md)
  * [A Guide to Using URs for Key Material](Docs/ur-2-keys.md)
  *[A Guide to Using URs for SSKRs](Docs/ur-3-sskrs.md)**
  * [A Guide to Using UR Request & Response](Docs/ur-99-request-response.md)
* Specs:
  * [Research 2020-05 - Uniform Resources (UR): Encoding Structured Binary
Data for Transport in URIs and QR Codes](
https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-005-ur.md
)
  * [Reserach 2020 - UR Type Definition for Hierarchical Deterministic (HD)
Keys](
https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-007-hdkey.md
)

If you have questions about these, drop by the Airgapped Wallet Community
on GitHub at https://github.com/BlockchainCommons/Airgapped-Wallet-Community

-- Christopher Allen

[-- Attachment #2: Type: text/html, Size: 3170 bytes --]