[bitcoin-dev] New PSBT version proposal

[bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi All,

I would like to propose a new PSBT version that addresses a few 
deficiencies in the current PSBT v0. As this will be backwards 
incompatible, a new PSBT version will be used, v1.

The primary change is to truly have all input and output data for each 
in their respective maps. Instead of having to parse an unsigned 
transaction and lookup some data from there, and other data from the 
correct map, all of the data for an input will be contained in its map. 
Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. 
Thus I propose that the following fields be added:

Global:
* PSBT_GLOBAL_TX_VERSION = 0x02
   * Key: empty
   * Value: 32-bit little endian unsigned integer for the transaction 
version number. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
   * Key: empty
   * Value: 32 bit little endian unsigned integer for the preferred 
transaction lock time. Must be omitted in PSBT v0. May be provided in 
PSBT v1, assumed to be 0 if not provided.
* PSBT_GLOBAL_INPUT_COUNT = 0x04
   * Key: empty
   * Value: Compact size unsigned integer. Number of inputs in this 
PSBT. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_OUTPUT_COUNT = 0x05
   * Key: empty
   * Value: Compact size unsigned integer. Number of outputs in this 
PSBT. Must be provided in PSBT v1 and omitted in v0.

Input:
* PSBT_IN_PREVIOUS_TXID = 0x0e
   * Key: empty
   * Value: 32 byte txid of the previous transaction whose output at 
PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and 
omitted in v0.
* PSBT_IN_OUTPUT_INDEX = 0x0f
   * Key: empty
   * Value: 32 bit little endian integer for the index of the output 
being spent. Must be provided in PSBT v1 and omitted in v0.
* PSBT_IN_SEQUENCE = 0x0f
   * Key: empty
   * Value: 32 bit unsigned little endian integer for the sequence 
number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed 
to be max sequence (0xffffffff) if not provided.
* PSBT_IN_REQUIRED_LOCKTIME = 0x10
   * Key: empty
   * Value: 32 bit unsigned little endian integer for the lock time that 
this input requires. Must be omitted in PSBT v0. May be provided in PSBT 
v1, assumed to be 0 if not provided.

Output:
* PSBT_OUT_VALUE = 0x03
   * Key: empty
   * Value: 64-bit unsigned little endian integer for the output's 
amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
* PSBT_OUT_OUTPUT_SCRIPT = 0x04
   * Key: empty
   * Value: The script for this output. Otherwise known as the 
scriptPubKey. Must be provided in PSBT v1 and omitted in v0.

This change allows for PSBT to be used in the construction of 
transactions. With these new fields, inputs and outputs can be added as 
needed. One caveat is that there is no longer a unique transaction 
identifier so more care must be taken when combining PSBTs. 
Additionally, adding new inputs and outputs must be done such that 
signatures are not invalidated. This may be harder to specify.

An important thing to note in this proposal are the fields 
PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin 
transaction only has a single locktime yet a PSBT may have multiple 
locktimes. To choose the locktime for the transaction, finalizers must 
choose the maximum of all of the *_LOCKTIME fields. 
PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those 
involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to 
be set. This field allows finalizers to choose a locktime that is high 
enough for all inputs without needing to understand the scripts 
involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if 
no inputs require a particular locktime.

As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 
needs the version number bump to enforce backwards incompatibility. 
However once the inputs and outputs of a PSBT are decided, a PSBT could 
be "downgraded" back to v0 by creating the unsigned transaction from the 
above fields, and then dropping these new fields.

If the list finds that these changes are reasonable, I will write a PR 
to modify BIP 174 to incorporate them.

Thanks,
Andrew Chow

Re: [bitcoin-dev] New PSBT version proposal

[Sanket Kanjalkar]

[-- Attachment #1: Type: text/plain, Size: 6089 bytes --]

>
> The primary change is to truly have all input and output data for each  in
> their respective maps

1) +1. It would be really great to have a complete map per input/output
that does not require an annoying lookup to a global field.

A Bitcoin transaction only has a single locktime yet a PSBT may have
> multiple lock times.

2) One other thing, the per input timelock also helps in detecting whether
the transaction contains a mix of block-based
timelocks and height based timelocks. Recall that such inputs can't be
spent together under the same nLocktime.

3) Finally, one last thing which I noted while implementing a generic
finalizer for Miniscript is the restriction on sighashType.
From the BIP

> Signatures for this input must use the sighash type, finalizers must fail
> to finalize inputs which have signatures that do not match the specified
> sighash type. Signers who cannot produce signatures with the sighash type
> must not provide a signature.

Is such a restriction necessary? If the purpose is to only suggest signer
which sighashType to use, then I think the finalizer
should not reject those. Along those lines, we can also mark with
suggestions for the type of nlockTime(block vs height) that
should be used. With such suggestions, input parties can decide which
branches in the satisfaction they should prefer and
sign with the corresponding signatures. Note that this purpose is different
from the stated purpose of
PSBT_GLOBAL_PREFERRED_LOCKTIME.

Cheers,
Sanket

On Wed, Dec 9, 2020 at 4:33 PM Andrew Chow via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi All,
>
> I would like to propose a new PSBT version that addresses a few
> deficiencies in the current PSBT v0. As this will be backwards
> incompatible, a new PSBT version will be used, v1.
>
> The primary change is to truly have all input and output data for each
> in their respective maps. Instead of having to parse an unsigned
> transaction and lookup some data from there, and other data from the
> correct map, all of the data for an input will be contained in its map.
> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
> Thus I propose that the following fields be added:
>
> Global:
> * PSBT_GLOBAL_TX_VERSION = 0x02
>    * Key: empty
>    * Value: 32-bit little endian unsigned integer for the transaction
> version number. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>    * Key: empty
>    * Value: 32 bit little endian unsigned integer for the preferred
> transaction lock time. Must be omitted in PSBT v0. May be provided in
> PSBT v1, assumed to be 0 if not provided.
> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>    * Key: empty
>    * Value: Compact size unsigned integer. Number of inputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>    * Key: empty
>    * Value: Compact size unsigned integer. Number of outputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
>
> Input:
> * PSBT_IN_PREVIOUS_TXID = 0x0e
>    * Key: empty
>    * Value: 32 byte txid of the previous transaction whose output at
> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
> omitted in v0.
> * PSBT_IN_OUTPUT_INDEX = 0x0f
>    * Key: empty
>    * Value: 32 bit little endian integer for the index of the output
> being spent. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_IN_SEQUENCE = 0x0f
>    * Key: empty
>    * Value: 32 bit unsigned little endian integer for the sequence
> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
> to be max sequence (0xffffffff) if not provided.
> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>    * Key: empty
>    * Value: 32 bit unsigned little endian integer for the lock time that
> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
> v1, assumed to be 0 if not provided.
>
> Output:
> * PSBT_OUT_VALUE = 0x03
>    * Key: empty
>    * Value: 64-bit unsigned little endian integer for the output's
> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>    * Key: empty
>    * Value: The script for this output. Otherwise known as the
> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>
> This change allows for PSBT to be used in the construction of
> transactions. With these new fields, inputs and outputs can be added as
> needed. One caveat is that there is no longer a unique transaction
> identifier so more care must be taken when combining PSBTs.
> Additionally, adding new inputs and outputs must be done such that
> signatures are not invalidated. This may be harder to specify.
>
> An important thing to note in this proposal are the fields
> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
> transaction only has a single locktime yet a PSBT may have multiple
> locktimes. To choose the locktime for the transaction, finalizers must
> choose the maximum of all of the *_LOCKTIME fields.
> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
> be set. This field allows finalizers to choose a locktime that is high
> enough for all inputs without needing to understand the scripts
> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
> no inputs require a particular locktime.
>
> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
> needs the version number bump to enforce backwards incompatibility.
> However once the inputs and outputs of a PSBT are decided, a PSBT could
> be "downgraded" back to v0 by creating the unsigned transaction from the
> above fields, and then dropping these new fields.
>
> If the list finds that these changes are reasonable, I will write a PR
> to modify BIP 174 to incorporate them.
>
> Thanks,
> Andrew Chow
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 7570 bytes --]

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Poelstra]

[-- Attachment #1: Type: text/plain, Size: 1554 bytes --]

On Wed, Dec 09, 2020 at 10:25:37PM +0000, Andrew Chow via bitcoin-dev wrote:
> Hi All,
> 
> I would like to propose a new PSBT version that addresses a few 
> deficiencies in the current PSBT v0. As this will be backwards 
> incompatible, a new PSBT version will be used, v1.
> 
> The primary change is to truly have all input and output data for each 
> in their respective maps. Instead of having to parse an unsigned 
> transaction and lookup some data from there, and other data from the 
> correct map, all of the data for an input will be contained in its map. 
> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. 
> Thus I propose that the following fields be added:
> 
> Global:
> * PSBT_GLOBAL_TX_VERSION = 0x02
>  ?? * Key: empty
>  ?? * Value: 32-bit little endian unsigned integer for the transaction 
> version number. Must be provided in PSBT v1 and omitted in v0.

All of these changes sound great. It would definitely make working with
PSBTs easier if all data was accessible in the same format, rather than
being split between the global unsigned tx and the main body.

One minor quibble is the version numbering -- you mention "v1" in this
post but set GLOBAL_TX_VERSION to 2. I think we should consistently use
2 everywhere; probably nobody thinks of the existing PSBT as "version 0".

-- 
Andrew Poelstra
Director of Research, Blockstream
Email: apoelstra at wpsoftware.net
Web:   https://www.wpsoftware.net/andrew

The sun is always shining in space
    -Justin Lewis-Webster


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi All,

I have some updates on this after speaking with some people off-list.

Firstly, the version number will be set to 2. In most discussions, this 
proposal was being referred to as PSBT version 2, so it'll be easier and 
clearer to set the version number to 2.

For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field, 
there will be 2 of them, one for a time based lock time, and the other 
for height based. These will be:
* PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
   * Key: empty
   * Value: 32 bit unsigned little endian integer greater than or equal 
to 500000000 representing the minimum Unix timestamp that this input 
requires to be set as the transaction's lock time. Must be omitted in 
PSBTv0, and may be omitted in PSBTv2
* PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
   * Key: empty
   * Value: 32 bit unsigned little endian integer less than 500000000 
representing the minimum block height that this input requires to be set 
as the transaction's lock time. Must be omitted in PSBTv0, and may be 
omitted in PSBTv2.

Having two lock time fields is necessary due to the behavior where all 
inputs must use the same type of lock time (height or time). Thus if an 
input requires a particular type of lock time, it must set the requisite 
field. Any new inputs being added must be able to accommodate all 
existing inputs' lock time type. This means they either must not have a 
lock time specified (i.e. no OP_CLTV involved), or have branches that 
allow the acceptance of either type. If an input has a lock time type 
that is incompatible with the rest of the transaction, it must not be added.

PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback 
option if no input lock time fields are present. If there are input lock 
times, all lock time calculations must ignore it.

Any role which does lock time calculation will first check if there are 
input lock time fields. If there are not, it must then check for a 
PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the 
transaction's lock time. If it does not, the lock time is 0. If there 
are input lock time fields, it must choose the type which does not 
invalidate any inputs. The lock time is then determined to be the 
maximum value of all of the lock time fields for the chosen type.


Additionally, I would like to add a new global field:
* PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
   * Key: empty
   * Value: A single byte as a boolean. 0 for False, 1 for True. All 
other values ore prohibited. Must be omitted for PSBTv0, may be omitted 
in PSBTv2.

PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and 
outputs can be added to the PSBT. This flag may be set to True when 
inputs and outputs are being updated, signed, and finalized. However 
care must be taken when there are existing signatures. If this field is 
omitted or set to False, no further inputs and outputs may be added to 
the PSBT.

Several rules must be followed to ensure that adding additional inputs 
and outputs will not invalidate existing signatures. First, an input or 
output adder must check for any existing signatures in all of the other 
inputs. If there are none, the input or output may be added in any 
position. If there are one or more signatures, each signature's sighash 
type must be examined. Inputs may only be added if all existing 
signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all 
existing signatures use SIGHASH_NONE. If an input has a signature using 
SIGHASH_SINGLE, the same number of inputs and outputs must be added 
before that input and it's corresponding output. For all other sighash 
types (i.e. SIGHASH_ALL and any future sighash types), no inputs or 
outputs may be added to the PSBT. Specific exceptions can be made in the 
future for additional sighash types.

Furthermore, these newly added inputs must follow additional lock time 
rules. Because all signatures, regardless of sighash type, sign the 
transaction lock time, newly added inputs when there are existing 
signatures must have the same type of lock time used in the transaction, 
and must be less than or equal to the transaction lock time. It must not 
cause the transaction lock time to change, otherwise the signatures will 
be invalidated.


Lastly, to uniquely identify transactions for combiners, a txid can be 
computed from the information present in the PSBT. Internally, combiners 
can create an unsigned transaction given the transaction version, the 
input prevouts, the outputs, and the computed locktime. This can then be 
used to calculate a txid and thus used as a way to identify PSBTs. 
Combiners will need to do this for all version 2 PSBTs in order to avoid 
combining distinct transactions.


Andrew Chow

On 12/9/20 5:25 PM, Andrew Chow wrote:
> Hi All,
>
> I would like to propose a new PSBT version that addresses a few
> deficiencies in the current PSBT v0. As this will be backwards
> incompatible, a new PSBT version will be used, v1.
>
> The primary change is to truly have all input and output data for each
> in their respective maps. Instead of having to parse an unsigned
> transaction and lookup some data from there, and other data from the
> correct map, all of the data for an input will be contained in its map.
> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
> Thus I propose that the following fields be added:
>
> Global:
> * PSBT_GLOBAL_TX_VERSION = 0x02
>     * Key: empty
>     * Value: 32-bit little endian unsigned integer for the transaction
> version number. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>     * Key: empty
>     * Value: 32 bit little endian unsigned integer for the preferred
> transaction lock time. Must be omitted in PSBT v0. May be provided in
> PSBT v1, assumed to be 0 if not provided.
> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>     * Key: empty
>     * Value: Compact size unsigned integer. Number of inputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>     * Key: empty
>     * Value: Compact size unsigned integer. Number of outputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
>
> Input:
> * PSBT_IN_PREVIOUS_TXID = 0x0e
>     * Key: empty
>     * Value: 32 byte txid of the previous transaction whose output at
> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
> omitted in v0.
> * PSBT_IN_OUTPUT_INDEX = 0x0f
>     * Key: empty
>     * Value: 32 bit little endian integer for the index of the output
> being spent. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_IN_SEQUENCE = 0x0f
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer for the sequence
> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
> to be max sequence (0xffffffff) if not provided.
> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer for the lock time that
> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
> v1, assumed to be 0 if not provided.
>
> Output:
> * PSBT_OUT_VALUE = 0x03
>     * Key: empty
>     * Value: 64-bit unsigned little endian integer for the output's
> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>     * Key: empty
>     * Value: The script for this output. Otherwise known as the
> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>
> This change allows for PSBT to be used in the construction of
> transactions. With these new fields, inputs and outputs can be added as
> needed. One caveat is that there is no longer a unique transaction
> identifier so more care must be taken when combining PSBTs.
> Additionally, adding new inputs and outputs must be done such that
> signatures are not invalidated. This may be harder to specify.
>
> An important thing to note in this proposal are the fields
> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
> transaction only has a single locktime yet a PSBT may have multiple
> locktimes. To choose the locktime for the transaction, finalizers must
> choose the maximum of all of the *_LOCKTIME fields.
> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
> be set. This field allows finalizers to choose a locktime that is high
> enough for all inputs without needing to understand the scripts
> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
> no inputs require a particular locktime.
>
> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
> needs the version number bump to enforce backwards incompatibility.
> However once the inputs and outputs of a PSBT are decided, a PSBT could
> be "downgraded" back to v0 by creating the unsigned transaction from the
> above fields, and then dropping these new fields.
>
> If the list finds that these changes are reasonable, I will write a PR
> to modify BIP 174 to incorporate them.
>
> Thanks,
> Andrew Chow

Re: [bitcoin-dev] New PSBT version proposal

[fiatjaf]

Hi Andrew.

I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured?

Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases.

Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html

 ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote ----
 > Hi All,
 > 
 > I have some updates on this after speaking with some people off-list.
 > 
 > Firstly, the version number will be set to 2. In most discussions, this 
 > proposal was being referred to as PSBT version 2, so it'll be easier and 
 > clearer to set the version number to 2.
 > 
 > For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field, 
 > there will be 2 of them, one for a time based lock time, and the other 
 > for height based. These will be:
 > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
 >    * Key: empty
 >    * Value: 32 bit unsigned little endian integer greater than or equal 
 > to 500000000 representing the minimum Unix timestamp that this input 
 > requires to be set as the transaction's lock time. Must be omitted in 
 > PSBTv0, and may be omitted in PSBTv2
 > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
 >    * Key: empty
 >    * Value: 32 bit unsigned little endian integer less than 500000000 
 > representing the minimum block height that this input requires to be set 
 > as the transaction's lock time. Must be omitted in PSBTv0, and may be 
 > omitted in PSBTv2.
 > 
 > Having two lock time fields is necessary due to the behavior where all 
 > inputs must use the same type of lock time (height or time). Thus if an 
 > input requires a particular type of lock time, it must set the requisite 
 > field. Any new inputs being added must be able to accommodate all 
 > existing inputs' lock time type. This means they either must not have a 
 > lock time specified (i.e. no OP_CLTV involved), or have branches that 
 > allow the acceptance of either type. If an input has a lock time type 
 > that is incompatible with the rest of the transaction, it must not be added.
 > 
 > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback 
 > option if no input lock time fields are present. If there are input lock 
 > times, all lock time calculations must ignore it.
 > 
 > Any role which does lock time calculation will first check if there are 
 > input lock time fields. If there are not, it must then check for a 
 > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the 
 > transaction's lock time. If it does not, the lock time is 0. If there 
 > are input lock time fields, it must choose the type which does not 
 > invalidate any inputs. The lock time is then determined to be the 
 > maximum value of all of the lock time fields for the chosen type.
 > 
 > 
 > Additionally, I would like to add a new global field:
 > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
 >    * Key: empty
 >    * Value: A single byte as a boolean. 0 for False, 1 for True. All 
 > other values ore prohibited. Must be omitted for PSBTv0, may be omitted 
 > in PSBTv2.
 > 
 > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and 
 > outputs can be added to the PSBT. This flag may be set to True when 
 > inputs and outputs are being updated, signed, and finalized. However 
 > care must be taken when there are existing signatures. If this field is 
 > omitted or set to False, no further inputs and outputs may be added to 
 > the PSBT.
 > 
 > Several rules must be followed to ensure that adding additional inputs 
 > and outputs will not invalidate existing signatures. First, an input or 
 > output adder must check for any existing signatures in all of the other 
 > inputs. If there are none, the input or output may be added in any 
 > position. If there are one or more signatures, each signature's sighash 
 > type must be examined. Inputs may only be added if all existing 
 > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all 
 > existing signatures use SIGHASH_NONE. If an input has a signature using 
 > SIGHASH_SINGLE, the same number of inputs and outputs must be added 
 > before that input and it's corresponding output. For all other sighash 
 > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or 
 > outputs may be added to the PSBT. Specific exceptions can be made in the 
 > future for additional sighash types.
 > 
 > Furthermore, these newly added inputs must follow additional lock time 
 > rules. Because all signatures, regardless of sighash type, sign the 
 > transaction lock time, newly added inputs when there are existing 
 > signatures must have the same type of lock time used in the transaction, 
 > and must be less than or equal to the transaction lock time. It must not 
 > cause the transaction lock time to change, otherwise the signatures will 
 > be invalidated.
 > 
 > 
 > Lastly, to uniquely identify transactions for combiners, a txid can be 
 > computed from the information present in the PSBT. Internally, combiners 
 > can create an unsigned transaction given the transaction version, the 
 > input prevouts, the outputs, and the computed locktime. This can then be 
 > used to calculate a txid and thus used as a way to identify PSBTs. 
 > Combiners will need to do this for all version 2 PSBTs in order to avoid 
 > combining distinct transactions.
 > 
 > 
 > Andrew Chow
 > 
 > On 12/9/20 5:25 PM, Andrew Chow wrote:
 > > Hi All,
 > >
 > > I would like to propose a new PSBT version that addresses a few
 > > deficiencies in the current PSBT v0. As this will be backwards
 > > incompatible, a new PSBT version will be used, v1.
 > >
 > > The primary change is to truly have all input and output data for each
 > > in their respective maps. Instead of having to parse an unsigned
 > > transaction and lookup some data from there, and other data from the
 > > correct map, all of the data for an input will be contained in its map.
 > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
 > > Thus I propose that the following fields be added:
 > >
 > > Global:
 > > * PSBT_GLOBAL_TX_VERSION = 0x02
 > >     * Key: empty
 > >     * Value: 32-bit little endian unsigned integer for the transaction
 > > version number. Must be provided in PSBT v1 and omitted in v0.
 > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
 > >     * Key: empty
 > >     * Value: 32 bit little endian unsigned integer for the preferred
 > > transaction lock time. Must be omitted in PSBT v0. May be provided in
 > > PSBT v1, assumed to be 0 if not provided.
 > > * PSBT_GLOBAL_INPUT_COUNT = 0x04
 > >     * Key: empty
 > >     * Value: Compact size unsigned integer. Number of inputs in this
 > > PSBT. Must be provided in PSBT v1 and omitted in v0.
 > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
 > >     * Key: empty
 > >     * Value: Compact size unsigned integer. Number of outputs in this
 > > PSBT. Must be provided in PSBT v1 and omitted in v0.
 > >
 > > Input:
 > > * PSBT_IN_PREVIOUS_TXID = 0x0e
 > >     * Key: empty
 > >     * Value: 32 byte txid of the previous transaction whose output at
 > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
 > > omitted in v0.
 > > * PSBT_IN_OUTPUT_INDEX = 0x0f
 > >     * Key: empty
 > >     * Value: 32 bit little endian integer for the index of the output
 > > being spent. Must be provided in PSBT v1 and omitted in v0.
 > > * PSBT_IN_SEQUENCE = 0x0f
 > >     * Key: empty
 > >     * Value: 32 bit unsigned little endian integer for the sequence
 > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
 > > to be max sequence (0xffffffff) if not provided.
 > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10
 > >     * Key: empty
 > >     * Value: 32 bit unsigned little endian integer for the lock time that
 > > this input requires. Must be omitted in PSBT v0. May be provided in PSBT
 > > v1, assumed to be 0 if not provided.
 > >
 > > Output:
 > > * PSBT_OUT_VALUE = 0x03
 > >     * Key: empty
 > >     * Value: 64-bit unsigned little endian integer for the output's
 > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
 > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04
 > >     * Key: empty
 > >     * Value: The script for this output. Otherwise known as the
 > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
 > >
 > > This change allows for PSBT to be used in the construction of
 > > transactions. With these new fields, inputs and outputs can be added as
 > > needed. One caveat is that there is no longer a unique transaction
 > > identifier so more care must be taken when combining PSBTs.
 > > Additionally, adding new inputs and outputs must be done such that
 > > signatures are not invalidated. This may be harder to specify.
 > >
 > > An important thing to note in this proposal are the fields
 > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
 > > transaction only has a single locktime yet a PSBT may have multiple
 > > locktimes. To choose the locktime for the transaction, finalizers must
 > > choose the maximum of all of the *_LOCKTIME fields.
 > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
 > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
 > > be set. This field allows finalizers to choose a locktime that is high
 > > enough for all inputs without needing to understand the scripts
 > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
 > > no inputs require a particular locktime.
 > >
 > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
 > > needs the version number bump to enforce backwards incompatibility.
 > > However once the inputs and outputs of a PSBT are decided, a PSBT could
 > > be "downgraded" back to v0 by creating the unsigned transaction from the
 > > above fields, and then dropping these new fields.
 > >
 > > If the list finds that these changes are reasonable, I will write a PR
 > > to modify BIP 174 to incorporate them.
 > >
 > > Thanks,
 > > Andrew Chow
 > 
 > 
 > _______________________________________________
 > bitcoin-dev mailing list
 > bitcoin-dev@lists.linuxfoundation.org
 > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
 > 

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Poelstra]

[-- Attachment #1: Type: text/plain, Size: 4068 bytes --]

On Wed, Dec 23, 2020 at 12:30:20AM -0300, fiatjaf via bitcoin-dev wrote:
> Hi Andrew.
> 
> I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured?
>

Yes, software will have to support both versions for a long time (likely
forever, at least in the case of Core). But I think this is okay, for a
couple of reasons:

1. it is very easy to convert from the old to new format, and from new to
   old (unless the new one uses features unsupported by the old). Indeed,
   the conversion logic is essentially the same as the logic that the
   Extractor role uses, so there isn't even that much redundant code.

2. There actually isn't a lot of software using PSBT out there, and most
   of that that does use PSBT is under rapid development. The obvious
   exception to this deployed hardware wallets, but as far as "software
   developers supporting old things for the sake of old hardware wallets"
   I think this transition is an order of magnitude simpler to handle
   than many of the ad-hoc protocol changes that individual vendors have
   done. In other words this is a "fact of life", and not even one of
   the grosser ones.

3. PSBT is pretty-much a dumb pure data format, and the diff between the
   new format and the old is pretty small.

> Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases.
>

The reasons for switching to PSBT 2 are actually more than just structuring
the data in a cleaner way. I agree that if the point of this upgrade were
just elegance, it would not be worth the compatibility loss. But there are
practical limitations that this proposal eliminates:

1. PSBT provides no way to modify the set of inputs or outputs after the
   Creator role is done.

2. Because of this, it forces certain things (e.g. locktimes and sequence
   numbers) to be chosen by the Creator, who may not have all the relevant
   information, and who certainly might not have it before any Updaters
   have done their part.

as well, of course, as elegance reasons:

3. Parsers of the existing PSBT need to understand the Bitcoin transaction
   format just to learn e.g. how many inputs and outputs there are. It is
   impossible to parse a PSBT without also parsing (almost) the whole
   transaction.

4. Similarly to cross-check fields like 'non_witness_utxo' which are
   committed to in the transaction, you have to parse the whole transaction
   just to make sure that the purely-redundant data is correctly redundant.

5. If you put a 0-input transaction into a PSBT (which would be pointless
   because there's no way to add inputs, but it's not forbidden so your
   software still has to deal with this somehow..), you need a different
   transaction parser than the normal one, because there is an ambiguity
   related to segwit that PSBT resolves differently.

It's also worth considering that PSBT is a young protocol, and future
extensions will be easier starting from PSBT 2 than starting from the
original version.

 
> Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html
> 

-- 
Andrew Poelstra
Director of Research, Blockstream
Email: apoelstra at wpsoftware.net
Web:   https://www.wpsoftware.net/andrew

The sun is always shining in space
    -Justin Lewis-Webster


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi,

On 12/22/20 10:30 PM, fiatjaf wrote:
> Hi Andrew.
>
> I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured?
No, it is not just "improvements in the way data is structured."

The primary reason for these changes is to allow PSBT to properly 
support adding inputs and outputs. This is a feature that many people 
have requested, and the ways that people have been doing it are honestly 
just hacks and not really the right way to be doing that. These changes 
allow for that feature to be supported well.

Furthermore, it is possible to downgrade and upgrade PSBTs between the 
two versions, once all inputs and outputs have been decided. Since 
PSBTv2 is essentially just taking all of the normal transaction fields 
and grouping them all with the rest of the data for those inputs and 
outputs, it is easy to reconstruct a global unsigned transaction and 
turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other 
way and break apart the global unsigned tx to turn a PSBTv0 into a 
PSBTv2. Originally, I had considered requiring that once a transaction 
was fully constructed it must be downgraded to a PSBTv0, but the 
structure changes that were made do make it easier to work with PSBT so 
I decided not to add this requirement.

Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be 
disallowed in PSBTv2 once the transaction is constructed? It would make 
things much more confusing though as it would no longer be a clean break.


Andrew Chow

> Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases.
>
> Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html
>
>   ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote ----
>   > Hi All,
>   >
>   > I have some updates on this after speaking with some people off-list.
>   >
>   > Firstly, the version number will be set to 2. In most discussions, this
>   > proposal was being referred to as PSBT version 2, so it'll be easier and
>   > clearer to set the version number to 2.
>   >
>   > For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
>   > there will be 2 of them, one for a time based lock time, and the other
>   > for height based. These will be:
>   > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>   >    * Key: empty
>   >    * Value: 32 bit unsigned little endian integer greater than or equal
>   > to 500000000 representing the minimum Unix timestamp that this input
>   > requires to be set as the transaction's lock time. Must be omitted in
>   > PSBTv0, and may be omitted in PSBTv2
>   > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>   >    * Key: empty
>   >    * Value: 32 bit unsigned little endian integer less than 500000000
>   > representing the minimum block height that this input requires to be set
>   > as the transaction's lock time. Must be omitted in PSBTv0, and may be
>   > omitted in PSBTv2.
>   >
>   > Having two lock time fields is necessary due to the behavior where all
>   > inputs must use the same type of lock time (height or time). Thus if an
>   > input requires a particular type of lock time, it must set the requisite
>   > field. Any new inputs being added must be able to accommodate all
>   > existing inputs' lock time type. This means they either must not have a
>   > lock time specified (i.e. no OP_CLTV involved), or have branches that
>   > allow the acceptance of either type. If an input has a lock time type
>   > that is incompatible with the rest of the transaction, it must not be added.
>   >
>   > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
>   > option if no input lock time fields are present. If there are input lock
>   > times, all lock time calculations must ignore it.
>   >
>   > Any role which does lock time calculation will first check if there are
>   > input lock time fields. If there are not, it must then check for a
>   > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the
>   > transaction's lock time. If it does not, the lock time is 0. If there
>   > are input lock time fields, it must choose the type which does not
>   > invalidate any inputs. The lock time is then determined to be the
>   > maximum value of all of the lock time fields for the chosen type.
>   >
>   >
>   > Additionally, I would like to add a new global field:
>   > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>   >    * Key: empty
>   >    * Value: A single byte as a boolean. 0 for False, 1 for True. All
>   > other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>   > in PSBTv2.
>   >
>   > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>   > outputs can be added to the PSBT. This flag may be set to True when
>   > inputs and outputs are being updated, signed, and finalized. However
>   > care must be taken when there are existing signatures. If this field is
>   > omitted or set to False, no further inputs and outputs may be added to
>   > the PSBT.
>   >
>   > Several rules must be followed to ensure that adding additional inputs
>   > and outputs will not invalidate existing signatures. First, an input or
>   > output adder must check for any existing signatures in all of the other
>   > inputs. If there are none, the input or output may be added in any
>   > position. If there are one or more signatures, each signature's sighash
>   > type must be examined. Inputs may only be added if all existing
>   > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
>   > existing signatures use SIGHASH_NONE. If an input has a signature using
>   > SIGHASH_SINGLE, the same number of inputs and outputs must be added
>   > before that input and it's corresponding output. For all other sighash
>   > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
>   > outputs may be added to the PSBT. Specific exceptions can be made in the
>   > future for additional sighash types.
>   >
>   > Furthermore, these newly added inputs must follow additional lock time
>   > rules. Because all signatures, regardless of sighash type, sign the
>   > transaction lock time, newly added inputs when there are existing
>   > signatures must have the same type of lock time used in the transaction,
>   > and must be less than or equal to the transaction lock time. It must not
>   > cause the transaction lock time to change, otherwise the signatures will
>   > be invalidated.
>   >
>   >
>   > Lastly, to uniquely identify transactions for combiners, a txid can be
>   > computed from the information present in the PSBT. Internally, combiners
>   > can create an unsigned transaction given the transaction version, the
>   > input prevouts, the outputs, and the computed locktime. This can then be
>   > used to calculate a txid and thus used as a way to identify PSBTs.
>   > Combiners will need to do this for all version 2 PSBTs in order to avoid
>   > combining distinct transactions.
>   >
>   >
>   > Andrew Chow
>   >
>   > On 12/9/20 5:25 PM, Andrew Chow wrote:
>   > > Hi All,
>   > >
>   > > I would like to propose a new PSBT version that addresses a few
>   > > deficiencies in the current PSBT v0. As this will be backwards
>   > > incompatible, a new PSBT version will be used, v1.
>   > >
>   > > The primary change is to truly have all input and output data for each
>   > > in their respective maps. Instead of having to parse an unsigned
>   > > transaction and lookup some data from there, and other data from the
>   > > correct map, all of the data for an input will be contained in its map.
>   > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
>   > > Thus I propose that the following fields be added:
>   > >
>   > > Global:
>   > > * PSBT_GLOBAL_TX_VERSION = 0x02
>   > >     * Key: empty
>   > >     * Value: 32-bit little endian unsigned integer for the transaction
>   > > version number. Must be provided in PSBT v1 and omitted in v0.
>   > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>   > >     * Key: empty
>   > >     * Value: 32 bit little endian unsigned integer for the preferred
>   > > transaction lock time. Must be omitted in PSBT v0. May be provided in
>   > > PSBT v1, assumed to be 0 if not provided.
>   > > * PSBT_GLOBAL_INPUT_COUNT = 0x04
>   > >     * Key: empty
>   > >     * Value: Compact size unsigned integer. Number of inputs in this
>   > > PSBT. Must be provided in PSBT v1 and omitted in v0.
>   > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>   > >     * Key: empty
>   > >     * Value: Compact size unsigned integer. Number of outputs in this
>   > > PSBT. Must be provided in PSBT v1 and omitted in v0.
>   > >
>   > > Input:
>   > > * PSBT_IN_PREVIOUS_TXID = 0x0e
>   > >     * Key: empty
>   > >     * Value: 32 byte txid of the previous transaction whose output at
>   > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
>   > > omitted in v0.
>   > > * PSBT_IN_OUTPUT_INDEX = 0x0f
>   > >     * Key: empty
>   > >     * Value: 32 bit little endian integer for the index of the output
>   > > being spent. Must be provided in PSBT v1 and omitted in v0.
>   > > * PSBT_IN_SEQUENCE = 0x0f
>   > >     * Key: empty
>   > >     * Value: 32 bit unsigned little endian integer for the sequence
>   > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
>   > > to be max sequence (0xffffffff) if not provided.
>   > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>   > >     * Key: empty
>   > >     * Value: 32 bit unsigned little endian integer for the lock time that
>   > > this input requires. Must be omitted in PSBT v0. May be provided in PSBT
>   > > v1, assumed to be 0 if not provided.
>   > >
>   > > Output:
>   > > * PSBT_OUT_VALUE = 0x03
>   > >     * Key: empty
>   > >     * Value: 64-bit unsigned little endian integer for the output's
>   > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
>   > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>   > >     * Key: empty
>   > >     * Value: The script for this output. Otherwise known as the
>   > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>   > >
>   > > This change allows for PSBT to be used in the construction of
>   > > transactions. With these new fields, inputs and outputs can be added as
>   > > needed. One caveat is that there is no longer a unique transaction
>   > > identifier so more care must be taken when combining PSBTs.
>   > > Additionally, adding new inputs and outputs must be done such that
>   > > signatures are not invalidated. This may be harder to specify.
>   > >
>   > > An important thing to note in this proposal are the fields
>   > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
>   > > transaction only has a single locktime yet a PSBT may have multiple
>   > > locktimes. To choose the locktime for the transaction, finalizers must
>   > > choose the maximum of all of the *_LOCKTIME fields.
>   > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
>   > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
>   > > be set. This field allows finalizers to choose a locktime that is high
>   > > enough for all inputs without needing to understand the scripts
>   > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
>   > > no inputs require a particular locktime.
>   > >
>   > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
>   > > needs the version number bump to enforce backwards incompatibility.
>   > > However once the inputs and outputs of a PSBT are decided, a PSBT could
>   > > be "downgraded" back to v0 by creating the unsigned transaction from the
>   > > above fields, and then dropping these new fields.
>   > >
>   > > If the list finds that these changes are reasonable, I will write a PR
>   > > to modify BIP 174 to incorporate them.
>   > >
>   > > Thanks,
>   > > Andrew Chow
>   >
>   >
>   > _______________________________________________
>   > bitcoin-dev mailing list
>   > bitcoin-dev@lists.linuxfoundation.org
>   > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>   >

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi All,

The full modified BIP can be read at 
https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki.

I will open a PR to the BIPs repo soon after further discussion on this.


Andrew

On 12/22/20 3:12 PM, Andrew Chow wrote:
> Hi All,
>
> I have some updates on this after speaking with some people off-list.
>
> Firstly, the version number will be set to 2. In most discussions, this
> proposal was being referred to as PSBT version 2, so it'll be easier and
> clearer to set the version number to 2.
>
> For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
> there will be 2 of them, one for a time based lock time, and the other
> for height based. These will be:
> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer greater than or equal
> to 500000000 representing the minimum Unix timestamp that this input
> requires to be set as the transaction's lock time. Must be omitted in
> PSBTv0, and may be omitted in PSBTv2
> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer less than 500000000
> representing the minimum block height that this input requires to be set
> as the transaction's lock time. Must be omitted in PSBTv0, and may be
> omitted in PSBTv2.
>
> Having two lock time fields is necessary due to the behavior where all
> inputs must use the same type of lock time (height or time). Thus if an
> input requires a particular type of lock time, it must set the requisite
> field. Any new inputs being added must be able to accommodate all
> existing inputs' lock time type. This means they either must not have a
> lock time specified (i.e. no OP_CLTV involved), or have branches that
> allow the acceptance of either type. If an input has a lock time type
> that is incompatible with the rest of the transaction, it must not be added.
>
> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
> option if no input lock time fields are present. If there are input lock
> times, all lock time calculations must ignore it.
>
> Any role which does lock time calculation will first check if there are
> input lock time fields. If there are not, it must then check for a
> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the
> transaction's lock time. If it does not, the lock time is 0. If there
> are input lock time fields, it must choose the type which does not
> invalidate any inputs. The lock time is then determined to be the
> maximum value of all of the lock time fields for the chosen type.
>
>
> Additionally, I would like to add a new global field:
> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>     * Key: empty
>     * Value: A single byte as a boolean. 0 for False, 1 for True. All
> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
> in PSBTv2.
>
> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
> outputs can be added to the PSBT. This flag may be set to True when
> inputs and outputs are being updated, signed, and finalized. However
> care must be taken when there are existing signatures. If this field is
> omitted or set to False, no further inputs and outputs may be added to
> the PSBT.
>
> Several rules must be followed to ensure that adding additional inputs
> and outputs will not invalidate existing signatures. First, an input or
> output adder must check for any existing signatures in all of the other
> inputs. If there are none, the input or output may be added in any
> position. If there are one or more signatures, each signature's sighash
> type must be examined. Inputs may only be added if all existing
> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
> existing signatures use SIGHASH_NONE. If an input has a signature using
> SIGHASH_SINGLE, the same number of inputs and outputs must be added
> before that input and it's corresponding output. For all other sighash
> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
> outputs may be added to the PSBT. Specific exceptions can be made in the
> future for additional sighash types.
>
> Furthermore, these newly added inputs must follow additional lock time
> rules. Because all signatures, regardless of sighash type, sign the
> transaction lock time, newly added inputs when there are existing
> signatures must have the same type of lock time used in the transaction,
> and must be less than or equal to the transaction lock time. It must not
> cause the transaction lock time to change, otherwise the signatures will
> be invalidated.
>
>
> Lastly, to uniquely identify transactions for combiners, a txid can be
> computed from the information present in the PSBT. Internally, combiners
> can create an unsigned transaction given the transaction version, the
> input prevouts, the outputs, and the computed locktime. This can then be
> used to calculate a txid and thus used as a way to identify PSBTs.
> Combiners will need to do this for all version 2 PSBTs in order to avoid
> combining distinct transactions.
>
>
> Andrew Chow
>
> On 12/9/20 5:25 PM, Andrew Chow wrote:
>> Hi All,
>>
>> I would like to propose a new PSBT version that addresses a few
>> deficiencies in the current PSBT v0. As this will be backwards
>> incompatible, a new PSBT version will be used, v1.
>>
>> The primary change is to truly have all input and output data for each
>> in their respective maps. Instead of having to parse an unsigned
>> transaction and lookup some data from there, and other data from the
>> correct map, all of the data for an input will be contained in its map.
>> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
>> Thus I propose that the following fields be added:
>>
>> Global:
>> * PSBT_GLOBAL_TX_VERSION = 0x02
>>      * Key: empty
>>      * Value: 32-bit little endian unsigned integer for the transaction
>> version number. Must be provided in PSBT v1 and omitted in v0.
>> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>>      * Key: empty
>>      * Value: 32 bit little endian unsigned integer for the preferred
>> transaction lock time. Must be omitted in PSBT v0. May be provided in
>> PSBT v1, assumed to be 0 if not provided.
>> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>>      * Key: empty
>>      * Value: Compact size unsigned integer. Number of inputs in this
>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>>      * Key: empty
>>      * Value: Compact size unsigned integer. Number of outputs in this
>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>>
>> Input:
>> * PSBT_IN_PREVIOUS_TXID = 0x0e
>>      * Key: empty
>>      * Value: 32 byte txid of the previous transaction whose output at
>> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
>> omitted in v0.
>> * PSBT_IN_OUTPUT_INDEX = 0x0f
>>      * Key: empty
>>      * Value: 32 bit little endian integer for the index of the output
>> being spent. Must be provided in PSBT v1 and omitted in v0.
>> * PSBT_IN_SEQUENCE = 0x0f
>>      * Key: empty
>>      * Value: 32 bit unsigned little endian integer for the sequence
>> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
>> to be max sequence (0xffffffff) if not provided.
>> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>>      * Key: empty
>>      * Value: 32 bit unsigned little endian integer for the lock time that
>> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
>> v1, assumed to be 0 if not provided.
>>
>> Output:
>> * PSBT_OUT_VALUE = 0x03
>>      * Key: empty
>>      * Value: 64-bit unsigned little endian integer for the output's
>> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
>> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>>      * Key: empty
>>      * Value: The script for this output. Otherwise known as the
>> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>>
>> This change allows for PSBT to be used in the construction of
>> transactions. With these new fields, inputs and outputs can be added as
>> needed. One caveat is that there is no longer a unique transaction
>> identifier so more care must be taken when combining PSBTs.
>> Additionally, adding new inputs and outputs must be done such that
>> signatures are not invalidated. This may be harder to specify.
>>
>> An important thing to note in this proposal are the fields
>> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
>> transaction only has a single locktime yet a PSBT may have multiple
>> locktimes. To choose the locktime for the transaction, finalizers must
>> choose the maximum of all of the *_LOCKTIME fields.
>> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
>> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
>> be set. This field allows finalizers to choose a locktime that is high
>> enough for all inputs without needing to understand the scripts
>> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
>> no inputs require a particular locktime.
>>
>> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
>> needs the version number bump to enforce backwards incompatibility.
>> However once the inputs and outputs of a PSBT are decided, a PSBT could
>> be "downgraded" back to v0 by creating the unsigned transaction from the
>> above fields, and then dropping these new fields.
>>
>> If the list finds that these changes are reasonable, I will write a PR
>> to modify BIP 174 to incorporate them.
>>
>> Thanks,
>> Andrew Chow

Re: [bitcoin-dev] New PSBT version proposal

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 14125 bytes --]

One thing I think should be added in V2 is the ability to specify sighash
flags per-key as opposed to per-input.

The per-key restriction is unfitting given that there are circumstances
where multisig signers may validate heterogenous logic.
--
@JeremyRubin <https://twitter.com/JeremyRubin>
<https://twitter.com/JeremyRubin>


On Wed, Dec 23, 2020 at 1:37 PM Andrew Chow via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi,
>
> On 12/22/20 10:30 PM, fiatjaf wrote:
> > Hi Andrew.
> >
> > I'm just a lurker here and I have not much experience with PSBTs, but
> still let me pose this very obvious question and concern: isn't this change
> going to create a compatibility nightmare, with some software supporting
> version 1, others supporting version 2, and the ones that care enough about
> UX and are still maintained being forced to support both versions -- and
> for no very important reason except some improvements in the way data is
> structured?
> No, it is not just "improvements in the way data is structured."
>
> The primary reason for these changes is to allow PSBT to properly
> support adding inputs and outputs. This is a feature that many people
> have requested, and the ways that people have been doing it are honestly
> just hacks and not really the right way to be doing that. These changes
> allow for that feature to be supported well.
>
> Furthermore, it is possible to downgrade and upgrade PSBTs between the
> two versions, once all inputs and outputs have been decided. Since
> PSBTv2 is essentially just taking all of the normal transaction fields
> and grouping them all with the rest of the data for those inputs and
> outputs, it is easy to reconstruct a global unsigned transaction and
> turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other
> way and break apart the global unsigned tx to turn a PSBTv0 into a
> PSBTv2. Originally, I had considered requiring that once a transaction
> was fully constructed it must be downgraded to a PSBTv0, but the
> structure changes that were made do make it easier to work with PSBT so
> I decided not to add this requirement.
>
> Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be
> disallowed in PSBTv2 once the transaction is constructed? It would make
> things much more confusing though as it would no longer be a clean break.
>
>
> Andrew Chow
>
> > Ultimately I don't think it should matter if some data is structured in
> not-the-best-possible way, as long as it is clear enough for the computer
> and for the libraries already written to deal with it.
> Backwards-compatibility and general interoperability is worth much more
> than anything else in these cases.
> >
> > Also let me leave this article here, which I find very important (even
> if for some reason it ends up not being relevant to this specific case):
> http://scripting.com/2017/05/09/rulesForStandardsmakers.html
> >
> >   ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote ----
> >   > Hi All,
> >   >
> >   > I have some updates on this after speaking with some people off-list.
> >   >
> >   > Firstly, the version number will be set to 2. In most discussions,
> this
> >   > proposal was being referred to as PSBT version 2, so it'll be easier
> and
> >   > clearer to set the version number to 2.
> >   >
> >   > For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
> >   > there will be 2 of them, one for a time based lock time, and the
> other
> >   > for height based. These will be:
> >   > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
> >   >    * Key: empty
> >   >    * Value: 32 bit unsigned little endian integer greater than or
> equal
> >   > to 500000000 representing the minimum Unix timestamp that this input
> >   > requires to be set as the transaction's lock time. Must be omitted in
> >   > PSBTv0, and may be omitted in PSBTv2
> >   > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
> >   >    * Key: empty
> >   >    * Value: 32 bit unsigned little endian integer less than 500000000
> >   > representing the minimum block height that this input requires to be
> set
> >   > as the transaction's lock time. Must be omitted in PSBTv0, and may be
> >   > omitted in PSBTv2.
> >   >
> >   > Having two lock time fields is necessary due to the behavior where
> all
> >   > inputs must use the same type of lock time (height or time). Thus if
> an
> >   > input requires a particular type of lock time, it must set the
> requisite
> >   > field. Any new inputs being added must be able to accommodate all
> >   > existing inputs' lock time type. This means they either must not
> have a
> >   > lock time specified (i.e. no OP_CLTV involved), or have branches that
> >   > allow the acceptance of either type. If an input has a lock time type
> >   > that is incompatible with the rest of the transaction, it must not
> be added.
> >   >
> >   > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
> >   > option if no input lock time fields are present. If there are input
> lock
> >   > times, all lock time calculations must ignore it.
> >   >
> >   > Any role which does lock time calculation will first check if there
> are
> >   > input lock time fields. If there are not, it must then check for a
> >   > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is
> the
> >   > transaction's lock time. If it does not, the lock time is 0. If there
> >   > are input lock time fields, it must choose the type which does not
> >   > invalidate any inputs. The lock time is then determined to be the
> >   > maximum value of all of the lock time fields for the chosen type.
> >   >
> >   >
> >   > Additionally, I would like to add a new global field:
> >   > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
> >   >    * Key: empty
> >   >    * Value: A single byte as a boolean. 0 for False, 1 for True. All
> >   > other values ore prohibited. Must be omitted for PSBTv0, may be
> omitted
> >   > in PSBTv2.
> >   >
> >   > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
> >   > outputs can be added to the PSBT. This flag may be set to True when
> >   > inputs and outputs are being updated, signed, and finalized. However
> >   > care must be taken when there are existing signatures. If this field
> is
> >   > omitted or set to False, no further inputs and outputs may be added
> to
> >   > the PSBT.
> >   >
> >   > Several rules must be followed to ensure that adding additional
> inputs
> >   > and outputs will not invalidate existing signatures. First, an input
> or
> >   > output adder must check for any existing signatures in all of the
> other
> >   > inputs. If there are none, the input or output may be added in any
> >   > position. If there are one or more signatures, each signature's
> sighash
> >   > type must be examined. Inputs may only be added if all existing
> >   > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
> >   > existing signatures use SIGHASH_NONE. If an input has a signature
> using
> >   > SIGHASH_SINGLE, the same number of inputs and outputs must be added
> >   > before that input and it's corresponding output. For all other
> sighash
> >   > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
> >   > outputs may be added to the PSBT. Specific exceptions can be made in
> the
> >   > future for additional sighash types.
> >   >
> >   > Furthermore, these newly added inputs must follow additional lock
> time
> >   > rules. Because all signatures, regardless of sighash type, sign the
> >   > transaction lock time, newly added inputs when there are existing
> >   > signatures must have the same type of lock time used in the
> transaction,
> >   > and must be less than or equal to the transaction lock time. It must
> not
> >   > cause the transaction lock time to change, otherwise the signatures
> will
> >   > be invalidated.
> >   >
> >   >
> >   > Lastly, to uniquely identify transactions for combiners, a txid can
> be
> >   > computed from the information present in the PSBT. Internally,
> combiners
> >   > can create an unsigned transaction given the transaction version, the
> >   > input prevouts, the outputs, and the computed locktime. This can
> then be
> >   > used to calculate a txid and thus used as a way to identify PSBTs.
> >   > Combiners will need to do this for all version 2 PSBTs in order to
> avoid
> >   > combining distinct transactions.
> >   >
> >   >
> >   > Andrew Chow
> >   >
> >   > On 12/9/20 5:25 PM, Andrew Chow wrote:
> >   > > Hi All,
> >   > >
> >   > > I would like to propose a new PSBT version that addresses a few
> >   > > deficiencies in the current PSBT v0. As this will be backwards
> >   > > incompatible, a new PSBT version will be used, v1.
> >   > >
> >   > > The primary change is to truly have all input and output data for
> each
> >   > > in their respective maps. Instead of having to parse an unsigned
> >   > > transaction and lookup some data from there, and other data from
> the
> >   > > correct map, all of the data for an input will be contained in its
> map.
> >   > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new
> version.
> >   > > Thus I propose that the following fields be added:
> >   > >
> >   > > Global:
> >   > > * PSBT_GLOBAL_TX_VERSION = 0x02
> >   > >     * Key: empty
> >   > >     * Value: 32-bit little endian unsigned integer for the
> transaction
> >   > > version number. Must be provided in PSBT v1 and omitted in v0.
> >   > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
> >   > >     * Key: empty
> >   > >     * Value: 32 bit little endian unsigned integer for the
> preferred
> >   > > transaction lock time. Must be omitted in PSBT v0. May be provided
> in
> >   > > PSBT v1, assumed to be 0 if not provided.
> >   > > * PSBT_GLOBAL_INPUT_COUNT = 0x04
> >   > >     * Key: empty
> >   > >     * Value: Compact size unsigned integer. Number of inputs in
> this
> >   > > PSBT. Must be provided in PSBT v1 and omitted in v0.
> >   > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
> >   > >     * Key: empty
> >   > >     * Value: Compact size unsigned integer. Number of outputs in
> this
> >   > > PSBT. Must be provided in PSBT v1 and omitted in v0.
> >   > >
> >   > > Input:
> >   > > * PSBT_IN_PREVIOUS_TXID = 0x0e
> >   > >     * Key: empty
> >   > >     * Value: 32 byte txid of the previous transaction whose output
> at
> >   > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1
> and
> >   > > omitted in v0.
> >   > > * PSBT_IN_OUTPUT_INDEX = 0x0f
> >   > >     * Key: empty
> >   > >     * Value: 32 bit little endian integer for the index of the
> output
> >   > > being spent. Must be provided in PSBT v1 and omitted in v0.
> >   > > * PSBT_IN_SEQUENCE = 0x0f
> >   > >     * Key: empty
> >   > >     * Value: 32 bit unsigned little endian integer for the sequence
> >   > > number. Must be omitted in PSBT v0. May be provided in PSBT v1
> assumed
> >   > > to be max sequence (0xffffffff) if not provided.
> >   > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10
> >   > >     * Key: empty
> >   > >     * Value: 32 bit unsigned little endian integer for the lock
> time that
> >   > > this input requires. Must be omitted in PSBT v0. May be provided
> in PSBT
> >   > > v1, assumed to be 0 if not provided.
> >   > >
> >   > > Output:
> >   > > * PSBT_OUT_VALUE = 0x03
> >   > >     * Key: empty
> >   > >     * Value: 64-bit unsigned little endian integer for the output's
> >   > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
> >   > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04
> >   > >     * Key: empty
> >   > >     * Value: The script for this output. Otherwise known as the
> >   > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
> >   > >
> >   > > This change allows for PSBT to be used in the construction of
> >   > > transactions. With these new fields, inputs and outputs can be
> added as
> >   > > needed. One caveat is that there is no longer a unique transaction
> >   > > identifier so more care must be taken when combining PSBTs.
> >   > > Additionally, adding new inputs and outputs must be done such that
> >   > > signatures are not invalidated. This may be harder to specify.
> >   > >
> >   > > An important thing to note in this proposal are the fields
> >   > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A
> Bitcoin
> >   > > transaction only has a single locktime yet a PSBT may have multiple
> >   > > locktimes. To choose the locktime for the transaction, finalizers
> must
> >   > > choose the maximum of all of the *_LOCKTIME fields.
> >   > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as
> those
> >   > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum
> locktime to
> >   > > be set. This field allows finalizers to choose a locktime that is
> high
> >   > > enough for all inputs without needing to understand the scripts
> >   > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to
> use if
> >   > > no inputs require a particular locktime.
> >   > >
> >   > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT
> v1
> >   > > needs the version number bump to enforce backwards incompatibility.
> >   > > However once the inputs and outputs of a PSBT are decided, a PSBT
> could
> >   > > be "downgraded" back to v0 by creating the unsigned transaction
> from the
> >   > > above fields, and then dropping these new fields.
> >   > >
> >   > > If the list finds that these changes are reasonable, I will write
> a PR
> >   > > to modify BIP 174 to incorporate them.
> >   > >
> >   > > Thanks,
> >   > > Andrew Chow
> >   >
> >   >
> >   > _______________________________________________
> >   > bitcoin-dev mailing list
> >   > bitcoin-dev@lists.linuxfoundation.org
> >   > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >   >
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 18104 bytes --]

Re: [bitcoin-dev] New PSBT version proposal

[Rusty Russell]

Hi Andrew et al,

        Very excited to see this progress; thanks for doing all the
work!  Sorry for the delayed feedback, I didn't get to this before the
break.

> Additionally, I would like to add a new global field:
> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>    * Key: empty
>    * Value: A single byte as a boolean. 0 for False, 1 for True. All 
> other values ore prohibited. Must be omitted for PSBTv0, may be omitted 
> in PSBTv2.
>
> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and 
> outputs can be added to the PSBT. This flag may be set to True when 
> inputs and outputs are being updated, signed, and finalized. However 
> care must be taken when there are existing signatures. If this field is 
> omitted or set to False, no further inputs and outputs may be added to 
> the PSBT.

I wonder if this can be flagged simply by omitting the (AFAICT
redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
are the purposes of those fields?

For our uses, there would be no signatures at this stage; it's simply a
subdivision of the Creator role.  This role would be terminated by
removing the under-construction marker.  For this, it could be clear
that such an under-construction PSBT SHOULD NOT be signed.

Otherwise, if an explicit marker is required, I would omit the value and
simply use its existence to as a flag.  Having two "false" values is
simply asking for trouble.

Thanks!
Rusty.
PS.  Perhaps we should change the name to PBT (Partial Bitcoin
     Transaction) now, since it's more than just signing...

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi Rusty,

On 1/6/21 6:26 PM, Rusty Russell wrote:
> Hi Andrew et al,
>
>          Very excited to see this progress; thanks for doing all the
> work!  Sorry for the delayed feedback, I didn't get to this before the
> break.
>
>> Additionally, I would like to add a new global field:
>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>     * Key: empty
>>     * Value: A single byte as a boolean. 0 for False, 1 for True. All
>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>> in PSBTv2.
>>
>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>> outputs can be added to the PSBT. This flag may be set to True when
>> inputs and outputs are being updated, signed, and finalized. However
>> care must be taken when there are existing signatures. If this field is
>> omitted or set to False, no further inputs and outputs may be added to
>> the PSBT.
> I wonder if this can be flagged simply by omitting the (AFAICT
> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
> are the purposes of those fields?
The purpose of those fields is to know how many input and output maps 
there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine 
whether a map is an input map or an output map. So the counts are there 
to allow that.
> For our uses, there would be no signatures at this stage; it's simply a
> subdivision of the Creator role.  This role would be terminated by
> removing the under-construction marker.  For this, it could be clear
> that such an under-construction PSBT SHOULD NOT be signed.
There are some protocols where signed inputs are added to transactions.
> Otherwise, if an explicit marker is required, I would omit the value and
> simply use its existence to as a flag.  Having two "false" values is
> simply asking for trouble.
Seems reasonable.


Andrew

> Thanks!
> Rusty.
> PS.  Perhaps we should change the name to PBT (Partial Bitcoin
>       Transaction) now, since it's more than just signing...

Re: [bitcoin-dev] New PSBT version proposal

[Rusty Russell]

Andrew Chow <achow101-lists@achow101.com> writes:
> Hi Rusty,
>
> On 1/6/21 6:26 PM, Rusty Russell wrote:
>> Hi Andrew et al,
>>
>>          Very excited to see this progress; thanks for doing all the
>> work!  Sorry for the delayed feedback, I didn't get to this before the
>> break.
>>
>>> Additionally, I would like to add a new global field:
>>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>>     * Key: empty
>>>     * Value: A single byte as a boolean. 0 for False, 1 for True. All
>>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>>> in PSBTv2.
>>>
>>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>>> outputs can be added to the PSBT. This flag may be set to True when
>>> inputs and outputs are being updated, signed, and finalized. However
>>> care must be taken when there are existing signatures. If this field is
>>> omitted or set to False, no further inputs and outputs may be added to
>>> the PSBT.
>> I wonder if this can be flagged simply by omitting the (AFAICT
>> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
>> are the purposes of those fields?
> The purpose of those fields is to know how many input and output maps 
> there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine 
> whether a map is an input map or an output map. So the counts are there 
> to allow that.

Ah, yeah, you need at least the number of input maps :(

It's generally preferable to have sections be self-describing;
internally if you have a function which takes all the input maps you
should be able to trivially tell if you're handed the output maps by
mistake.  Similarly, it would have been nice to have an input map be a
distinctly marked type from global or output maps.

Nonetheless, that's a bigger change.  You could just require a double-00
terminator between the global, input and output sections though.

>> For our uses, there would be no signatures at this stage; it's simply a
>> subdivision of the Creator role.  This role would be terminated by
>> removing the under-construction marker.  For this, it could be clear
>> that such an under-construction PSBT SHOULD NOT be signed.
>
> There are some protocols where signed inputs are added to transactions.

Sure, but you can't solve every problem.  We've now created the
possibility that a PSBT is "under construction" but can't be modified,
*and* a very invasive requirement to determine that.

I disagree with Andrew's goal here:

>   1. PSBT provides no way to modify the set of inputs or outputs after the
>      Creator role is done.

It's simpler if, "the under-construction PSBT can be used within the
Creator role, which can now have sub-roles".

If you really want to allow this (and I think we need to explore
concrete examples to justify this complexity!), better to add data to
PSBT_GLOBAL_UNDER_CONSTRUCTION:
1. a flag to indicate whether inputs are modifiable.
2. a flag to indicate whether outputs are modifiable.
3. a bitmap of what inputs are SIGHASH_SINGLE.

If you add a signature which is not SIGHASH_NONE, you clear the "outputs
modifiable" flag.  If you add a signature which is not
SIGHASH_ANYONECANPAY, you clear the "inputs modifiable" flag.  If you
clear both flags, you remove the PSBT_GLOBAL_UNDER_CONSTRUCTION
altogether.  You similarly set the bitmap depending on whether all sigs
are SIGHASH_SINGLE.

Cheers,
Rusty.

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

On 1/7/21 7:40 PM, Rusty Russell wrote:
> Andrew Chow <achow101-lists@achow101.com> writes:
>> Hi Rusty,
>>
>> On 1/6/21 6:26 PM, Rusty Russell wrote:
>>> Hi Andrew et al,
>>>
>>>           Very excited to see this progress; thanks for doing all the
>>> work!  Sorry for the delayed feedback, I didn't get to this before the
>>> break.
>>>
>>>> Additionally, I would like to add a new global field:
>>>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>>>      * Key: empty
>>>>      * Value: A single byte as a boolean. 0 for False, 1 for True. All
>>>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>>>> in PSBTv2.
>>>>
>>>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>>>> outputs can be added to the PSBT. This flag may be set to True when
>>>> inputs and outputs are being updated, signed, and finalized. However
>>>> care must be taken when there are existing signatures. If this field is
>>>> omitted or set to False, no further inputs and outputs may be added to
>>>> the PSBT.
>>> I wonder if this can be flagged simply by omitting the (AFAICT
>>> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
>>> are the purposes of those fields?
>> The purpose of those fields is to know how many input and output maps
>> there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine
>> whether a map is an input map or an output map. So the counts are there
>> to allow that.
> Ah, yeah, you need at least the number of input maps :(
>
> It's generally preferable to have sections be self-describing;
> internally if you have a function which takes all the input maps you
> should be able to trivially tell if you're handed the output maps by
> mistake.  Similarly, it would have been nice to have an input map be a
> distinctly marked type from global or output maps.
>
> Nonetheless, that's a bigger change.  You could just require a double-00
> terminator between the global, input and output sections though.
Changing that is a bigger change and I'd rather keep this as minimal as 
possible. Having only new fields is simpler.

>>> For our uses, there would be no signatures at this stage; it's simply a
>>> subdivision of the Creator role.  This role would be terminated by
>>> removing the under-construction marker.  For this, it could be clear
>>> that such an under-construction PSBT SHOULD NOT be signed.
>> There are some protocols where signed inputs are added to transactions.
> Sure, but you can't solve every problem.  We've now created the
> possibility that a PSBT is "under construction" but can't be modified,
> *and* a very invasive requirement to determine that.
>
> I disagree with Andrew's goal here:
>
>>    1. PSBT provides no way to modify the set of inputs or outputs after the
>>       Creator role is done.
> It's simpler if, "the under-construction PSBT can be used within the
> Creator role, which can now have sub-roles".
>
> If you really want to allow this (and I think we need to explore
> concrete examples to justify this complexity!), better to add data to
> PSBT_GLOBAL_UNDER_CONSTRUCTION:
> 1. a flag to indicate whether inputs are modifiable.
> 2. a flag to indicate whether outputs are modifiable.
> 3. a bitmap of what inputs are SIGHASH_SINGLE.
>
> If you add a signature which is not SIGHASH_NONE, you clear the "outputs
> modifiable" flag.  If you add a signature which is not
> SIGHASH_ANYONECANPAY, you clear the "inputs modifiable" flag.  If you
> clear both flags, you remove the PSBT_GLOBAL_UNDER_CONSTRUCTION
> altogether.  You similarly set the bitmap depending on whether all sigs
> are SIGHASH_SINGLE.
I think we do need to support adding signed inputs and adding inputs to 
signed transactions. Someone had asked for this feature before. 
Additionally Nicolas Dorier informed me that his NTumbleBit project does 
those things.

In that case, I will include your suggestions for 
PSBT_GLOBAL_UNDER_CONSTRUCTION in the BIP.


Andrew Chow
>
> Cheers,
> Rusty.

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

Hi All,

I've made some reorganization changes to the way that new PSBT versions 
should be handled in BIP 174 (see 
https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted 
as a separate BIP. The full document can be read at 
https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I 
have also included it in this email.

I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and 
made a few modifications. First, the field will be named 
PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and 
outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as 
a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most 
PSBTs to not have to carry around a useless bitmap.

Andrew

***

<pre>
   BIP: PSBTv2
   Layer: Applications
   Title: PSBT Version 2
   Author: Andrew Chow <achow101@gmail.com>
   Comments-Summary: No comments yet.
   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2
   Status: Draft
   Type: Standards Track
   Created: 2021-01-14
   License: BSD-2-Clause
</pre>

==Introduction==

===Abstract===

This document proposes a second version of the Partially Signed Bitcoin 
Transaction format
described in BIP 174 which allows for inputs and outputs to be added to 
the PSBT after creation.

===Copyright===

This BIP is licensed under the 2-clause BSD license.

===Motivation===

Partially Signed Bitcoin Transaction Version 0 as described in BIP 174 
is unable to have new
inputs and outputs be added to the transaction. The fixed global 
unsigned transaction
cannot be changed which prevents any additional inputs or outputs to be 
added.
PSBT Version 2 is intended to rectify this problem.

An additional benficial side effect is that all information for a given 
input or output will be
provided by its <tt><input-map></tt> or <tt><output-map></tt>. With 
Version 0, to retrieve
all of the information for an input or output, data would need to be 
found in two locations:
the <tt><input-map></tt>/<tt><output-map></tt> and the global unsigned 
transaction. PSBT
Version 2 now moves all related information to one place.

==Specification==

PSBT Version 2 (PSBTv2) only specifies new fields and field 
inclusion/exclusion requirements.

<tt>PSBT_GLOBAL_UNSIGNED_TX</tt> must be excluded in PSBTv2.
<tt>PSBT_GLOBAL_VERSION</tt> must be included in PSBTv2 and set to 
version number 2<ref>'''What happened to version number 1?'''
Version number 1 is skipped because PSBT Version 0 has been colloquially 
referred to as version 1. Originally this BIP was to be
version 1, but because it has been colloquially referred to as version 2 
during its design phrase, it was decided to change the
version number to 2 so that there would not be any confusion</ref>.

The new global types for PSBT Version 2 are as follows:

{|
! Name
! <tt><keytype></tt>
! <tt><keydata></tt>
! <tt><keydata></tt> Description
! <tt><valuedata></tt>
! <tt><valuedata></tt> Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Transaction Version
| <tt>PSBT_GLOBAL_TX_VERSION = 0x02</tt>
| None
| No key data
| <tt><32-bit uint></tt>
| The 32-bit little endian signed integer representing the version 
number of the transaction being created. Note that this is not the same 
as the PSBT version number specified by the PSBT_GLOBAL_VERSION field.
| 2
| 0
| 2
|-
| Fallback Locktime
| <tt>PSBT_GLOBAL_FALLBACK_LOCKTIME = 0x03</tt>
| None
| No key data
| <tt><32-bit uint></tt>
| The 32-bit little endian unsigned integer representing the transaction 
locktime to use if no inputs specify a required locktime.
|
| 0
| 2
|-
| Input Count
| <tt>PSBT_GLOBAL_INPUT_COUNT = 0x04</tt>
| None
| No key data
| <tt><compact size uint></tt>
| Compact size unsigned integer representing the number of inputs in 
this PSBT.
| 2
| 0
| 2
|-
| Output Count
| <tt>PSBT_GLOBAL_OUTPUT_COUNT = 0x05</tt>
| None
| No key data
| <tt><compact size uint></tt>
| Compact size unsigned integer representing the number of outputs in 
this PSBT.
| 2
| 0
| 2
|-
| Transaction Modifiable Flags
| <tt>PSBT_GLOBAL_TX_MODIFIABLE = 0x06</tt>
| None
| No key data
| <tt><single byte boolean> <single byte boolean></tt>
| A single byte boolean (0 for False, 1 for True) representing whether 
inputs can be modified, referred to as the Inputs Modifiable Flag. This 
is followed by a single byte boolean representing whether outputs can be 
modified, referred to as the Outputs Modifiable Flag.
|
| 0
| 2
|-
| SIGHASH_SINGLE Inputs
| <tt>PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS = 0x07</tt>
| None
| No key data
| <tt><bit vector></tt>
| A bit vector representing which input indexes use SIGHASH_SINGLE. If 
the bit for an index is set to 1, then the input and output pair at that 
index are tied together with SIGHASH_SINGLE and must be moved together.
|
| 0
| 2
|}

The new per-input types for PSBT Version 2 are defined as follows:

{|
! Name
! <tt><keytype></tt>
! <tt><keydata></tt>
! <tt><keydata></tt> Description
! <tt><valuedata></tt>
! <tt><valuedata></tt> Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Previous TXID
| <tt>PSBT_IN_PREVIOUS_TXID = 0x0e</tt>
| None
| No key data
| <tt><txid></tt>
| 32 byte txid of the previous transaction whose output at 
PSBT_IN_OUTPUT_INDEX is being spent.
| 2
| 0
| 2
|-
| Spent Output Index
| <tt>PSBT_IN_OUTPUT_INDEX = 0x0f</tt>
| None
| No key data
| <tt><32-bit uint></tt>
| 32 bit little endian integer representing the index of the output 
being spent in the transaction with the txid of PSBT_IN_PREVIOUS_TXID.
| 2
| 0
| 2
|-
| Sequence Number
| <tt>PSBT_IN_SEQUENCE = 0x10</tt>
| None
| No key data
| <tt><32-bit uint></tt>
| The 32 bit unsigned little endian integer for the sequence number of 
this input. If omitted, the sequence number is assumed to be the final 
sequence number (0xffffffff).
|
| 0
| 2
|-
| Required Time-based Locktime
| <tt>PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x11</tt>
| None
| No key data
| <tt><32-bit uint></tt>
| 32 bit unsigned little endian integer greater than or equal to 
500000000 representing the minimum Unix timestamp that this input 
requires to be set as the transaction's lock time.
|
| 0
| 2
|-
| Required Height-based Locktime
| <tt>PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x12</tt>
| None
| No key data
| <tt><32-bit uiht></tt>
| 32 bit unsigned little endian integer less than 500000000 representing 
the minimum block height that this input requires to be set as the 
transaction's lock time.
|
| 0
| 2
|}

The new per-output types for PSBT Version 2 are defined as follows:

{|
! Name
! <tt><keytype></tt>
! <tt><keydata></tt>
! <tt><keydata></tt> Description
! <tt><valuedata></tt>
! <tt><valuedata></tt> Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Output Amount
| <tt>PSBT_OUT_AMOUNT = 0x03</tt>
| None
| No key data
| <tt><64-bit uint></tt>
| 64 bit signed little endian integer representing the output's amount 
in satoshis.
| 2
| 0
| 2
|-
| Output Script
| <tt>PSBT_OUT_SCRIPT = 0x03</tt>
| None
| No key data
| <tt><script></tt>
| The script for this output, also known as the scriptPubKey. Must be 
omitted in PSBTv0. Must be provided in PSBTv2.
| 2
| 0
| 2
|}

===Determining Lock Time===

The nLockTime field of a transaction is determined by inspecting the 
PSBT_GLOBAL_PREFERRED_LOCKTIME and each input's 
PSBT_IN_REQUIRED_TIME_LOCKTIME and PSBT_IN_REQUIRED_HEIGHT_LOCKTIME fields.
If none of the inputs have a PSBT_IN_REQUIRED_TIME_LOCKTIME and 
PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then PSBT_GLOBAL_PREFERRED_LOCKTIME 
must be used.
If PSBT_GLOBAL_PREFERRED_LOCKTIME is not provided, then it is assumed to 
be 0.

If one or more inuts have a PSBT_IN_REQUIRED_TIME_LOCKTIME or 
PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then the field chosen is the one which 
is supported by all of the inputs.
This can be determined by looking at all of the inputs which specify a 
locktime in either of those fields, and choosing the field which is 
present in all of those inputs.
Inputs not specifying a lock time field can take both types of lock 
times, as can those that specify both.
The lock time chosen is then the maximum value of the chosen type of 
lock time.

===Unique Identification===

PSBTv2s can be uniquely identified by constructing an unsigned 
transaction given the information provided in the PSBT and computing the 
transaction ID of that transaction.
Since PSBT_IN_SEQUENCE can be changed by Updaters and Combiners, the 
sequence number in this unsigned transaction must be set to 0 (not 
final, nor the sequence in PSBT_IN_SEQUENCE).
The lock time in this unsigned transaction must be computed as described 
previously.

==Roles==

PSBTv2 introduces new roles and modifies some existing roles.

===Creator===

In PSBTv2, the Creator initializes the PSBT with 0 inputs and 0 outputs.
The PSBT version number is set to 2. The transaction version number must 
be set to at least 2. <ref>'''Why does the transaction version number 
need to be at least 2?''' The transaction version number is part of the 
validation rules for some features such as OP_CHECKSEQUENCEVERIFY. Since 
it is backwards compatible, and there are other ways to disable those 
features (e.g. through sequence numbers), it is easier to require 
transactions be able to support these features than to try to negotiate 
the transaction version number.</ref>
The Creator should also PSBT_GLOBAL_PREFERRED_LOCKTIME.
If the Creator is not also a Constructor and will be giving the PSBT to 
others to add inputs and outputs, the PSBT_GLOBAL_TX_MODIFIABLE field 
must be present and and the Inputs Modifiable and Outputs Modifiable 
flags set appropriately.
If the Creator is a Constructor and no inputs and outputs will be added 
by other entities, PSBT_GLOBAL_TX_MODIFIABLE may be omitted.

===Constructor===

This Constructor is only present for PSBTv2.
Once a Creator initializes the PSBT, a constructor will add inputs and 
outputs.
Before any input or output may be added, the constructor must check the 
PSBT_GLOBAL_TX_MODIFIABLE field.
Inputs may only be added if the Inputs Modifiable flag (first boolean) 
is True.
Outputs may only be added if the Outputs Modifiable flag (second 
boolean) is True.
If PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS is present and its bitvector 
indicates any inputs use SIGHASH_SINGLE, then the same number of inputs 
and outputs must be added before that input and its corresponding outputs.

When an input or output is added, the corresponding 
PSBT_GLOBAL_INPUT_COUNT or PSBT_GLOBAL_OUTPUT_COUNT must be incremeted 
to reflect the number of inputs and outputs in the PSBT.
When an input is added, it must have PSBT_IN_PREVIOUS_TXID and 
PSBT_IN_OUTPUT_INDEX set.
When an output is added, it must have PSBT_OUT_VALUE and 
PSBT_OUT_OUTPUT_SCRIPT set.
If the input has a required timelock, Constructors must set the 
requisite timelock field.
If the input has a required time based timelock, then 
PSBT_IN_REQUIRED_TIME_TIMELOCK.
If the input has a required height based timelock, then 
PSBT_IN_REQUIRED_HEIGHT_TIMELOCK.
If an input has both types of timelocks, then both may be set.
In some cases, an input that can allow both types, but a particular 
branch supporting only one type of timelock will be taken, then the type 
of timelock that will be used can be the only one set.

When adding a new input, the new input must be compatible with the 
timelock types of all existing inputs.
Since Bitcoin requires that a transaction uses only one type of 
timelock, all of the inputs must be able to support the same type of 
timelock.
If the type of timelock is incompatible with the timelock type of the 
other inputs, then the input must not be added.

Since a Constructor may be adding inputs to a PSBT that has inputs with 
existing signatures, the Constructor must be careful to not invalidate 
any existing signatures.
The PSBT_GLOBAL_TX_MODIFIABLE and PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS 
fields caches this information so that constructors do not need to 
inspect every input.
If the PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS bit vector indicates an input 
index uses SIGHASH_SINGLE, the same number of inputs and outputs must be 
added before that input and its corresponding output.
When an input is added, the bit vector, if present, must be expanded to 
include a bit for this input in the correct position.
When there are signatures, in addition to respecting the lock time rules 
described previously, the newly added inputs must not change the lock 
time used by the transaction.
It cannot cause the lock time to change as that will invalidate all 
signatures since they all include the lock time regardless of the 
sighash type.

A Constructor may choose to declare that no further inputs and outputs 
can be added to the transaction by setting the booleans in 
PSBT_GLOBAL_TX_MODIFIABLE to False or by removing this field entirely.

A single entity is likely to be both a Creator and Constructor.

===Updater===

For PSBTv2, an Updater can set the sequence number.

===Signer===

For PSBTv2s, a signer must update the PSBT_GLOBAL_TX_MODIFIABLE and 
PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS fields after signing inputs so
that these fields accurately reflects the state of the PSBT.
If the Signer added a signature that does not use SIGHASH_ANYONECANPAY, 
the Input Modifiable flag must be set to False.
If the Signer added a signature that does not use SIGHASH_NONE, the 
Outputs Modifiable flag must be set to False.
If the Signer added a signature that uses SIGHASH_SINGLE, a 
PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS field must be added if not present, 
and the corresponding bit in its bit vector must be set to 1.

===Transaction Extractor===

For PSBTv2s, the transaction is constructed using the PSBTv2 fields.
The lock time for this transaction is determined as described in the 
Determining Lock Time section.
The Extractor should produce a fully valid, network serialized 
transaction if all inputs are complete.

==Compatibility==

PSBTv2 shares the same gemeric format as PSBTv0 as defined in BIP 174. 
Parsers for PSBTv0 should
be able to deserialize PSBTv2 with only changes to support the new fields.

However PSBTv2 is incompatible with PSBTv0, and vice versa due to the 
use of the PSBT_GLOBAL_VERSION.
This incompatibility is intentional so that PSBT_GLOBAL_UNSIGNED_TX 
could be removed in PSBTv2.
However it is possible to convert a PSBTv2 to a PSBTv0 by creating an 
unsigned
transaction from the PSBTv2 fields.

==Test Vectors==

TBD

==Rationale==

<references/>

==Reference implementation==

The reference implementation of the PSBT format is available at 
https://github.com/achow101/bitcoin/tree/psbt2.


On 12/23/20 4:32 PM, Andrew Chow wrote:
> Hi All,
>
> The full modified BIP can be read at
> https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki.
>
> I will open a PR to the BIPs repo soon after further discussion on this.
>
>
> Andrew
>
> On 12/22/20 3:12 PM, Andrew Chow wrote:
>> Hi All,
>>
>> I have some updates on this after speaking with some people off-list.
>>
>> Firstly, the version number will be set to 2. In most discussions, this
>> proposal was being referred to as PSBT version 2, so it'll be easier and
>> clearer to set the version number to 2.
>>
>> For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
>> there will be 2 of them, one for a time based lock time, and the other
>> for height based. These will be:
>> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>>      * Key: empty
>>      * Value: 32 bit unsigned little endian integer greater than or equal
>> to 500000000 representing the minimum Unix timestamp that this input
>> requires to be set as the transaction's lock time. Must be omitted in
>> PSBTv0, and may be omitted in PSBTv2
>> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>>      * Key: empty
>>      * Value: 32 bit unsigned little endian integer less than 500000000
>> representing the minimum block height that this input requires to be set
>> as the transaction's lock time. Must be omitted in PSBTv0, and may be
>> omitted in PSBTv2.
>>
>> Having two lock time fields is necessary due to the behavior where all
>> inputs must use the same type of lock time (height or time). Thus if an
>> input requires a particular type of lock time, it must set the requisite
>> field. Any new inputs being added must be able to accommodate all
>> existing inputs' lock time type. This means they either must not have a
>> lock time specified (i.e. no OP_CLTV involved), or have branches that
>> allow the acceptance of either type. If an input has a lock time type
>> that is incompatible with the rest of the transaction, it must not be added.
>>
>> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
>> option if no input lock time fields are present. If there are input lock
>> times, all lock time calculations must ignore it.
>>
>> Any role which does lock time calculation will first check if there are
>> input lock time fields. If there are not, it must then check for a
>> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the
>> transaction's lock time. If it does not, the lock time is 0. If there
>> are input lock time fields, it must choose the type which does not
>> invalidate any inputs. The lock time is then determined to be the
>> maximum value of all of the lock time fields for the chosen type.
>>
>>
>> Additionally, I would like to add a new global field:
>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>      * Key: empty
>>      * Value: A single byte as a boolean. 0 for False, 1 for True. All
>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>> in PSBTv2.
>>
>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>> outputs can be added to the PSBT. This flag may be set to True when
>> inputs and outputs are being updated, signed, and finalized. However
>> care must be taken when there are existing signatures. If this field is
>> omitted or set to False, no further inputs and outputs may be added to
>> the PSBT.
>>
>> Several rules must be followed to ensure that adding additional inputs
>> and outputs will not invalidate existing signatures. First, an input or
>> output adder must check for any existing signatures in all of the other
>> inputs. If there are none, the input or output may be added in any
>> position. If there are one or more signatures, each signature's sighash
>> type must be examined. Inputs may only be added if all existing
>> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
>> existing signatures use SIGHASH_NONE. If an input has a signature using
>> SIGHASH_SINGLE, the same number of inputs and outputs must be added
>> before that input and it's corresponding output. For all other sighash
>> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
>> outputs may be added to the PSBT. Specific exceptions can be made in the
>> future for additional sighash types.
>>
>> Furthermore, these newly added inputs must follow additional lock time
>> rules. Because all signatures, regardless of sighash type, sign the
>> transaction lock time, newly added inputs when there are existing
>> signatures must have the same type of lock time used in the transaction,
>> and must be less than or equal to the transaction lock time. It must not
>> cause the transaction lock time to change, otherwise the signatures will
>> be invalidated.
>>
>>
>> Lastly, to uniquely identify transactions for combiners, a txid can be
>> computed from the information present in the PSBT. Internally, combiners
>> can create an unsigned transaction given the transaction version, the
>> input prevouts, the outputs, and the computed locktime. This can then be
>> used to calculate a txid and thus used as a way to identify PSBTs.
>> Combiners will need to do this for all version 2 PSBTs in order to avoid
>> combining distinct transactions.
>>
>>
>> Andrew Chow
>>
>> On 12/9/20 5:25 PM, Andrew Chow wrote:
>>> Hi All,
>>>
>>> I would like to propose a new PSBT version that addresses a few
>>> deficiencies in the current PSBT v0. As this will be backwards
>>> incompatible, a new PSBT version will be used, v1.
>>>
>>> The primary change is to truly have all input and output data for each
>>> in their respective maps. Instead of having to parse an unsigned
>>> transaction and lookup some data from there, and other data from the
>>> correct map, all of the data for an input will be contained in its map.
>>> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
>>> Thus I propose that the following fields be added:
>>>
>>> Global:
>>> * PSBT_GLOBAL_TX_VERSION = 0x02
>>>       * Key: empty
>>>       * Value: 32-bit little endian unsigned integer for the transaction
>>> version number. Must be provided in PSBT v1 and omitted in v0.
>>> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>>>       * Key: empty
>>>       * Value: 32 bit little endian unsigned integer for the preferred
>>> transaction lock time. Must be omitted in PSBT v0. May be provided in
>>> PSBT v1, assumed to be 0 if not provided.
>>> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>>>       * Key: empty
>>>       * Value: Compact size unsigned integer. Number of inputs in this
>>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>>> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>>>       * Key: empty
>>>       * Value: Compact size unsigned integer. Number of outputs in this
>>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>>>
>>> Input:
>>> * PSBT_IN_PREVIOUS_TXID = 0x0e
>>>       * Key: empty
>>>       * Value: 32 byte txid of the previous transaction whose output at
>>> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
>>> omitted in v0.
>>> * PSBT_IN_OUTPUT_INDEX = 0x0f
>>>       * Key: empty
>>>       * Value: 32 bit little endian integer for the index of the output
>>> being spent. Must be provided in PSBT v1 and omitted in v0.
>>> * PSBT_IN_SEQUENCE = 0x0f
>>>       * Key: empty
>>>       * Value: 32 bit unsigned little endian integer for the sequence
>>> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
>>> to be max sequence (0xffffffff) if not provided.
>>> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>>>       * Key: empty
>>>       * Value: 32 bit unsigned little endian integer for the lock time that
>>> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
>>> v1, assumed to be 0 if not provided.
>>>
>>> Output:
>>> * PSBT_OUT_VALUE = 0x03
>>>       * Key: empty
>>>       * Value: 64-bit unsigned little endian integer for the output's
>>> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
>>> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>>>       * Key: empty
>>>       * Value: The script for this output. Otherwise known as the
>>> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>>>
>>> This change allows for PSBT to be used in the construction of
>>> transactions. With these new fields, inputs and outputs can be added as
>>> needed. One caveat is that there is no longer a unique transaction
>>> identifier so more care must be taken when combining PSBTs.
>>> Additionally, adding new inputs and outputs must be done such that
>>> signatures are not invalidated. This may be harder to specify.
>>>
>>> An important thing to note in this proposal are the fields
>>> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
>>> transaction only has a single locktime yet a PSBT may have multiple
>>> locktimes. To choose the locktime for the transaction, finalizers must
>>> choose the maximum of all of the *_LOCKTIME fields.
>>> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
>>> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
>>> be set. This field allows finalizers to choose a locktime that is high
>>> enough for all inputs without needing to understand the scripts
>>> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
>>> no inputs require a particular locktime.
>>>
>>> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
>>> needs the version number bump to enforce backwards incompatibility.
>>> However once the inputs and outputs of a PSBT are decided, a PSBT could
>>> be "downgraded" back to v0 by creating the unsigned transaction from the
>>> above fields, and then dropping these new fields.
>>>
>>> If the list finds that these changes are reasonable, I will write a PR
>>> to modify BIP 174 to incorporate them.
>>>
>>> Thanks,
>>> Andrew Chow

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow]

While working on the reference implementation for this, it occurred to 
me that the Inputs Modifiable flag needs to be more than just a boolean.

If there are existing signatures in the PSBT, then any added inputs 
cannot change the transaction's locktime as all signatures, regardless 
of sighash type, commit to the locktime. Additionally if an input with a 
signature is being added, it also needs to set the locktime for the 
transaction.

It also seems like the SIGHASH_SINGLE bitmap is unnecessary. Signers can 
instead iterate all inputs, check for existing signatures, and extract 
the sighash byte from those signatures to determine whether any are 
SIGHASH_SINGLE. This bitmap doesn't seem to provide much benefit and 
also causes headaches for implementation. So I've decided to remove it.

But it is still useful to know that there are SIGHASH_SINGLE inputs and 
that iteration of the inputs vector will be necessary. It is also useful 
to know that there are already some signatures in the transaction so the 
locktime must be preserved. Thus I would like to change 
PSBT_GLOBAL_TX_MODIFIABLE to include those. I propose making 
PSBT_GLOBAL_TX_MODIFIABLE an 8 bit unsigned little endian integer that 
is treated as a bit field. If bit 0 is set, inputs may be added. This 
will be the Inputs Modifiable flag. If bit 1 is set, outputs may be 
added. This will be the Outputs Modifiable flag. If bit 2 is set, the 
transaction contains signatures and locktime must be preserved. This 
will be the Has Signatures flag. If bit 3 is set, the transaction 
contains SIGHASH_SINGLE inputs and their index pairings must be 
preserved. This will be the Has SIGHASH_SINGLE flag.

Changing PSBT_GLOBAL_TX_MODIFIABLE to a bitfield like this allows us to 
include more conditions that need to be considered when adding inputs 
and outputs. I think these are all of the conditions for now, but with 8 
bits, there is still some space for additional conditions in the future. 
Perhaps it should be changed to be larger if we think there will be more 
conditions, but I think that is unlikely.


Andrew

On 1/15/21 12:28 PM, Andrew Chow wrote:
> Hi All,
>
> I've made some reorganization changes to the way that new PSBT versions
> should be handled in BIP 174 (see
> https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted
> as a separate BIP. The full document can be read at
> https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I
> have also included it in this email.
>
> I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and
> made a few modifications. First, the field will be named
> PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and
> outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as
> a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most
> PSBTs to not have to carry around a useless bitmap.
>
> Andrew
>
> ***
>
> <pre>
>     BIP: PSBTv2
>     Layer: Applications
>     Title: PSBT Version 2
>     Author: Andrew Chow <achow101@gmail.com>
>     Comments-Summary: No comments yet.
>     Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2
>     Status: Draft
>     Type: Standards Track
>     Created: 2021-01-14
>     License: BSD-2-Clause
> </pre>
>
> ==Introduction==
>
> ===Abstract===
>
> This document proposes a second version of the Partially Signed Bitcoin
> Transaction format
> described in BIP 174 which allows for inputs and outputs to be added to
> the PSBT after creation.
>
> ===Copyright===
>
> This BIP is licensed under the 2-clause BSD license.
>
> ===Motivation===
>
> Partially Signed Bitcoin Transaction Version 0 as described in BIP 174
> is unable to have new
> inputs and outputs be added to the transaction. The fixed global
> unsigned transaction
> cannot be changed which prevents any additional inputs or outputs to be
> added.
> PSBT Version 2 is intended to rectify this problem.
>
> An additional benficial side effect is that all information for a given
> input or output will be
> provided by its <tt><input-map></tt> or <tt><output-map></tt>. With
> Version 0, to retrieve
> all of the information for an input or output, data would need to be
> found in two locations:
> the <tt><input-map></tt>/<tt><output-map></tt> and the global unsigned
> transaction. PSBT
> Version 2 now moves all related information to one place.
>
> ==Specification==
>
> PSBT Version 2 (PSBTv2) only specifies new fields and field
> inclusion/exclusion requirements.
>
> <tt>PSBT_GLOBAL_UNSIGNED_TX</tt> must be excluded in PSBTv2.
> <tt>PSBT_GLOBAL_VERSION</tt> must be included in PSBTv2 and set to
> version number 2<ref>'''What happened to version number 1?'''
> Version number 1 is skipped because PSBT Version 0 has been colloquially
> referred to as version 1. Originally this BIP was to be
> version 1, but because it has been colloquially referred to as version 2
> during its design phrase, it was decided to change the
> version number to 2 so that there would not be any confusion</ref>.
>
> The new global types for PSBT Version 2 are as follows:
>
> {|
> ! Name
> ! <tt><keytype></tt>
> ! <tt><keydata></tt>
> ! <tt><keydata></tt> Description
> ! <tt><valuedata></tt>
> ! <tt><valuedata></tt> Description
> ! Versions Requiring Inclusion
> ! Versions Requiring Exclusion
> ! Versions Allowing Inclusion
> |-
> | Transaction Version
> | <tt>PSBT_GLOBAL_TX_VERSION = 0x02</tt>
> | None
> | No key data
> | <tt><32-bit uint></tt>
> | The 32-bit little endian signed integer representing the version
> number of the transaction being created. Note that this is not the same
> as the PSBT version number specified by the PSBT_GLOBAL_VERSION field.
> | 2
> | 0
> | 2
> |-
> | Fallback Locktime
> | <tt>PSBT_GLOBAL_FALLBACK_LOCKTIME = 0x03</tt>
> | None
> | No key data
> | <tt><32-bit uint></tt>
> | The 32-bit little endian unsigned integer representing the transaction
> locktime to use if no inputs specify a required locktime.
> |
> | 0
> | 2
> |-
> | Input Count
> | <tt>PSBT_GLOBAL_INPUT_COUNT = 0x04</tt>
> | None
> | No key data
> | <tt><compact size uint></tt>
> | Compact size unsigned integer representing the number of inputs in
> this PSBT.
> | 2
> | 0
> | 2
> |-
> | Output Count
> | <tt>PSBT_GLOBAL_OUTPUT_COUNT = 0x05</tt>
> | None
> | No key data
> | <tt><compact size uint></tt>
> | Compact size unsigned integer representing the number of outputs in
> this PSBT.
> | 2
> | 0
> | 2
> |-
> | Transaction Modifiable Flags
> | <tt>PSBT_GLOBAL_TX_MODIFIABLE = 0x06</tt>
> | None
> | No key data
> | <tt><single byte boolean> <single byte boolean></tt>
> | A single byte boolean (0 for False, 1 for True) representing whether
> inputs can be modified, referred to as the Inputs Modifiable Flag. This
> is followed by a single byte boolean representing whether outputs can be
> modified, referred to as the Outputs Modifiable Flag.
> |
> | 0
> | 2
> |-
> | SIGHASH_SINGLE Inputs
> | <tt>PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS = 0x07</tt>
> | None
> | No key data
> | <tt><bit vector></tt>
> | A bit vector representing which input indexes use SIGHASH_SINGLE. If
> the bit for an index is set to 1, then the input and output pair at that
> index are tied together with SIGHASH_SINGLE and must be moved together.
> |
> | 0
> | 2
> |}
>
> The new per-input types for PSBT Version 2 are defined as follows:
>
> {|
> ! Name
> ! <tt><keytype></tt>
> ! <tt><keydata></tt>
> ! <tt><keydata></tt> Description
> ! <tt><valuedata></tt>
> ! <tt><valuedata></tt> Description
> ! Versions Requiring Inclusion
> ! Versions Requiring Exclusion
> ! Versions Allowing Inclusion
> |-
> | Previous TXID
> | <tt>PSBT_IN_PREVIOUS_TXID = 0x0e</tt>
> | None
> | No key data
> | <tt><txid></tt>
> | 32 byte txid of the previous transaction whose output at
> PSBT_IN_OUTPUT_INDEX is being spent.
> | 2
> | 0
> | 2
> |-
> | Spent Output Index
> | <tt>PSBT_IN_OUTPUT_INDEX = 0x0f</tt>
> | None
> | No key data
> | <tt><32-bit uint></tt>
> | 32 bit little endian integer representing the index of the output
> being spent in the transaction with the txid of PSBT_IN_PREVIOUS_TXID.
> | 2
> | 0
> | 2
> |-
> | Sequence Number
> | <tt>PSBT_IN_SEQUENCE = 0x10</tt>
> | None
> | No key data
> | <tt><32-bit uint></tt>
> | The 32 bit unsigned little endian integer for the sequence number of
> this input. If omitted, the sequence number is assumed to be the final
> sequence number (0xffffffff).
> |
> | 0
> | 2
> |-
> | Required Time-based Locktime
> | <tt>PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x11</tt>
> | None
> | No key data
> | <tt><32-bit uint></tt>
> | 32 bit unsigned little endian integer greater than or equal to
> 500000000 representing the minimum Unix timestamp that this input
> requires to be set as the transaction's lock time.
> |
> | 0
> | 2
> |-
> | Required Height-based Locktime
> | <tt>PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x12</tt>
> | None
> | No key data
> | <tt><32-bit uiht></tt>
> | 32 bit unsigned little endian integer less than 500000000 representing
> the minimum block height that this input requires to be set as the
> transaction's lock time.
> |
> | 0
> | 2
> |}
>
> The new per-output types for PSBT Version 2 are defined as follows:
>
> {|
> ! Name
> ! <tt><keytype></tt>
> ! <tt><keydata></tt>
> ! <tt><keydata></tt> Description
> ! <tt><valuedata></tt>
> ! <tt><valuedata></tt> Description
> ! Versions Requiring Inclusion
> ! Versions Requiring Exclusion
> ! Versions Allowing Inclusion
> |-
> | Output Amount
> | <tt>PSBT_OUT_AMOUNT = 0x03</tt>
> | None
> | No key data
> | <tt><64-bit uint></tt>
> | 64 bit signed little endian integer representing the output's amount
> in satoshis.
> | 2
> | 0
> | 2
> |-
> | Output Script
> | <tt>PSBT_OUT_SCRIPT = 0x03</tt>
> | None
> | No key data
> | <tt><script></tt>
> | The script for this output, also known as the scriptPubKey. Must be
> omitted in PSBTv0. Must be provided in PSBTv2.
> | 2
> | 0
> | 2
> |}
>
> ===Determining Lock Time===
>
> The nLockTime field of a transaction is determined by inspecting the
> PSBT_GLOBAL_PREFERRED_LOCKTIME and each input's
> PSBT_IN_REQUIRED_TIME_LOCKTIME and PSBT_IN_REQUIRED_HEIGHT_LOCKTIME fields.
> If none of the inputs have a PSBT_IN_REQUIRED_TIME_LOCKTIME and
> PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then PSBT_GLOBAL_PREFERRED_LOCKTIME
> must be used.
> If PSBT_GLOBAL_PREFERRED_LOCKTIME is not provided, then it is assumed to
> be 0.
>
> If one or more inuts have a PSBT_IN_REQUIRED_TIME_LOCKTIME or
> PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then the field chosen is the one which
> is supported by all of the inputs.
> This can be determined by looking at all of the inputs which specify a
> locktime in either of those fields, and choosing the field which is
> present in all of those inputs.
> Inputs not specifying a lock time field can take both types of lock
> times, as can those that specify both.
> The lock time chosen is then the maximum value of the chosen type of
> lock time.
>
> ===Unique Identification===
>
> PSBTv2s can be uniquely identified by constructing an unsigned
> transaction given the information provided in the PSBT and computing the
> transaction ID of that transaction.
> Since PSBT_IN_SEQUENCE can be changed by Updaters and Combiners, the
> sequence number in this unsigned transaction must be set to 0 (not
> final, nor the sequence in PSBT_IN_SEQUENCE).
> The lock time in this unsigned transaction must be computed as described
> previously.
>
> ==Roles==
>
> PSBTv2 introduces new roles and modifies some existing roles.
>
> ===Creator===
>
> In PSBTv2, the Creator initializes the PSBT with 0 inputs and 0 outputs.
> The PSBT version number is set to 2. The transaction version number must
> be set to at least 2. <ref>'''Why does the transaction version number
> need to be at least 2?''' The transaction version number is part of the
> validation rules for some features such as OP_CHECKSEQUENCEVERIFY. Since
> it is backwards compatible, and there are other ways to disable those
> features (e.g. through sequence numbers), it is easier to require
> transactions be able to support these features than to try to negotiate
> the transaction version number.</ref>
> The Creator should also PSBT_GLOBAL_PREFERRED_LOCKTIME.
> If the Creator is not also a Constructor and will be giving the PSBT to
> others to add inputs and outputs, the PSBT_GLOBAL_TX_MODIFIABLE field
> must be present and and the Inputs Modifiable and Outputs Modifiable
> flags set appropriately.
> If the Creator is a Constructor and no inputs and outputs will be added
> by other entities, PSBT_GLOBAL_TX_MODIFIABLE may be omitted.
>
> ===Constructor===
>
> This Constructor is only present for PSBTv2.
> Once a Creator initializes the PSBT, a constructor will add inputs and
> outputs.
> Before any input or output may be added, the constructor must check the
> PSBT_GLOBAL_TX_MODIFIABLE field.
> Inputs may only be added if the Inputs Modifiable flag (first boolean)
> is True.
> Outputs may only be added if the Outputs Modifiable flag (second
> boolean) is True.
> If PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS is present and its bitvector
> indicates any inputs use SIGHASH_SINGLE, then the same number of inputs
> and outputs must be added before that input and its corresponding outputs.
>
> When an input or output is added, the corresponding
> PSBT_GLOBAL_INPUT_COUNT or PSBT_GLOBAL_OUTPUT_COUNT must be incremeted
> to reflect the number of inputs and outputs in the PSBT.
> When an input is added, it must have PSBT_IN_PREVIOUS_TXID and
> PSBT_IN_OUTPUT_INDEX set.
> When an output is added, it must have PSBT_OUT_VALUE and
> PSBT_OUT_OUTPUT_SCRIPT set.
> If the input has a required timelock, Constructors must set the
> requisite timelock field.
> If the input has a required time based timelock, then
> PSBT_IN_REQUIRED_TIME_TIMELOCK.
> If the input has a required height based timelock, then
> PSBT_IN_REQUIRED_HEIGHT_TIMELOCK.
> If an input has both types of timelocks, then both may be set.
> In some cases, an input that can allow both types, but a particular
> branch supporting only one type of timelock will be taken, then the type
> of timelock that will be used can be the only one set.
>
> When adding a new input, the new input must be compatible with the
> timelock types of all existing inputs.
> Since Bitcoin requires that a transaction uses only one type of
> timelock, all of the inputs must be able to support the same type of
> timelock.
> If the type of timelock is incompatible with the timelock type of the
> other inputs, then the input must not be added.
>
> Since a Constructor may be adding inputs to a PSBT that has inputs with
> existing signatures, the Constructor must be careful to not invalidate
> any existing signatures.
> The PSBT_GLOBAL_TX_MODIFIABLE and PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS
> fields caches this information so that constructors do not need to
> inspect every input.
> If the PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS bit vector indicates an input
> index uses SIGHASH_SINGLE, the same number of inputs and outputs must be
> added before that input and its corresponding output.
> When an input is added, the bit vector, if present, must be expanded to
> include a bit for this input in the correct position.
> When there are signatures, in addition to respecting the lock time rules
> described previously, the newly added inputs must not change the lock
> time used by the transaction.
> It cannot cause the lock time to change as that will invalidate all
> signatures since they all include the lock time regardless of the
> sighash type.
>
> A Constructor may choose to declare that no further inputs and outputs
> can be added to the transaction by setting the booleans in
> PSBT_GLOBAL_TX_MODIFIABLE to False or by removing this field entirely.
>
> A single entity is likely to be both a Creator and Constructor.
>
> ===Updater===
>
> For PSBTv2, an Updater can set the sequence number.
>
> ===Signer===
>
> For PSBTv2s, a signer must update the PSBT_GLOBAL_TX_MODIFIABLE and
> PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS fields after signing inputs so
> that these fields accurately reflects the state of the PSBT.
> If the Signer added a signature that does not use SIGHASH_ANYONECANPAY,
> the Input Modifiable flag must be set to False.
> If the Signer added a signature that does not use SIGHASH_NONE, the
> Outputs Modifiable flag must be set to False.
> If the Signer added a signature that uses SIGHASH_SINGLE, a
> PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS field must be added if not present,
> and the corresponding bit in its bit vector must be set to 1.
>
> ===Transaction Extractor===
>
> For PSBTv2s, the transaction is constructed using the PSBTv2 fields.
> The lock time for this transaction is determined as described in the
> Determining Lock Time section.
> The Extractor should produce a fully valid, network serialized
> transaction if all inputs are complete.
>
> ==Compatibility==
>
> PSBTv2 shares the same gemeric format as PSBTv0 as defined in BIP 174.
> Parsers for PSBTv0 should
> be able to deserialize PSBTv2 with only changes to support the new fields.
>
> However PSBTv2 is incompatible with PSBTv0, and vice versa due to the
> use of the PSBT_GLOBAL_VERSION.
> This incompatibility is intentional so that PSBT_GLOBAL_UNSIGNED_TX
> could be removed in PSBTv2.
> However it is possible to convert a PSBTv2 to a PSBTv0 by creating an
> unsigned
> transaction from the PSBTv2 fields.
>
> ==Test Vectors==
>
> TBD
>
> ==Rationale==
>
> <references/>
>
> ==Reference implementation==
>
> The reference implementation of the PSBT format is available at
> https://github.com/achow101/bitcoin/tree/psbt2.
>
>
> On 12/23/20 4:32 PM, Andrew Chow wrote:
>> Hi All,
>>
>> The full modified BIP can be read at
>> https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki.
>>
>> I will open a PR to the BIPs repo soon after further discussion on this.
>>
>>
>> Andrew
>>
>> On 12/22/20 3:12 PM, Andrew Chow wrote:
>>> Hi All,
>>>
>>> I have some updates on this after speaking with some people off-list.
>>>
>>> Firstly, the version number will be set to 2. In most discussions, this
>>> proposal was being referred to as PSBT version 2, so it'll be easier and
>>> clearer to set the version number to 2.
>>>
>>> For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
>>> there will be 2 of them, one for a time based lock time, and the other
>>> for height based. These will be:
>>> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>>>       * Key: empty
>>>       * Value: 32 bit unsigned little endian integer greater than or equal
>>> to 500000000 representing the minimum Unix timestamp that this input
>>> requires to be set as the transaction's lock time. Must be omitted in
>>> PSBTv0, and may be omitted in PSBTv2
>>> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>>>       * Key: empty
>>>       * Value: 32 bit unsigned little endian integer less than 500000000
>>> representing the minimum block height that this input requires to be set
>>> as the transaction's lock time. Must be omitted in PSBTv0, and may be
>>> omitted in PSBTv2.
>>>
>>> Having two lock time fields is necessary due to the behavior where all
>>> inputs must use the same type of lock time (height or time). Thus if an
>>> input requires a particular type of lock time, it must set the requisite
>>> field. Any new inputs being added must be able to accommodate all
>>> existing inputs' lock time type. This means they either must not have a
>>> lock time specified (i.e. no OP_CLTV involved), or have branches that
>>> allow the acceptance of either type. If an input has a lock time type
>>> that is incompatible with the rest of the transaction, it must not be added.
>>>
>>> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
>>> option if no input lock time fields are present. If there are input lock
>>> times, all lock time calculations must ignore it.
>>>
>>> Any role which does lock time calculation will first check if there are
>>> input lock time fields. If there are not, it must then check for a
>>> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the
>>> transaction's lock time. If it does not, the lock time is 0. If there
>>> are input lock time fields, it must choose the type which does not
>>> invalidate any inputs. The lock time is then determined to be the
>>> maximum value of all of the lock time fields for the chosen type.
>>>
>>>
>>> Additionally, I would like to add a new global field:
>>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>>       * Key: empty
>>>       * Value: A single byte as a boolean. 0 for False, 1 for True. All
>>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>>> in PSBTv2.
>>>
>>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>>> outputs can be added to the PSBT. This flag may be set to True when
>>> inputs and outputs are being updated, signed, and finalized. However
>>> care must be taken when there are existing signatures. If this field is
>>> omitted or set to False, no further inputs and outputs may be added to
>>> the PSBT.
>>>
>>> Several rules must be followed to ensure that adding additional inputs
>>> and outputs will not invalidate existing signatures. First, an input or
>>> output adder must check for any existing signatures in all of the other
>>> inputs. If there are none, the input or output may be added in any
>>> position. If there are one or more signatures, each signature's sighash
>>> type must be examined. Inputs may only be added if all existing
>>> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
>>> existing signatures use SIGHASH_NONE. If an input has a signature using
>>> SIGHASH_SINGLE, the same number of inputs and outputs must be added
>>> before that input and it's corresponding output. For all other sighash
>>> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
>>> outputs may be added to the PSBT. Specific exceptions can be made in the
>>> future for additional sighash types.
>>>
>>> Furthermore, these newly added inputs must follow additional lock time
>>> rules. Because all signatures, regardless of sighash type, sign the
>>> transaction lock time, newly added inputs when there are existing
>>> signatures must have the same type of lock time used in the transaction,
>>> and must be less than or equal to the transaction lock time. It must not
>>> cause the transaction lock time to change, otherwise the signatures will
>>> be invalidated.
>>>
>>>
>>> Lastly, to uniquely identify transactions for combiners, a txid can be
>>> computed from the information present in the PSBT. Internally, combiners
>>> can create an unsigned transaction given the transaction version, the
>>> input prevouts, the outputs, and the computed locktime. This can then be
>>> used to calculate a txid and thus used as a way to identify PSBTs.
>>> Combiners will need to do this for all version 2 PSBTs in order to avoid
>>> combining distinct transactions.
>>>
>>>
>>> Andrew Chow
>>>
>>> On 12/9/20 5:25 PM, Andrew Chow wrote:
>>>> Hi All,
>>>>
>>>> I would like to propose a new PSBT version that addresses a few
>>>> deficiencies in the current PSBT v0. As this will be backwards
>>>> incompatible, a new PSBT version will be used, v1.
>>>>
>>>> The primary change is to truly have all input and output data for each
>>>> in their respective maps. Instead of having to parse an unsigned
>>>> transaction and lookup some data from there, and other data from the
>>>> correct map, all of the data for an input will be contained in its map.
>>>> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
>>>> Thus I propose that the following fields be added:
>>>>
>>>> Global:
>>>> * PSBT_GLOBAL_TX_VERSION = 0x02
>>>>        * Key: empty
>>>>        * Value: 32-bit little endian unsigned integer for the transaction
>>>> version number. Must be provided in PSBT v1 and omitted in v0.
>>>> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>>>>        * Key: empty
>>>>        * Value: 32 bit little endian unsigned integer for the preferred
>>>> transaction lock time. Must be omitted in PSBT v0. May be provided in
>>>> PSBT v1, assumed to be 0 if not provided.
>>>> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>>>>        * Key: empty
>>>>        * Value: Compact size unsigned integer. Number of inputs in this
>>>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>>>> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>>>>        * Key: empty
>>>>        * Value: Compact size unsigned integer. Number of outputs in this
>>>> PSBT. Must be provided in PSBT v1 and omitted in v0.
>>>>
>>>> Input:
>>>> * PSBT_IN_PREVIOUS_TXID = 0x0e
>>>>        * Key: empty
>>>>        * Value: 32 byte txid of the previous transaction whose output at
>>>> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
>>>> omitted in v0.
>>>> * PSBT_IN_OUTPUT_INDEX = 0x0f
>>>>        * Key: empty
>>>>        * Value: 32 bit little endian integer for the index of the output
>>>> being spent. Must be provided in PSBT v1 and omitted in v0.
>>>> * PSBT_IN_SEQUENCE = 0x0f
>>>>        * Key: empty
>>>>        * Value: 32 bit unsigned little endian integer for the sequence
>>>> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
>>>> to be max sequence (0xffffffff) if not provided.
>>>> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>>>>        * Key: empty
>>>>        * Value: 32 bit unsigned little endian integer for the lock time that
>>>> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
>>>> v1, assumed to be 0 if not provided.
>>>>
>>>> Output:
>>>> * PSBT_OUT_VALUE = 0x03
>>>>        * Key: empty
>>>>        * Value: 64-bit unsigned little endian integer for the output's
>>>> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
>>>> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>>>>        * Key: empty
>>>>        * Value: The script for this output. Otherwise known as the
>>>> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>>>>
>>>> This change allows for PSBT to be used in the construction of
>>>> transactions. With these new fields, inputs and outputs can be added as
>>>> needed. One caveat is that there is no longer a unique transaction
>>>> identifier so more care must be taken when combining PSBTs.
>>>> Additionally, adding new inputs and outputs must be done such that
>>>> signatures are not invalidated. This may be harder to specify.
>>>>
>>>> An important thing to note in this proposal are the fields
>>>> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
>>>> transaction only has a single locktime yet a PSBT may have multiple
>>>> locktimes. To choose the locktime for the transaction, finalizers must
>>>> choose the maximum of all of the *_LOCKTIME fields.
>>>> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
>>>> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
>>>> be set. This field allows finalizers to choose a locktime that is high
>>>> enough for all inputs without needing to understand the scripts
>>>> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
>>>> no inputs require a particular locktime.
>>>>
>>>> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
>>>> needs the version number bump to enforce backwards incompatibility.
>>>> However once the inputs and outputs of a PSBT are decided, a PSBT could
>>>> be "downgraded" back to v0 by creating the unsigned transaction from the
>>>> above fields, and then dropping these new fields.
>>>>
>>>> If the list finds that these changes are reasonable, I will write a PR
>>>> to modify BIP 174 to incorporate them.
>>>>
>>>> Thanks,
>>>> Andrew Chow

Re: [bitcoin-dev] New PSBT version proposal

[Lloyd Fournier]

[-- Attachment #1: Type: text/plain, Size: 5238 bytes --]

Hi Andrew & all,

I've been working with PSBTs for a little while now. FWIW I agree with the
change of removing the global tx and having the input/output data stored
together in the new unified structures.

One thing I've been wondering about is how output descriptors could fit
into PSBTs. They are useful since they allow you to determine the maximum
satisfaction weight for inputs so you can properly align fees as things get
added. I haven't seen any discussion about including them in this revision.
Is it simply a matter of time before they make it into a subsequent PSBT
spec or is there something I'm missing conceptually?

Cheers,

LL

On Thu, 10 Dec 2020 at 09:33, Andrew Chow via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi All,
>
> I would like to propose a new PSBT version that addresses a few
> deficiencies in the current PSBT v0. As this will be backwards
> incompatible, a new PSBT version will be used, v1.
>
> The primary change is to truly have all input and output data for each
> in their respective maps. Instead of having to parse an unsigned
> transaction and lookup some data from there, and other data from the
> correct map, all of the data for an input will be contained in its map.
> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
> Thus I propose that the following fields be added:
>
> Global:
> * PSBT_GLOBAL_TX_VERSION = 0x02
>    * Key: empty
>    * Value: 32-bit little endian unsigned integer for the transaction
> version number. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
>    * Key: empty
>    * Value: 32 bit little endian unsigned integer for the preferred
> transaction lock time. Must be omitted in PSBT v0. May be provided in
> PSBT v1, assumed to be 0 if not provided.
> * PSBT_GLOBAL_INPUT_COUNT = 0x04
>    * Key: empty
>    * Value: Compact size unsigned integer. Number of inputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05
>    * Key: empty
>    * Value: Compact size unsigned integer. Number of outputs in this
> PSBT. Must be provided in PSBT v1 and omitted in v0.
>
> Input:
> * PSBT_IN_PREVIOUS_TXID = 0x0e
>    * Key: empty
>    * Value: 32 byte txid of the previous transaction whose output at
> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
> omitted in v0.
> * PSBT_IN_OUTPUT_INDEX = 0x0f
>    * Key: empty
>    * Value: 32 bit little endian integer for the index of the output
> being spent. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_IN_SEQUENCE = 0x0f
>    * Key: empty
>    * Value: 32 bit unsigned little endian integer for the sequence
> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
> to be max sequence (0xffffffff) if not provided.
> * PSBT_IN_REQUIRED_LOCKTIME = 0x10
>    * Key: empty
>    * Value: 32 bit unsigned little endian integer for the lock time that
> this input requires. Must be omitted in PSBT v0. May be provided in PSBT
> v1, assumed to be 0 if not provided.
>
> Output:
> * PSBT_OUT_VALUE = 0x03
>    * Key: empty
>    * Value: 64-bit unsigned little endian integer for the output's
> amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
> * PSBT_OUT_OUTPUT_SCRIPT = 0x04
>    * Key: empty
>    * Value: The script for this output. Otherwise known as the
> scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
>
> This change allows for PSBT to be used in the construction of
> transactions. With these new fields, inputs and outputs can be added as
> needed. One caveat is that there is no longer a unique transaction
> identifier so more care must be taken when combining PSBTs.
> Additionally, adding new inputs and outputs must be done such that
> signatures are not invalidated. This may be harder to specify.
>
> An important thing to note in this proposal are the fields
> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
> transaction only has a single locktime yet a PSBT may have multiple
> locktimes. To choose the locktime for the transaction, finalizers must
> choose the maximum of all of the *_LOCKTIME fields.
> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
> be set. This field allows finalizers to choose a locktime that is high
> enough for all inputs without needing to understand the scripts
> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
> no inputs require a particular locktime.
>
> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
> needs the version number bump to enforce backwards incompatibility.
> However once the inputs and outputs of a PSBT are decided, a PSBT could
> be "downgraded" back to v0 by creating the unsigned transaction from the
> above fields, and then dropping these new fields.
>
> If the list finds that these changes are reasonable, I will write a PR
> to modify BIP 174 to incorporate them.
>
> Thanks,
> Andrew Chow
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 6248 bytes --]

Re: [bitcoin-dev] New PSBT version proposal

[Lloyd Fournier]

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]

On Wed, 10 Mar 2021 at 11:20, Lloyd Fournier <lloyd.fourn@gmail.com> wrote:

> Hi Andrew & all,
>
> I've been working with PSBTs for a little while now. FWIW I agree with the
> change of removing the global tx and having the input/output data stored
> together in the new unified structures.
>
> One thing I've been wondering about is how output descriptors could fit
> into PSBTs. They are useful since they allow you to determine the maximum
> satisfaction weight for inputs so you can properly align fees as things get
> added. I haven't seen any discussion about including them in this revision.
> Is it simply a matter of time before they make it into a subsequent PSBT
> spec or is there something I'm missing conceptually?
>


Sipa replied to me off list some time ago and explained what I was missing.
PSBTs have all the information you could want from a descriptor already.
For example the maximum satisfaction weight can be determined from the
witness/redeem script (I had forgot these fields existed). Therefore
descriptors are more useful in higher level applications while PSBTs are
useful for communicating with signing devices. Therefore there is no reason
for PSBTs to support descriptors.

LL

[-- Attachment #2: Type: text/html, Size: 1684 bytes --]