From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 077A7C0893 for ; Wed, 23 Dec 2020 21:30:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 017118738B for ; Wed, 23 Dec 2020 21:30:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uZhy4aYXVlSR for ; Wed, 23 Dec 2020 21:30:22 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-41103.protonmail.ch (mail-41103.protonmail.ch [185.70.41.103]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3EA0D87388 for ; Wed, 23 Dec 2020 21:30:22 +0000 (UTC) Received: from mail-03.mail-europe.com (mail-03.mail-europe.com [91.134.188.129]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail-41103.protonmail.ch (Postfix) with ESMTPS id 2EF452003144 for ; Wed, 23 Dec 2020 21:30:20 +0000 (UTC) Authentication-Results: mail-41103.protonmail.ch; dkim=pass (2048-bit key) header.d=achow101.com header.i=@achow101.com header.b="Z6xWIBX6" Date: Wed, 23 Dec 2020 21:30:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=achow101.com; s=protonmail2; t=1608759007; bh=rseYu2K+tdlHmaTj5sfUGJLNzrSLu1K/ohmN1Sv7a/4=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=Z6xWIBX6Tgyn/+OfZam9b1mtwuIhJVVzvj5fW9emEOpy252PtnEVPzpBJFX7xMxzY ORxTLtGSwL+bGmWIQoEtAPOnA6MkMkC3R988NlOFdyjfKSw4R0Qrcdngr5JWiAWeSm jZej5SjirsOEXW06XI2ID2BqrS0NkqrV+Qo63VF/jeeprZEO3f6Gi4r/f4i8wB9Pz7 yYjH8YS1feot16DoxCj3kL/nxr2Kug0VkS/Uggeoa5kQtkCemyCHWpWFcjSAcBUr5t 7hnZLOBxEnaW7Zof0XWrheVXG+M8VVGJDz7l9A4y614O3bFEw8hLq+OlCOc4N4eZE3 8RoI1r1EI0skw== To: fiatjaf , Bitcoin Protocol Discussion From: Andrew Chow Reply-To: Andrew Chow Message-ID: In-Reply-To: <1768da5c6f0.f63a34dc413157.2741477427673569054@alhur.es> References: <1dd8c285-e3f4-4f03-d608-103a5026146d@achow101.com> <5a4697cb-b9cb-b925-e78f-d5b53f025704@achow101.com> <1768da5c6f0.f63a34dc413157.2741477427673569054@alhur.es> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [bitcoin-dev] New PSBT version proposal X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Dec 2020 21:30:25 -0000 Hi, On 12/22/20 10:30 PM, fiatjaf wrote: > Hi Andrew. > > I'm just a lurker here and I have not much experience with PSBTs, but sti= ll let me pose this very obvious question and concern: isn't this change go= ing to create a compatibility nightmare, with some software supporting vers= ion 1, others supporting version 2, and the ones that care enough about UX = and are still maintained being forced to support both versions -- and for n= o very important reason except some improvements in the way data is structu= red? No, it is not just "improvements in the way data is structured." The primary reason for these changes is to allow PSBT to properly=20 support adding inputs and outputs. This is a feature that many people=20 have requested, and the ways that people have been doing it are honestly=20 just hacks and not really the right way to be doing that. These changes=20 allow for that feature to be supported well. Furthermore, it is possible to downgrade and upgrade PSBTs between the=20 two versions, once all inputs and outputs have been decided. Since=20 PSBTv2 is essentially just taking all of the normal transaction fields=20 and grouping them all with the rest of the data for those inputs and=20 outputs, it is easy to reconstruct a global unsigned transaction and=20 turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other=20 way and break apart the global unsigned tx to turn a PSBTv0 into a=20 PSBTv2. Originally, I had considered requiring that once a transaction=20 was fully constructed it must be downgraded to a PSBTv0, but the=20 structure changes that were made do make it easier to work with PSBT so=20 I decided not to add this requirement. Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be=20 disallowed in PSBTv2 once the transaction is constructed? It would make=20 things much more confusing though as it would no longer be a clean break. Andrew Chow > Ultimately I don't think it should matter if some data is structured in n= ot-the-best-possible way, as long as it is clear enough for the computer an= d for the libraries already written to deal with it. Backwards-compatibilit= y and general interoperability is worth much more than anything else in the= se cases. > > Also let me leave this article here, which I find very important (even if= for some reason it ends up not being relevant to this specific case): http= ://scripting.com/2017/05/09/rulesForStandardsmakers.html > > ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev wrote ---- > > Hi All, > > > > I have some updates on this after speaking with some people off-list. > > > > Firstly, the version number will be set to 2. In most discussions, th= is > > proposal was being referred to as PSBT version 2, so it'll be easier = and > > clearer to set the version number to 2. > > > > For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, > > there will be 2 of them, one for a time based lock time, and the othe= r > > for height based. These will be: > > * PSBT_IN_REQUIRED_TIME_LOCKTIME =3D 0x10 > > * Key: empty > > * Value: 32 bit unsigned little endian integer greater than or equ= al > > to 500000000 representing the minimum Unix timestamp that this input > > requires to be set as the transaction's lock time. Must be omitted in > > PSBTv0, and may be omitted in PSBTv2 > > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME =3D 0x11 > > * Key: empty > > * Value: 32 bit unsigned little endian integer less than 500000000 > > representing the minimum block height that this input requires to be = set > > as the transaction's lock time. Must be omitted in PSBTv0, and may be > > omitted in PSBTv2. > > > > Having two lock time fields is necessary due to the behavior where al= l > > inputs must use the same type of lock time (height or time). Thus if = an > > input requires a particular type of lock time, it must set the requis= ite > > field. Any new inputs being added must be able to accommodate all > > existing inputs' lock time type. This means they either must not have= a > > lock time specified (i.e. no OP_CLTV involved), or have branches that > > allow the acceptance of either type. If an input has a lock time type > > that is incompatible with the rest of the transaction, it must not be= added. > > > > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback > > option if no input lock time fields are present. If there are input l= ock > > times, all lock time calculations must ignore it. > > > > Any role which does lock time calculation will first check if there a= re > > input lock time fields. If there are not, it must then check for a > > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is th= e > > transaction's lock time. If it does not, the lock time is 0. If there > > are input lock time fields, it must choose the type which does not > > invalidate any inputs. The lock time is then determined to be the > > maximum value of all of the lock time fields for the chosen type. > > > > > > Additionally, I would like to add a new global field: > > * PSBT_GLOBAL_UNDER_CONSTRUCTION =3D 0x05 > > * Key: empty > > * Value: A single byte as a boolean. 0 for False, 1 for True. All > > other values ore prohibited. Must be omitted for PSBTv0, may be omitt= ed > > in PSBTv2. > > > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > > outputs can be added to the PSBT. This flag may be set to True when > > inputs and outputs are being updated, signed, and finalized. However > > care must be taken when there are existing signatures. If this field = is > > omitted or set to False, no further inputs and outputs may be added t= o > > the PSBT. > > > > Several rules must be followed to ensure that adding additional input= s > > and outputs will not invalidate existing signatures. First, an input = or > > output adder must check for any existing signatures in all of the oth= er > > inputs. If there are none, the input or output may be added in any > > position. If there are one or more signatures, each signature's sigha= sh > > type must be examined. Inputs may only be added if all existing > > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all > > existing signatures use SIGHASH_NONE. If an input has a signature usi= ng > > SIGHASH_SINGLE, the same number of inputs and outputs must be added > > before that input and it's corresponding output. For all other sighas= h > > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or > > outputs may be added to the PSBT. Specific exceptions can be made in = the > > future for additional sighash types. > > > > Furthermore, these newly added inputs must follow additional lock tim= e > > rules. Because all signatures, regardless of sighash type, sign the > > transaction lock time, newly added inputs when there are existing > > signatures must have the same type of lock time used in the transacti= on, > > and must be less than or equal to the transaction lock time. It must = not > > cause the transaction lock time to change, otherwise the signatures w= ill > > be invalidated. > > > > > > Lastly, to uniquely identify transactions for combiners, a txid can b= e > > computed from the information present in the PSBT. Internally, combin= ers > > can create an unsigned transaction given the transaction version, the > > input prevouts, the outputs, and the computed locktime. This can then= be > > used to calculate a txid and thus used as a way to identify PSBTs. > > Combiners will need to do this for all version 2 PSBTs in order to av= oid > > combining distinct transactions. > > > > > > Andrew Chow > > > > On 12/9/20 5:25 PM, Andrew Chow wrote: > > > Hi All, > > > > > > I would like to propose a new PSBT version that addresses a few > > > deficiencies in the current PSBT v0. As this will be backwards > > > incompatible, a new PSBT version will be used, v1. > > > > > > The primary change is to truly have all input and output data for e= ach > > > in their respective maps. Instead of having to parse an unsigned > > > transaction and lookup some data from there, and other data from th= e > > > correct map, all of the data for an input will be contained in its = map. > > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version= . > > > Thus I propose that the following fields be added: > > > > > > Global: > > > * PSBT_GLOBAL_TX_VERSION =3D 0x02 > > > * Key: empty > > > * Value: 32-bit little endian unsigned integer for the transact= ion > > > version number. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_GLOBAL_PREFERRED_LOCKTIME =3D 0x03 > > > * Key: empty > > > * Value: 32 bit little endian unsigned integer for the preferre= d > > > transaction lock time. Must be omitted in PSBT v0. May be provided = in > > > PSBT v1, assumed to be 0 if not provided. > > > * PSBT_GLOBAL_INPUT_COUNT =3D 0x04 > > > * Key: empty > > > * Value: Compact size unsigned integer. Number of inputs in thi= s > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_GLOBAL_OUTPUT_COUNT =3D 0x05 > > > * Key: empty > > > * Value: Compact size unsigned integer. Number of outputs in th= is > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > > > > Input: > > > * PSBT_IN_PREVIOUS_TXID =3D 0x0e > > > * Key: empty > > > * Value: 32 byte txid of the previous transaction whose output = at > > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 an= d > > > omitted in v0. > > > * PSBT_IN_OUTPUT_INDEX =3D 0x0f > > > * Key: empty > > > * Value: 32 bit little endian integer for the index of the outp= ut > > > being spent. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_IN_SEQUENCE =3D 0x0f > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer for the sequence > > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assu= med > > > to be max sequence (0xffffffff) if not provided. > > > * PSBT_IN_REQUIRED_LOCKTIME =3D 0x10 > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer for the lock tim= e that > > > this input requires. Must be omitted in PSBT v0. May be provided in= PSBT > > > v1, assumed to be 0 if not provided. > > > > > > Output: > > > * PSBT_OUT_VALUE =3D 0x03 > > > * Key: empty > > > * Value: 64-bit unsigned little endian integer for the output's > > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_OUT_OUTPUT_SCRIPT =3D 0x04 > > > * Key: empty > > > * Value: The script for this output. Otherwise known as the > > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > > > > > This change allows for PSBT to be used in the construction of > > > transactions. With these new fields, inputs and outputs can be adde= d as > > > needed. One caveat is that there is no longer a unique transaction > > > identifier so more care must be taken when combining PSBTs. > > > Additionally, adding new inputs and outputs must be done such that > > > signatures are not invalidated. This may be harder to specify. > > > > > > An important thing to note in this proposal are the fields > > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bit= coin > > > transaction only has a single locktime yet a PSBT may have multiple > > > locktimes. To choose the locktime for the transaction, finalizers m= ust > > > choose the maximum of all of the *_LOCKTIME fields. > > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as tho= se > > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum lockti= me to > > > be set. This field allows finalizers to choose a locktime that is h= igh > > > enough for all inputs without needing to understand the scripts > > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use= if > > > no inputs require a particular locktime. > > > > > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v= 1 > > > needs the version number bump to enforce backwards incompatibility. > > > However once the inputs and outputs of a PSBT are decided, a PSBT c= ould > > > be "downgraded" back to v0 by creating the unsigned transaction fro= m the > > > above fields, and then dropping these new fields. > > > > > > If the list finds that these changes are reasonable, I will write a= PR > > > to modify BIP 174 to incorporate them. > > > > > > Thanks, > > > Andrew Chow > > > > > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists.linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >