* [bitcoin-dev] New PSBT version proposal
@ 2020-12-09 22:25 Andrew Chow
2020-12-10 11:28 ` Sanket Kanjalkar
` (3 more replies)
0 siblings, 4 replies; 17+ messages in thread
From: Andrew Chow @ 2020-12-09 22:25 UTC (permalink / raw)
To: Bitcoin Protocol Discussion
Hi All,
I would like to propose a new PSBT version that addresses a few
deficiencies in the current PSBT v0. As this will be backwards
incompatible, a new PSBT version will be used, v1.
The primary change is to truly have all input and output data for each
in their respective maps. Instead of having to parse an unsigned
transaction and lookup some data from there, and other data from the
correct map, all of the data for an input will be contained in its map.
Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version.
Thus I propose that the following fields be added:
Global:
* PSBT_GLOBAL_TX_VERSION = 0x02
* Key: empty
* Value: 32-bit little endian unsigned integer for the transaction
version number. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
* Key: empty
* Value: 32 bit little endian unsigned integer for the preferred
transaction lock time. Must be omitted in PSBT v0. May be provided in
PSBT v1, assumed to be 0 if not provided.
* PSBT_GLOBAL_INPUT_COUNT = 0x04
* Key: empty
* Value: Compact size unsigned integer. Number of inputs in this
PSBT. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_OUTPUT_COUNT = 0x05
* Key: empty
* Value: Compact size unsigned integer. Number of outputs in this
PSBT. Must be provided in PSBT v1 and omitted in v0.
Input:
* PSBT_IN_PREVIOUS_TXID = 0x0e
* Key: empty
* Value: 32 byte txid of the previous transaction whose output at
PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and
omitted in v0.
* PSBT_IN_OUTPUT_INDEX = 0x0f
* Key: empty
* Value: 32 bit little endian integer for the index of the output
being spent. Must be provided in PSBT v1 and omitted in v0.
* PSBT_IN_SEQUENCE = 0x0f
* Key: empty
* Value: 32 bit unsigned little endian integer for the sequence
number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed
to be max sequence (0xffffffff) if not provided.
* PSBT_IN_REQUIRED_LOCKTIME = 0x10
* Key: empty
* Value: 32 bit unsigned little endian integer for the lock time that
this input requires. Must be omitted in PSBT v0. May be provided in PSBT
v1, assumed to be 0 if not provided.
Output:
* PSBT_OUT_VALUE = 0x03
* Key: empty
* Value: 64-bit unsigned little endian integer for the output's
amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
* PSBT_OUT_OUTPUT_SCRIPT = 0x04
* Key: empty
* Value: The script for this output. Otherwise known as the
scriptPubKey. Must be provided in PSBT v1 and omitted in v0.
This change allows for PSBT to be used in the construction of
transactions. With these new fields, inputs and outputs can be added as
needed. One caveat is that there is no longer a unique transaction
identifier so more care must be taken when combining PSBTs.
Additionally, adding new inputs and outputs must be done such that
signatures are not invalidated. This may be harder to specify.
An important thing to note in this proposal are the fields
PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin
transaction only has a single locktime yet a PSBT may have multiple
locktimes. To choose the locktime for the transaction, finalizers must
choose the maximum of all of the *_LOCKTIME fields.
PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those
involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to
be set. This field allows finalizers to choose a locktime that is high
enough for all inputs without needing to understand the scripts
involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if
no inputs require a particular locktime.
As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1
needs the version number bump to enforce backwards incompatibility.
However once the inputs and outputs of a PSBT are decided, a PSBT could
be "downgraded" back to v0 by creating the unsigned transaction from the
above fields, and then dropping these new fields.
If the list finds that these changes are reasonable, I will write a PR
to modify BIP 174 to incorporate them.
Thanks,
Andrew Chow
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [bitcoin-dev] New PSBT version proposal 2020-12-09 22:25 [bitcoin-dev] New PSBT version proposal Andrew Chow @ 2020-12-10 11:28 ` Sanket Kanjalkar 2020-12-16 17:44 ` Andrew Poelstra ` (2 subsequent siblings) 3 siblings, 0 replies; 17+ messages in thread From: Sanket Kanjalkar @ 2020-12-10 11:28 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 6089 bytes --] > > The primary change is to truly have all input and output data for each in > their respective maps 1) +1. It would be really great to have a complete map per input/output that does not require an annoying lookup to a global field. A Bitcoin transaction only has a single locktime yet a PSBT may have > multiple lock times. 2) One other thing, the per input timelock also helps in detecting whether the transaction contains a mix of block-based timelocks and height based timelocks. Recall that such inputs can't be spent together under the same nLocktime. 3) Finally, one last thing which I noted while implementing a generic finalizer for Miniscript is the restriction on sighashType. From the BIP > Signatures for this input must use the sighash type, finalizers must fail > to finalize inputs which have signatures that do not match the specified > sighash type. Signers who cannot produce signatures with the sighash type > must not provide a signature. Is such a restriction necessary? If the purpose is to only suggest signer which sighashType to use, then I think the finalizer should not reject those. Along those lines, we can also mark with suggestions for the type of nlockTime(block vs height) that should be used. With such suggestions, input parties can decide which branches in the satisfaction they should prefer and sign with the corresponding signatures. Note that this purpose is different from the stated purpose of PSBT_GLOBAL_PREFERRED_LOCKTIME. Cheers, Sanket On Wed, Dec 9, 2020 at 4:33 PM Andrew Chow via bitcoin-dev < bitcoin-dev@lists•linuxfoundation.org> wrote: > Hi All, > > I would like to propose a new PSBT version that addresses a few > deficiencies in the current PSBT v0. As this will be backwards > incompatible, a new PSBT version will be used, v1. > > The primary change is to truly have all input and output data for each > in their respective maps. Instead of having to parse an unsigned > transaction and lookup some data from there, and other data from the > correct map, all of the data for an input will be contained in its map. > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > Thus I propose that the following fields be added: > > Global: > * PSBT_GLOBAL_TX_VERSION = 0x02 > * Key: empty > * Value: 32-bit little endian unsigned integer for the transaction > version number. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > * Key: empty > * Value: 32 bit little endian unsigned integer for the preferred > transaction lock time. Must be omitted in PSBT v0. May be provided in > PSBT v1, assumed to be 0 if not provided. > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > * Key: empty > * Value: Compact size unsigned integer. Number of inputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > * Key: empty > * Value: Compact size unsigned integer. Number of outputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > > Input: > * PSBT_IN_PREVIOUS_TXID = 0x0e > * Key: empty > * Value: 32 byte txid of the previous transaction whose output at > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and > omitted in v0. > * PSBT_IN_OUTPUT_INDEX = 0x0f > * Key: empty > * Value: 32 bit little endian integer for the index of the output > being spent. Must be provided in PSBT v1 and omitted in v0. > * PSBT_IN_SEQUENCE = 0x0f > * Key: empty > * Value: 32 bit unsigned little endian integer for the sequence > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed > to be max sequence (0xffffffff) if not provided. > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > * Key: empty > * Value: 32 bit unsigned little endian integer for the lock time that > this input requires. Must be omitted in PSBT v0. May be provided in PSBT > v1, assumed to be 0 if not provided. > > Output: > * PSBT_OUT_VALUE = 0x03 > * Key: empty > * Value: 64-bit unsigned little endian integer for the output's > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > * Key: empty > * Value: The script for this output. Otherwise known as the > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > This change allows for PSBT to be used in the construction of > transactions. With these new fields, inputs and outputs can be added as > needed. One caveat is that there is no longer a unique transaction > identifier so more care must be taken when combining PSBTs. > Additionally, adding new inputs and outputs must be done such that > signatures are not invalidated. This may be harder to specify. > > An important thing to note in this proposal are the fields > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin > transaction only has a single locktime yet a PSBT may have multiple > locktimes. To choose the locktime for the transaction, finalizers must > choose the maximum of all of the *_LOCKTIME fields. > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to > be set. This field allows finalizers to choose a locktime that is high > enough for all inputs without needing to understand the scripts > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if > no inputs require a particular locktime. > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 > needs the version number bump to enforce backwards incompatibility. > However once the inputs and outputs of a PSBT are decided, a PSBT could > be "downgraded" back to v0 by creating the unsigned transaction from the > above fields, and then dropping these new fields. > > If the list finds that these changes are reasonable, I will write a PR > to modify BIP 174 to incorporate them. > > Thanks, > Andrew Chow > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 7570 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-09 22:25 [bitcoin-dev] New PSBT version proposal Andrew Chow 2020-12-10 11:28 ` Sanket Kanjalkar @ 2020-12-16 17:44 ` Andrew Poelstra 2020-12-22 20:12 ` Andrew Chow 2021-03-10 0:20 ` Lloyd Fournier 3 siblings, 0 replies; 17+ messages in thread From: Andrew Poelstra @ 2020-12-16 17:44 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 1554 bytes --] On Wed, Dec 09, 2020 at 10:25:37PM +0000, Andrew Chow via bitcoin-dev wrote: > Hi All, > > I would like to propose a new PSBT version that addresses a few > deficiencies in the current PSBT v0. As this will be backwards > incompatible, a new PSBT version will be used, v1. > > The primary change is to truly have all input and output data for each > in their respective maps. Instead of having to parse an unsigned > transaction and lookup some data from there, and other data from the > correct map, all of the data for an input will be contained in its map. > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > Thus I propose that the following fields be added: > > Global: > * PSBT_GLOBAL_TX_VERSION = 0x02 > ?? * Key: empty > ?? * Value: 32-bit little endian unsigned integer for the transaction > version number. Must be provided in PSBT v1 and omitted in v0. All of these changes sound great. It would definitely make working with PSBTs easier if all data was accessible in the same format, rather than being split between the global unsigned tx and the main body. One minor quibble is the version numbering -- you mention "v1" in this post but set GLOBAL_TX_VERSION to 2. I think we should consistently use 2 everywhere; probably nobody thinks of the existing PSBT as "version 0". -- Andrew Poelstra Director of Research, Blockstream Email: apoelstra at wpsoftware.net Web: https://www.wpsoftware.net/andrew The sun is always shining in space -Justin Lewis-Webster [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-09 22:25 [bitcoin-dev] New PSBT version proposal Andrew Chow 2020-12-10 11:28 ` Sanket Kanjalkar 2020-12-16 17:44 ` Andrew Poelstra @ 2020-12-22 20:12 ` Andrew Chow 2020-12-23 3:30 ` fiatjaf ` (2 more replies) 2021-03-10 0:20 ` Lloyd Fournier 3 siblings, 3 replies; 17+ messages in thread From: Andrew Chow @ 2020-12-22 20:12 UTC (permalink / raw) To: Bitcoin Protocol Discussion Hi All, I have some updates on this after speaking with some people off-list. Firstly, the version number will be set to 2. In most discussions, this proposal was being referred to as PSBT version 2, so it'll be easier and clearer to set the version number to 2. For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, there will be 2 of them, one for a time based lock time, and the other for height based. These will be: * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 * Key: empty * Value: 32 bit unsigned little endian integer greater than or equal to 500000000 representing the minimum Unix timestamp that this input requires to be set as the transaction's lock time. Must be omitted in PSBTv0, and may be omitted in PSBTv2 * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 * Key: empty * Value: 32 bit unsigned little endian integer less than 500000000 representing the minimum block height that this input requires to be set as the transaction's lock time. Must be omitted in PSBTv0, and may be omitted in PSBTv2. Having two lock time fields is necessary due to the behavior where all inputs must use the same type of lock time (height or time). Thus if an input requires a particular type of lock time, it must set the requisite field. Any new inputs being added must be able to accommodate all existing inputs' lock time type. This means they either must not have a lock time specified (i.e. no OP_CLTV involved), or have branches that allow the acceptance of either type. If an input has a lock time type that is incompatible with the rest of the transaction, it must not be added. PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback option if no input lock time fields are present. If there are input lock times, all lock time calculations must ignore it. Any role which does lock time calculation will first check if there are input lock time fields. If there are not, it must then check for a PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the transaction's lock time. If it does not, the lock time is 0. If there are input lock time fields, it must choose the type which does not invalidate any inputs. The lock time is then determined to be the maximum value of all of the lock time fields for the chosen type. Additionally, I would like to add a new global field: * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 * Key: empty * Value: A single byte as a boolean. 0 for False, 1 for True. All other values ore prohibited. Must be omitted for PSBTv0, may be omitted in PSBTv2. PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and outputs can be added to the PSBT. This flag may be set to True when inputs and outputs are being updated, signed, and finalized. However care must be taken when there are existing signatures. If this field is omitted or set to False, no further inputs and outputs may be added to the PSBT. Several rules must be followed to ensure that adding additional inputs and outputs will not invalidate existing signatures. First, an input or output adder must check for any existing signatures in all of the other inputs. If there are none, the input or output may be added in any position. If there are one or more signatures, each signature's sighash type must be examined. Inputs may only be added if all existing signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all existing signatures use SIGHASH_NONE. If an input has a signature using SIGHASH_SINGLE, the same number of inputs and outputs must be added before that input and it's corresponding output. For all other sighash types (i.e. SIGHASH_ALL and any future sighash types), no inputs or outputs may be added to the PSBT. Specific exceptions can be made in the future for additional sighash types. Furthermore, these newly added inputs must follow additional lock time rules. Because all signatures, regardless of sighash type, sign the transaction lock time, newly added inputs when there are existing signatures must have the same type of lock time used in the transaction, and must be less than or equal to the transaction lock time. It must not cause the transaction lock time to change, otherwise the signatures will be invalidated. Lastly, to uniquely identify transactions for combiners, a txid can be computed from the information present in the PSBT. Internally, combiners can create an unsigned transaction given the transaction version, the input prevouts, the outputs, and the computed locktime. This can then be used to calculate a txid and thus used as a way to identify PSBTs. Combiners will need to do this for all version 2 PSBTs in order to avoid combining distinct transactions. Andrew Chow On 12/9/20 5:25 PM, Andrew Chow wrote: > Hi All, > > I would like to propose a new PSBT version that addresses a few > deficiencies in the current PSBT v0. As this will be backwards > incompatible, a new PSBT version will be used, v1. > > The primary change is to truly have all input and output data for each > in their respective maps. Instead of having to parse an unsigned > transaction and lookup some data from there, and other data from the > correct map, all of the data for an input will be contained in its map. > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > Thus I propose that the following fields be added: > > Global: > * PSBT_GLOBAL_TX_VERSION = 0x02 > * Key: empty > * Value: 32-bit little endian unsigned integer for the transaction > version number. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > * Key: empty > * Value: 32 bit little endian unsigned integer for the preferred > transaction lock time. Must be omitted in PSBT v0. May be provided in > PSBT v1, assumed to be 0 if not provided. > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > * Key: empty > * Value: Compact size unsigned integer. Number of inputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > * Key: empty > * Value: Compact size unsigned integer. Number of outputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > > Input: > * PSBT_IN_PREVIOUS_TXID = 0x0e > * Key: empty > * Value: 32 byte txid of the previous transaction whose output at > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and > omitted in v0. > * PSBT_IN_OUTPUT_INDEX = 0x0f > * Key: empty > * Value: 32 bit little endian integer for the index of the output > being spent. Must be provided in PSBT v1 and omitted in v0. > * PSBT_IN_SEQUENCE = 0x0f > * Key: empty > * Value: 32 bit unsigned little endian integer for the sequence > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed > to be max sequence (0xffffffff) if not provided. > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > * Key: empty > * Value: 32 bit unsigned little endian integer for the lock time that > this input requires. Must be omitted in PSBT v0. May be provided in PSBT > v1, assumed to be 0 if not provided. > > Output: > * PSBT_OUT_VALUE = 0x03 > * Key: empty > * Value: 64-bit unsigned little endian integer for the output's > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > * Key: empty > * Value: The script for this output. Otherwise known as the > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > This change allows for PSBT to be used in the construction of > transactions. With these new fields, inputs and outputs can be added as > needed. One caveat is that there is no longer a unique transaction > identifier so more care must be taken when combining PSBTs. > Additionally, adding new inputs and outputs must be done such that > signatures are not invalidated. This may be harder to specify. > > An important thing to note in this proposal are the fields > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin > transaction only has a single locktime yet a PSBT may have multiple > locktimes. To choose the locktime for the transaction, finalizers must > choose the maximum of all of the *_LOCKTIME fields. > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to > be set. This field allows finalizers to choose a locktime that is high > enough for all inputs without needing to understand the scripts > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if > no inputs require a particular locktime. > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 > needs the version number bump to enforce backwards incompatibility. > However once the inputs and outputs of a PSBT are decided, a PSBT could > be "downgraded" back to v0 by creating the unsigned transaction from the > above fields, and then dropping these new fields. > > If the list finds that these changes are reasonable, I will write a PR > to modify BIP 174 to incorporate them. > > Thanks, > Andrew Chow ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-22 20:12 ` Andrew Chow @ 2020-12-23 3:30 ` fiatjaf 2020-12-23 15:22 ` Andrew Poelstra 2020-12-23 21:30 ` Andrew Chow 2020-12-23 21:32 ` Andrew Chow 2021-01-06 23:26 ` Rusty Russell 2 siblings, 2 replies; 17+ messages in thread From: fiatjaf @ 2020-12-23 3:30 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion Hi Andrew. I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured? Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases. Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote ---- > Hi All, > > I have some updates on this after speaking with some people off-list. > > Firstly, the version number will be set to 2. In most discussions, this > proposal was being referred to as PSBT version 2, so it'll be easier and > clearer to set the version number to 2. > > For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, > there will be 2 of them, one for a time based lock time, and the other > for height based. These will be: > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 > * Key: empty > * Value: 32 bit unsigned little endian integer greater than or equal > to 500000000 representing the minimum Unix timestamp that this input > requires to be set as the transaction's lock time. Must be omitted in > PSBTv0, and may be omitted in PSBTv2 > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 > * Key: empty > * Value: 32 bit unsigned little endian integer less than 500000000 > representing the minimum block height that this input requires to be set > as the transaction's lock time. Must be omitted in PSBTv0, and may be > omitted in PSBTv2. > > Having two lock time fields is necessary due to the behavior where all > inputs must use the same type of lock time (height or time). Thus if an > input requires a particular type of lock time, it must set the requisite > field. Any new inputs being added must be able to accommodate all > existing inputs' lock time type. This means they either must not have a > lock time specified (i.e. no OP_CLTV involved), or have branches that > allow the acceptance of either type. If an input has a lock time type > that is incompatible with the rest of the transaction, it must not be added. > > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback > option if no input lock time fields are present. If there are input lock > times, all lock time calculations must ignore it. > > Any role which does lock time calculation will first check if there are > input lock time fields. If there are not, it must then check for a > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the > transaction's lock time. If it does not, the lock time is 0. If there > are input lock time fields, it must choose the type which does not > invalidate any inputs. The lock time is then determined to be the > maximum value of all of the lock time fields for the chosen type. > > > Additionally, I would like to add a new global field: > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 > * Key: empty > * Value: A single byte as a boolean. 0 for False, 1 for True. All > other values ore prohibited. Must be omitted for PSBTv0, may be omitted > in PSBTv2. > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > outputs can be added to the PSBT. This flag may be set to True when > inputs and outputs are being updated, signed, and finalized. However > care must be taken when there are existing signatures. If this field is > omitted or set to False, no further inputs and outputs may be added to > the PSBT. > > Several rules must be followed to ensure that adding additional inputs > and outputs will not invalidate existing signatures. First, an input or > output adder must check for any existing signatures in all of the other > inputs. If there are none, the input or output may be added in any > position. If there are one or more signatures, each signature's sighash > type must be examined. Inputs may only be added if all existing > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all > existing signatures use SIGHASH_NONE. If an input has a signature using > SIGHASH_SINGLE, the same number of inputs and outputs must be added > before that input and it's corresponding output. For all other sighash > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or > outputs may be added to the PSBT. Specific exceptions can be made in the > future for additional sighash types. > > Furthermore, these newly added inputs must follow additional lock time > rules. Because all signatures, regardless of sighash type, sign the > transaction lock time, newly added inputs when there are existing > signatures must have the same type of lock time used in the transaction, > and must be less than or equal to the transaction lock time. It must not > cause the transaction lock time to change, otherwise the signatures will > be invalidated. > > > Lastly, to uniquely identify transactions for combiners, a txid can be > computed from the information present in the PSBT. Internally, combiners > can create an unsigned transaction given the transaction version, the > input prevouts, the outputs, and the computed locktime. This can then be > used to calculate a txid and thus used as a way to identify PSBTs. > Combiners will need to do this for all version 2 PSBTs in order to avoid > combining distinct transactions. > > > Andrew Chow > > On 12/9/20 5:25 PM, Andrew Chow wrote: > > Hi All, > > > > I would like to propose a new PSBT version that addresses a few > > deficiencies in the current PSBT v0. As this will be backwards > > incompatible, a new PSBT version will be used, v1. > > > > The primary change is to truly have all input and output data for each > > in their respective maps. Instead of having to parse an unsigned > > transaction and lookup some data from there, and other data from the > > correct map, all of the data for an input will be contained in its map. > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > > Thus I propose that the following fields be added: > > > > Global: > > * PSBT_GLOBAL_TX_VERSION = 0x02 > > * Key: empty > > * Value: 32-bit little endian unsigned integer for the transaction > > version number. Must be provided in PSBT v1 and omitted in v0. > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > > * Key: empty > > * Value: 32 bit little endian unsigned integer for the preferred > > transaction lock time. Must be omitted in PSBT v0. May be provided in > > PSBT v1, assumed to be 0 if not provided. > > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > > * Key: empty > > * Value: Compact size unsigned integer. Number of inputs in this > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > > * Key: empty > > * Value: Compact size unsigned integer. Number of outputs in this > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > > Input: > > * PSBT_IN_PREVIOUS_TXID = 0x0e > > * Key: empty > > * Value: 32 byte txid of the previous transaction whose output at > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and > > omitted in v0. > > * PSBT_IN_OUTPUT_INDEX = 0x0f > > * Key: empty > > * Value: 32 bit little endian integer for the index of the output > > being spent. Must be provided in PSBT v1 and omitted in v0. > > * PSBT_IN_SEQUENCE = 0x0f > > * Key: empty > > * Value: 32 bit unsigned little endian integer for the sequence > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed > > to be max sequence (0xffffffff) if not provided. > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > > * Key: empty > > * Value: 32 bit unsigned little endian integer for the lock time that > > this input requires. Must be omitted in PSBT v0. May be provided in PSBT > > v1, assumed to be 0 if not provided. > > > > Output: > > * PSBT_OUT_VALUE = 0x03 > > * Key: empty > > * Value: 64-bit unsigned little endian integer for the output's > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > > * Key: empty > > * Value: The script for this output. Otherwise known as the > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > > > This change allows for PSBT to be used in the construction of > > transactions. With these new fields, inputs and outputs can be added as > > needed. One caveat is that there is no longer a unique transaction > > identifier so more care must be taken when combining PSBTs. > > Additionally, adding new inputs and outputs must be done such that > > signatures are not invalidated. This may be harder to specify. > > > > An important thing to note in this proposal are the fields > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin > > transaction only has a single locktime yet a PSBT may have multiple > > locktimes. To choose the locktime for the transaction, finalizers must > > choose the maximum of all of the *_LOCKTIME fields. > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to > > be set. This field allows finalizers to choose a locktime that is high > > enough for all inputs without needing to understand the scripts > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if > > no inputs require a particular locktime. > > > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 > > needs the version number bump to enforce backwards incompatibility. > > However once the inputs and outputs of a PSBT are decided, a PSBT could > > be "downgraded" back to v0 by creating the unsigned transaction from the > > above fields, and then dropping these new fields. > > > > If the list finds that these changes are reasonable, I will write a PR > > to modify BIP 174 to incorporate them. > > > > Thanks, > > Andrew Chow > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-23 3:30 ` fiatjaf @ 2020-12-23 15:22 ` Andrew Poelstra 2020-12-23 21:30 ` Andrew Chow 1 sibling, 0 replies; 17+ messages in thread From: Andrew Poelstra @ 2020-12-23 15:22 UTC (permalink / raw) To: fiatjaf, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 4068 bytes --] On Wed, Dec 23, 2020 at 12:30:20AM -0300, fiatjaf via bitcoin-dev wrote: > Hi Andrew. > > I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured? > Yes, software will have to support both versions for a long time (likely forever, at least in the case of Core). But I think this is okay, for a couple of reasons: 1. it is very easy to convert from the old to new format, and from new to old (unless the new one uses features unsupported by the old). Indeed, the conversion logic is essentially the same as the logic that the Extractor role uses, so there isn't even that much redundant code. 2. There actually isn't a lot of software using PSBT out there, and most of that that does use PSBT is under rapid development. The obvious exception to this deployed hardware wallets, but as far as "software developers supporting old things for the sake of old hardware wallets" I think this transition is an order of magnitude simpler to handle than many of the ad-hoc protocol changes that individual vendors have done. In other words this is a "fact of life", and not even one of the grosser ones. 3. PSBT is pretty-much a dumb pure data format, and the diff between the new format and the old is pretty small. > Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases. > The reasons for switching to PSBT 2 are actually more than just structuring the data in a cleaner way. I agree that if the point of this upgrade were just elegance, it would not be worth the compatibility loss. But there are practical limitations that this proposal eliminates: 1. PSBT provides no way to modify the set of inputs or outputs after the Creator role is done. 2. Because of this, it forces certain things (e.g. locktimes and sequence numbers) to be chosen by the Creator, who may not have all the relevant information, and who certainly might not have it before any Updaters have done their part. as well, of course, as elegance reasons: 3. Parsers of the existing PSBT need to understand the Bitcoin transaction format just to learn e.g. how many inputs and outputs there are. It is impossible to parse a PSBT without also parsing (almost) the whole transaction. 4. Similarly to cross-check fields like 'non_witness_utxo' which are committed to in the transaction, you have to parse the whole transaction just to make sure that the purely-redundant data is correctly redundant. 5. If you put a 0-input transaction into a PSBT (which would be pointless because there's no way to add inputs, but it's not forbidden so your software still has to deal with this somehow..), you need a different transaction parser than the normal one, because there is an ambiguity related to segwit that PSBT resolves differently. It's also worth considering that PSBT is a young protocol, and future extensions will be easier starting from PSBT 2 than starting from the original version. > Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html > -- Andrew Poelstra Director of Research, Blockstream Email: apoelstra at wpsoftware.net Web: https://www.wpsoftware.net/andrew The sun is always shining in space -Justin Lewis-Webster [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-23 3:30 ` fiatjaf 2020-12-23 15:22 ` Andrew Poelstra @ 2020-12-23 21:30 ` Andrew Chow 2021-01-02 6:34 ` Jeremy 1 sibling, 1 reply; 17+ messages in thread From: Andrew Chow @ 2020-12-23 21:30 UTC (permalink / raw) To: fiatjaf, Bitcoin Protocol Discussion Hi, On 12/22/20 10:30 PM, fiatjaf wrote: > Hi Andrew. > > I'm just a lurker here and I have not much experience with PSBTs, but still let me pose this very obvious question and concern: isn't this change going to create a compatibility nightmare, with some software supporting version 1, others supporting version 2, and the ones that care enough about UX and are still maintained being forced to support both versions -- and for no very important reason except some improvements in the way data is structured? No, it is not just "improvements in the way data is structured." The primary reason for these changes is to allow PSBT to properly support adding inputs and outputs. This is a feature that many people have requested, and the ways that people have been doing it are honestly just hacks and not really the right way to be doing that. These changes allow for that feature to be supported well. Furthermore, it is possible to downgrade and upgrade PSBTs between the two versions, once all inputs and outputs have been decided. Since PSBTv2 is essentially just taking all of the normal transaction fields and grouping them all with the rest of the data for those inputs and outputs, it is easy to reconstruct a global unsigned transaction and turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other way and break apart the global unsigned tx to turn a PSBTv0 into a PSBTv2. Originally, I had considered requiring that once a transaction was fully constructed it must be downgraded to a PSBTv0, but the structure changes that were made do make it easier to work with PSBT so I decided not to add this requirement. Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be disallowed in PSBTv2 once the transaction is constructed? It would make things much more confusing though as it would no longer be a clean break. Andrew Chow > Ultimately I don't think it should matter if some data is structured in not-the-best-possible way, as long as it is clear enough for the computer and for the libraries already written to deal with it. Backwards-compatibility and general interoperability is worth much more than anything else in these cases. > > Also let me leave this article here, which I find very important (even if for some reason it ends up not being relevant to this specific case): http://scripting.com/2017/05/09/rulesForStandardsmakers.html > > ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote ---- > > Hi All, > > > > I have some updates on this after speaking with some people off-list. > > > > Firstly, the version number will be set to 2. In most discussions, this > > proposal was being referred to as PSBT version 2, so it'll be easier and > > clearer to set the version number to 2. > > > > For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, > > there will be 2 of them, one for a time based lock time, and the other > > for height based. These will be: > > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 > > * Key: empty > > * Value: 32 bit unsigned little endian integer greater than or equal > > to 500000000 representing the minimum Unix timestamp that this input > > requires to be set as the transaction's lock time. Must be omitted in > > PSBTv0, and may be omitted in PSBTv2 > > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 > > * Key: empty > > * Value: 32 bit unsigned little endian integer less than 500000000 > > representing the minimum block height that this input requires to be set > > as the transaction's lock time. Must be omitted in PSBTv0, and may be > > omitted in PSBTv2. > > > > Having two lock time fields is necessary due to the behavior where all > > inputs must use the same type of lock time (height or time). Thus if an > > input requires a particular type of lock time, it must set the requisite > > field. Any new inputs being added must be able to accommodate all > > existing inputs' lock time type. This means they either must not have a > > lock time specified (i.e. no OP_CLTV involved), or have branches that > > allow the acceptance of either type. If an input has a lock time type > > that is incompatible with the rest of the transaction, it must not be added. > > > > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback > > option if no input lock time fields are present. If there are input lock > > times, all lock time calculations must ignore it. > > > > Any role which does lock time calculation will first check if there are > > input lock time fields. If there are not, it must then check for a > > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the > > transaction's lock time. If it does not, the lock time is 0. If there > > are input lock time fields, it must choose the type which does not > > invalidate any inputs. The lock time is then determined to be the > > maximum value of all of the lock time fields for the chosen type. > > > > > > Additionally, I would like to add a new global field: > > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 > > * Key: empty > > * Value: A single byte as a boolean. 0 for False, 1 for True. All > > other values ore prohibited. Must be omitted for PSBTv0, may be omitted > > in PSBTv2. > > > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > > outputs can be added to the PSBT. This flag may be set to True when > > inputs and outputs are being updated, signed, and finalized. However > > care must be taken when there are existing signatures. If this field is > > omitted or set to False, no further inputs and outputs may be added to > > the PSBT. > > > > Several rules must be followed to ensure that adding additional inputs > > and outputs will not invalidate existing signatures. First, an input or > > output adder must check for any existing signatures in all of the other > > inputs. If there are none, the input or output may be added in any > > position. If there are one or more signatures, each signature's sighash > > type must be examined. Inputs may only be added if all existing > > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all > > existing signatures use SIGHASH_NONE. If an input has a signature using > > SIGHASH_SINGLE, the same number of inputs and outputs must be added > > before that input and it's corresponding output. For all other sighash > > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or > > outputs may be added to the PSBT. Specific exceptions can be made in the > > future for additional sighash types. > > > > Furthermore, these newly added inputs must follow additional lock time > > rules. Because all signatures, regardless of sighash type, sign the > > transaction lock time, newly added inputs when there are existing > > signatures must have the same type of lock time used in the transaction, > > and must be less than or equal to the transaction lock time. It must not > > cause the transaction lock time to change, otherwise the signatures will > > be invalidated. > > > > > > Lastly, to uniquely identify transactions for combiners, a txid can be > > computed from the information present in the PSBT. Internally, combiners > > can create an unsigned transaction given the transaction version, the > > input prevouts, the outputs, and the computed locktime. This can then be > > used to calculate a txid and thus used as a way to identify PSBTs. > > Combiners will need to do this for all version 2 PSBTs in order to avoid > > combining distinct transactions. > > > > > > Andrew Chow > > > > On 12/9/20 5:25 PM, Andrew Chow wrote: > > > Hi All, > > > > > > I would like to propose a new PSBT version that addresses a few > > > deficiencies in the current PSBT v0. As this will be backwards > > > incompatible, a new PSBT version will be used, v1. > > > > > > The primary change is to truly have all input and output data for each > > > in their respective maps. Instead of having to parse an unsigned > > > transaction and lookup some data from there, and other data from the > > > correct map, all of the data for an input will be contained in its map. > > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > > > Thus I propose that the following fields be added: > > > > > > Global: > > > * PSBT_GLOBAL_TX_VERSION = 0x02 > > > * Key: empty > > > * Value: 32-bit little endian unsigned integer for the transaction > > > version number. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > > > * Key: empty > > > * Value: 32 bit little endian unsigned integer for the preferred > > > transaction lock time. Must be omitted in PSBT v0. May be provided in > > > PSBT v1, assumed to be 0 if not provided. > > > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > > > * Key: empty > > > * Value: Compact size unsigned integer. Number of inputs in this > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > > > * Key: empty > > > * Value: Compact size unsigned integer. Number of outputs in this > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > > > > Input: > > > * PSBT_IN_PREVIOUS_TXID = 0x0e > > > * Key: empty > > > * Value: 32 byte txid of the previous transaction whose output at > > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and > > > omitted in v0. > > > * PSBT_IN_OUTPUT_INDEX = 0x0f > > > * Key: empty > > > * Value: 32 bit little endian integer for the index of the output > > > being spent. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_IN_SEQUENCE = 0x0f > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer for the sequence > > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed > > > to be max sequence (0xffffffff) if not provided. > > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer for the lock time that > > > this input requires. Must be omitted in PSBT v0. May be provided in PSBT > > > v1, assumed to be 0 if not provided. > > > > > > Output: > > > * PSBT_OUT_VALUE = 0x03 > > > * Key: empty > > > * Value: 64-bit unsigned little endian integer for the output's > > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > > > * Key: empty > > > * Value: The script for this output. Otherwise known as the > > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > > > > > This change allows for PSBT to be used in the construction of > > > transactions. With these new fields, inputs and outputs can be added as > > > needed. One caveat is that there is no longer a unique transaction > > > identifier so more care must be taken when combining PSBTs. > > > Additionally, adding new inputs and outputs must be done such that > > > signatures are not invalidated. This may be harder to specify. > > > > > > An important thing to note in this proposal are the fields > > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin > > > transaction only has a single locktime yet a PSBT may have multiple > > > locktimes. To choose the locktime for the transaction, finalizers must > > > choose the maximum of all of the *_LOCKTIME fields. > > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those > > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to > > > be set. This field allows finalizers to choose a locktime that is high > > > enough for all inputs without needing to understand the scripts > > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if > > > no inputs require a particular locktime. > > > > > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 > > > needs the version number bump to enforce backwards incompatibility. > > > However once the inputs and outputs of a PSBT are decided, a PSBT could > > > be "downgraded" back to v0 by creating the unsigned transaction from the > > > above fields, and then dropping these new fields. > > > > > > If the list finds that these changes are reasonable, I will write a PR > > > to modify BIP 174 to incorporate them. > > > > > > Thanks, > > > Andrew Chow > > > > > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-23 21:30 ` Andrew Chow @ 2021-01-02 6:34 ` Jeremy 0 siblings, 0 replies; 17+ messages in thread From: Jeremy @ 2021-01-02 6:34 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 14125 bytes --] One thing I think should be added in V2 is the ability to specify sighash flags per-key as opposed to per-input. The per-key restriction is unfitting given that there are circumstances where multisig signers may validate heterogenous logic. -- @JeremyRubin <https://twitter.com/JeremyRubin> <https://twitter.com/JeremyRubin> On Wed, Dec 23, 2020 at 1:37 PM Andrew Chow via bitcoin-dev < bitcoin-dev@lists•linuxfoundation.org> wrote: > Hi, > > On 12/22/20 10:30 PM, fiatjaf wrote: > > Hi Andrew. > > > > I'm just a lurker here and I have not much experience with PSBTs, but > still let me pose this very obvious question and concern: isn't this change > going to create a compatibility nightmare, with some software supporting > version 1, others supporting version 2, and the ones that care enough about > UX and are still maintained being forced to support both versions -- and > for no very important reason except some improvements in the way data is > structured? > No, it is not just "improvements in the way data is structured." > > The primary reason for these changes is to allow PSBT to properly > support adding inputs and outputs. This is a feature that many people > have requested, and the ways that people have been doing it are honestly > just hacks and not really the right way to be doing that. These changes > allow for that feature to be supported well. > > Furthermore, it is possible to downgrade and upgrade PSBTs between the > two versions, once all inputs and outputs have been decided. Since > PSBTv2 is essentially just taking all of the normal transaction fields > and grouping them all with the rest of the data for those inputs and > outputs, it is easy to reconstruct a global unsigned transaction and > turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other > way and break apart the global unsigned tx to turn a PSBTv0 into a > PSBTv2. Originally, I had considered requiring that once a transaction > was fully constructed it must be downgraded to a PSBTv0, but the > structure changes that were made do make it easier to work with PSBT so > I decided not to add this requirement. > > Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be > disallowed in PSBTv2 once the transaction is constructed? It would make > things much more confusing though as it would no longer be a clean break. > > > Andrew Chow > > > Ultimately I don't think it should matter if some data is structured in > not-the-best-possible way, as long as it is clear enough for the computer > and for the libraries already written to deal with it. > Backwards-compatibility and general interoperability is worth much more > than anything else in these cases. > > > > Also let me leave this article here, which I find very important (even > if for some reason it ends up not being relevant to this specific case): > http://scripting.com/2017/05/09/rulesForStandardsmakers.html > > > > ---- On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation.org> wrote ---- > > > Hi All, > > > > > > I have some updates on this after speaking with some people off-list. > > > > > > Firstly, the version number will be set to 2. In most discussions, > this > > > proposal was being referred to as PSBT version 2, so it'll be easier > and > > > clearer to set the version number to 2. > > > > > > For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, > > > there will be 2 of them, one for a time based lock time, and the > other > > > for height based. These will be: > > > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer greater than or > equal > > > to 500000000 representing the minimum Unix timestamp that this input > > > requires to be set as the transaction's lock time. Must be omitted in > > > PSBTv0, and may be omitted in PSBTv2 > > > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 > > > * Key: empty > > > * Value: 32 bit unsigned little endian integer less than 500000000 > > > representing the minimum block height that this input requires to be > set > > > as the transaction's lock time. Must be omitted in PSBTv0, and may be > > > omitted in PSBTv2. > > > > > > Having two lock time fields is necessary due to the behavior where > all > > > inputs must use the same type of lock time (height or time). Thus if > an > > > input requires a particular type of lock time, it must set the > requisite > > > field. Any new inputs being added must be able to accommodate all > > > existing inputs' lock time type. This means they either must not > have a > > > lock time specified (i.e. no OP_CLTV involved), or have branches that > > > allow the acceptance of either type. If an input has a lock time type > > > that is incompatible with the rest of the transaction, it must not > be added. > > > > > > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback > > > option if no input lock time fields are present. If there are input > lock > > > times, all lock time calculations must ignore it. > > > > > > Any role which does lock time calculation will first check if there > are > > > input lock time fields. If there are not, it must then check for a > > > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is > the > > > transaction's lock time. If it does not, the lock time is 0. If there > > > are input lock time fields, it must choose the type which does not > > > invalidate any inputs. The lock time is then determined to be the > > > maximum value of all of the lock time fields for the chosen type. > > > > > > > > > Additionally, I would like to add a new global field: > > > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 > > > * Key: empty > > > * Value: A single byte as a boolean. 0 for False, 1 for True. All > > > other values ore prohibited. Must be omitted for PSBTv0, may be > omitted > > > in PSBTv2. > > > > > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > > > outputs can be added to the PSBT. This flag may be set to True when > > > inputs and outputs are being updated, signed, and finalized. However > > > care must be taken when there are existing signatures. If this field > is > > > omitted or set to False, no further inputs and outputs may be added > to > > > the PSBT. > > > > > > Several rules must be followed to ensure that adding additional > inputs > > > and outputs will not invalidate existing signatures. First, an input > or > > > output adder must check for any existing signatures in all of the > other > > > inputs. If there are none, the input or output may be added in any > > > position. If there are one or more signatures, each signature's > sighash > > > type must be examined. Inputs may only be added if all existing > > > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all > > > existing signatures use SIGHASH_NONE. If an input has a signature > using > > > SIGHASH_SINGLE, the same number of inputs and outputs must be added > > > before that input and it's corresponding output. For all other > sighash > > > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or > > > outputs may be added to the PSBT. Specific exceptions can be made in > the > > > future for additional sighash types. > > > > > > Furthermore, these newly added inputs must follow additional lock > time > > > rules. Because all signatures, regardless of sighash type, sign the > > > transaction lock time, newly added inputs when there are existing > > > signatures must have the same type of lock time used in the > transaction, > > > and must be less than or equal to the transaction lock time. It must > not > > > cause the transaction lock time to change, otherwise the signatures > will > > > be invalidated. > > > > > > > > > Lastly, to uniquely identify transactions for combiners, a txid can > be > > > computed from the information present in the PSBT. Internally, > combiners > > > can create an unsigned transaction given the transaction version, the > > > input prevouts, the outputs, and the computed locktime. This can > then be > > > used to calculate a txid and thus used as a way to identify PSBTs. > > > Combiners will need to do this for all version 2 PSBTs in order to > avoid > > > combining distinct transactions. > > > > > > > > > Andrew Chow > > > > > > On 12/9/20 5:25 PM, Andrew Chow wrote: > > > > Hi All, > > > > > > > > I would like to propose a new PSBT version that addresses a few > > > > deficiencies in the current PSBT v0. As this will be backwards > > > > incompatible, a new PSBT version will be used, v1. > > > > > > > > The primary change is to truly have all input and output data for > each > > > > in their respective maps. Instead of having to parse an unsigned > > > > transaction and lookup some data from there, and other data from > the > > > > correct map, all of the data for an input will be contained in its > map. > > > > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new > version. > > > > Thus I propose that the following fields be added: > > > > > > > > Global: > > > > * PSBT_GLOBAL_TX_VERSION = 0x02 > > > > * Key: empty > > > > * Value: 32-bit little endian unsigned integer for the > transaction > > > > version number. Must be provided in PSBT v1 and omitted in v0. > > > > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > > > > * Key: empty > > > > * Value: 32 bit little endian unsigned integer for the > preferred > > > > transaction lock time. Must be omitted in PSBT v0. May be provided > in > > > > PSBT v1, assumed to be 0 if not provided. > > > > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > > > > * Key: empty > > > > * Value: Compact size unsigned integer. Number of inputs in > this > > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > > > > * Key: empty > > > > * Value: Compact size unsigned integer. Number of outputs in > this > > > > PSBT. Must be provided in PSBT v1 and omitted in v0. > > > > > > > > Input: > > > > * PSBT_IN_PREVIOUS_TXID = 0x0e > > > > * Key: empty > > > > * Value: 32 byte txid of the previous transaction whose output > at > > > > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 > and > > > > omitted in v0. > > > > * PSBT_IN_OUTPUT_INDEX = 0x0f > > > > * Key: empty > > > > * Value: 32 bit little endian integer for the index of the > output > > > > being spent. Must be provided in PSBT v1 and omitted in v0. > > > > * PSBT_IN_SEQUENCE = 0x0f > > > > * Key: empty > > > > * Value: 32 bit unsigned little endian integer for the sequence > > > > number. Must be omitted in PSBT v0. May be provided in PSBT v1 > assumed > > > > to be max sequence (0xffffffff) if not provided. > > > > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > > > > * Key: empty > > > > * Value: 32 bit unsigned little endian integer for the lock > time that > > > > this input requires. Must be omitted in PSBT v0. May be provided > in PSBT > > > > v1, assumed to be 0 if not provided. > > > > > > > > Output: > > > > * PSBT_OUT_VALUE = 0x03 > > > > * Key: empty > > > > * Value: 64-bit unsigned little endian integer for the output's > > > > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > > > > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > > > > * Key: empty > > > > * Value: The script for this output. Otherwise known as the > > > > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > > > > > > > This change allows for PSBT to be used in the construction of > > > > transactions. With these new fields, inputs and outputs can be > added as > > > > needed. One caveat is that there is no longer a unique transaction > > > > identifier so more care must be taken when combining PSBTs. > > > > Additionally, adding new inputs and outputs must be done such that > > > > signatures are not invalidated. This may be harder to specify. > > > > > > > > An important thing to note in this proposal are the fields > > > > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A > Bitcoin > > > > transaction only has a single locktime yet a PSBT may have multiple > > > > locktimes. To choose the locktime for the transaction, finalizers > must > > > > choose the maximum of all of the *_LOCKTIME fields. > > > > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as > those > > > > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum > locktime to > > > > be set. This field allows finalizers to choose a locktime that is > high > > > > enough for all inputs without needing to understand the scripts > > > > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to > use if > > > > no inputs require a particular locktime. > > > > > > > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT > v1 > > > > needs the version number bump to enforce backwards incompatibility. > > > > However once the inputs and outputs of a PSBT are decided, a PSBT > could > > > > be "downgraded" back to v0 by creating the unsigned transaction > from the > > > > above fields, and then dropping these new fields. > > > > > > > > If the list finds that these changes are reasonable, I will write > a PR > > > > to modify BIP 174 to incorporate them. > > > > > > > > Thanks, > > > > Andrew Chow > > > > > > > > > _______________________________________________ > > > bitcoin-dev mailing list > > > bitcoin-dev@lists•linuxfoundation.org > > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 18104 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-22 20:12 ` Andrew Chow 2020-12-23 3:30 ` fiatjaf @ 2020-12-23 21:32 ` Andrew Chow 2021-01-15 17:28 ` Andrew Chow 2021-01-06 23:26 ` Rusty Russell 2 siblings, 1 reply; 17+ messages in thread From: Andrew Chow @ 2020-12-23 21:32 UTC (permalink / raw) To: Bitcoin Protocol Discussion Hi All, The full modified BIP can be read at https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki. I will open a PR to the BIPs repo soon after further discussion on this. Andrew On 12/22/20 3:12 PM, Andrew Chow wrote: > Hi All, > > I have some updates on this after speaking with some people off-list. > > Firstly, the version number will be set to 2. In most discussions, this > proposal was being referred to as PSBT version 2, so it'll be easier and > clearer to set the version number to 2. > > For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, > there will be 2 of them, one for a time based lock time, and the other > for height based. These will be: > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 > * Key: empty > * Value: 32 bit unsigned little endian integer greater than or equal > to 500000000 representing the minimum Unix timestamp that this input > requires to be set as the transaction's lock time. Must be omitted in > PSBTv0, and may be omitted in PSBTv2 > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 > * Key: empty > * Value: 32 bit unsigned little endian integer less than 500000000 > representing the minimum block height that this input requires to be set > as the transaction's lock time. Must be omitted in PSBTv0, and may be > omitted in PSBTv2. > > Having two lock time fields is necessary due to the behavior where all > inputs must use the same type of lock time (height or time). Thus if an > input requires a particular type of lock time, it must set the requisite > field. Any new inputs being added must be able to accommodate all > existing inputs' lock time type. This means they either must not have a > lock time specified (i.e. no OP_CLTV involved), or have branches that > allow the acceptance of either type. If an input has a lock time type > that is incompatible with the rest of the transaction, it must not be added. > > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback > option if no input lock time fields are present. If there are input lock > times, all lock time calculations must ignore it. > > Any role which does lock time calculation will first check if there are > input lock time fields. If there are not, it must then check for a > PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the > transaction's lock time. If it does not, the lock time is 0. If there > are input lock time fields, it must choose the type which does not > invalidate any inputs. The lock time is then determined to be the > maximum value of all of the lock time fields for the chosen type. > > > Additionally, I would like to add a new global field: > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 > * Key: empty > * Value: A single byte as a boolean. 0 for False, 1 for True. All > other values ore prohibited. Must be omitted for PSBTv0, may be omitted > in PSBTv2. > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > outputs can be added to the PSBT. This flag may be set to True when > inputs and outputs are being updated, signed, and finalized. However > care must be taken when there are existing signatures. If this field is > omitted or set to False, no further inputs and outputs may be added to > the PSBT. > > Several rules must be followed to ensure that adding additional inputs > and outputs will not invalidate existing signatures. First, an input or > output adder must check for any existing signatures in all of the other > inputs. If there are none, the input or output may be added in any > position. If there are one or more signatures, each signature's sighash > type must be examined. Inputs may only be added if all existing > signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all > existing signatures use SIGHASH_NONE. If an input has a signature using > SIGHASH_SINGLE, the same number of inputs and outputs must be added > before that input and it's corresponding output. For all other sighash > types (i.e. SIGHASH_ALL and any future sighash types), no inputs or > outputs may be added to the PSBT. Specific exceptions can be made in the > future for additional sighash types. > > Furthermore, these newly added inputs must follow additional lock time > rules. Because all signatures, regardless of sighash type, sign the > transaction lock time, newly added inputs when there are existing > signatures must have the same type of lock time used in the transaction, > and must be less than or equal to the transaction lock time. It must not > cause the transaction lock time to change, otherwise the signatures will > be invalidated. > > > Lastly, to uniquely identify transactions for combiners, a txid can be > computed from the information present in the PSBT. Internally, combiners > can create an unsigned transaction given the transaction version, the > input prevouts, the outputs, and the computed locktime. This can then be > used to calculate a txid and thus used as a way to identify PSBTs. > Combiners will need to do this for all version 2 PSBTs in order to avoid > combining distinct transactions. > > > Andrew Chow > > On 12/9/20 5:25 PM, Andrew Chow wrote: >> Hi All, >> >> I would like to propose a new PSBT version that addresses a few >> deficiencies in the current PSBT v0. As this will be backwards >> incompatible, a new PSBT version will be used, v1. >> >> The primary change is to truly have all input and output data for each >> in their respective maps. Instead of having to parse an unsigned >> transaction and lookup some data from there, and other data from the >> correct map, all of the data for an input will be contained in its map. >> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. >> Thus I propose that the following fields be added: >> >> Global: >> * PSBT_GLOBAL_TX_VERSION = 0x02 >> * Key: empty >> * Value: 32-bit little endian unsigned integer for the transaction >> version number. Must be provided in PSBT v1 and omitted in v0. >> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 >> * Key: empty >> * Value: 32 bit little endian unsigned integer for the preferred >> transaction lock time. Must be omitted in PSBT v0. May be provided in >> PSBT v1, assumed to be 0 if not provided. >> * PSBT_GLOBAL_INPUT_COUNT = 0x04 >> * Key: empty >> * Value: Compact size unsigned integer. Number of inputs in this >> PSBT. Must be provided in PSBT v1 and omitted in v0. >> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 >> * Key: empty >> * Value: Compact size unsigned integer. Number of outputs in this >> PSBT. Must be provided in PSBT v1 and omitted in v0. >> >> Input: >> * PSBT_IN_PREVIOUS_TXID = 0x0e >> * Key: empty >> * Value: 32 byte txid of the previous transaction whose output at >> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and >> omitted in v0. >> * PSBT_IN_OUTPUT_INDEX = 0x0f >> * Key: empty >> * Value: 32 bit little endian integer for the index of the output >> being spent. Must be provided in PSBT v1 and omitted in v0. >> * PSBT_IN_SEQUENCE = 0x0f >> * Key: empty >> * Value: 32 bit unsigned little endian integer for the sequence >> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed >> to be max sequence (0xffffffff) if not provided. >> * PSBT_IN_REQUIRED_LOCKTIME = 0x10 >> * Key: empty >> * Value: 32 bit unsigned little endian integer for the lock time that >> this input requires. Must be omitted in PSBT v0. May be provided in PSBT >> v1, assumed to be 0 if not provided. >> >> Output: >> * PSBT_OUT_VALUE = 0x03 >> * Key: empty >> * Value: 64-bit unsigned little endian integer for the output's >> amount in satoshis. Must be provided in PSBT v1 and omitted in v0. >> * PSBT_OUT_OUTPUT_SCRIPT = 0x04 >> * Key: empty >> * Value: The script for this output. Otherwise known as the >> scriptPubKey. Must be provided in PSBT v1 and omitted in v0. >> >> This change allows for PSBT to be used in the construction of >> transactions. With these new fields, inputs and outputs can be added as >> needed. One caveat is that there is no longer a unique transaction >> identifier so more care must be taken when combining PSBTs. >> Additionally, adding new inputs and outputs must be done such that >> signatures are not invalidated. This may be harder to specify. >> >> An important thing to note in this proposal are the fields >> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin >> transaction only has a single locktime yet a PSBT may have multiple >> locktimes. To choose the locktime for the transaction, finalizers must >> choose the maximum of all of the *_LOCKTIME fields. >> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those >> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to >> be set. This field allows finalizers to choose a locktime that is high >> enough for all inputs without needing to understand the scripts >> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if >> no inputs require a particular locktime. >> >> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 >> needs the version number bump to enforce backwards incompatibility. >> However once the inputs and outputs of a PSBT are decided, a PSBT could >> be "downgraded" back to v0 by creating the unsigned transaction from the >> above fields, and then dropping these new fields. >> >> If the list finds that these changes are reasonable, I will write a PR >> to modify BIP 174 to incorporate them. >> >> Thanks, >> Andrew Chow ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-23 21:32 ` Andrew Chow @ 2021-01-15 17:28 ` Andrew Chow 2021-01-21 19:50 ` Andrew Chow 0 siblings, 1 reply; 17+ messages in thread From: Andrew Chow @ 2021-01-15 17:28 UTC (permalink / raw) To: Bitcoin Protocol Discussion Hi All, I've made some reorganization changes to the way that new PSBT versions should be handled in BIP 174 (see https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted as a separate BIP. The full document can be read at https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I have also included it in this email. I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and made a few modifications. First, the field will be named PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most PSBTs to not have to carry around a useless bitmap. Andrew *** <pre> BIP: PSBTv2 Layer: Applications Title: PSBT Version 2 Author: Andrew Chow <achow101@gmail•com> Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2 Status: Draft Type: Standards Track Created: 2021-01-14 License: BSD-2-Clause </pre> ==Introduction== ===Abstract=== This document proposes a second version of the Partially Signed Bitcoin Transaction format described in BIP 174 which allows for inputs and outputs to be added to the PSBT after creation. ===Copyright=== This BIP is licensed under the 2-clause BSD license. ===Motivation=== Partially Signed Bitcoin Transaction Version 0 as described in BIP 174 is unable to have new inputs and outputs be added to the transaction. The fixed global unsigned transaction cannot be changed which prevents any additional inputs or outputs to be added. PSBT Version 2 is intended to rectify this problem. An additional benficial side effect is that all information for a given input or output will be provided by its <tt><input-map></tt> or <tt><output-map></tt>. With Version 0, to retrieve all of the information for an input or output, data would need to be found in two locations: the <tt><input-map></tt>/<tt><output-map></tt> and the global unsigned transaction. PSBT Version 2 now moves all related information to one place. ==Specification== PSBT Version 2 (PSBTv2) only specifies new fields and field inclusion/exclusion requirements. <tt>PSBT_GLOBAL_UNSIGNED_TX</tt> must be excluded in PSBTv2. <tt>PSBT_GLOBAL_VERSION</tt> must be included in PSBTv2 and set to version number 2<ref>'''What happened to version number 1?''' Version number 1 is skipped because PSBT Version 0 has been colloquially referred to as version 1. Originally this BIP was to be version 1, but because it has been colloquially referred to as version 2 during its design phrase, it was decided to change the version number to 2 so that there would not be any confusion</ref>. The new global types for PSBT Version 2 are as follows: {| ! Name ! <tt><keytype></tt> ! <tt><keydata></tt> ! <tt><keydata></tt> Description ! <tt><valuedata></tt> ! <tt><valuedata></tt> Description ! Versions Requiring Inclusion ! Versions Requiring Exclusion ! Versions Allowing Inclusion |- | Transaction Version | <tt>PSBT_GLOBAL_TX_VERSION = 0x02</tt> | None | No key data | <tt><32-bit uint></tt> | The 32-bit little endian signed integer representing the version number of the transaction being created. Note that this is not the same as the PSBT version number specified by the PSBT_GLOBAL_VERSION field. | 2 | 0 | 2 |- | Fallback Locktime | <tt>PSBT_GLOBAL_FALLBACK_LOCKTIME = 0x03</tt> | None | No key data | <tt><32-bit uint></tt> | The 32-bit little endian unsigned integer representing the transaction locktime to use if no inputs specify a required locktime. | | 0 | 2 |- | Input Count | <tt>PSBT_GLOBAL_INPUT_COUNT = 0x04</tt> | None | No key data | <tt><compact size uint></tt> | Compact size unsigned integer representing the number of inputs in this PSBT. | 2 | 0 | 2 |- | Output Count | <tt>PSBT_GLOBAL_OUTPUT_COUNT = 0x05</tt> | None | No key data | <tt><compact size uint></tt> | Compact size unsigned integer representing the number of outputs in this PSBT. | 2 | 0 | 2 |- | Transaction Modifiable Flags | <tt>PSBT_GLOBAL_TX_MODIFIABLE = 0x06</tt> | None | No key data | <tt><single byte boolean> <single byte boolean></tt> | A single byte boolean (0 for False, 1 for True) representing whether inputs can be modified, referred to as the Inputs Modifiable Flag. This is followed by a single byte boolean representing whether outputs can be modified, referred to as the Outputs Modifiable Flag. | | 0 | 2 |- | SIGHASH_SINGLE Inputs | <tt>PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS = 0x07</tt> | None | No key data | <tt><bit vector></tt> | A bit vector representing which input indexes use SIGHASH_SINGLE. If the bit for an index is set to 1, then the input and output pair at that index are tied together with SIGHASH_SINGLE and must be moved together. | | 0 | 2 |} The new per-input types for PSBT Version 2 are defined as follows: {| ! Name ! <tt><keytype></tt> ! <tt><keydata></tt> ! <tt><keydata></tt> Description ! <tt><valuedata></tt> ! <tt><valuedata></tt> Description ! Versions Requiring Inclusion ! Versions Requiring Exclusion ! Versions Allowing Inclusion |- | Previous TXID | <tt>PSBT_IN_PREVIOUS_TXID = 0x0e</tt> | None | No key data | <tt><txid></tt> | 32 byte txid of the previous transaction whose output at PSBT_IN_OUTPUT_INDEX is being spent. | 2 | 0 | 2 |- | Spent Output Index | <tt>PSBT_IN_OUTPUT_INDEX = 0x0f</tt> | None | No key data | <tt><32-bit uint></tt> | 32 bit little endian integer representing the index of the output being spent in the transaction with the txid of PSBT_IN_PREVIOUS_TXID. | 2 | 0 | 2 |- | Sequence Number | <tt>PSBT_IN_SEQUENCE = 0x10</tt> | None | No key data | <tt><32-bit uint></tt> | The 32 bit unsigned little endian integer for the sequence number of this input. If omitted, the sequence number is assumed to be the final sequence number (0xffffffff). | | 0 | 2 |- | Required Time-based Locktime | <tt>PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x11</tt> | None | No key data | <tt><32-bit uint></tt> | 32 bit unsigned little endian integer greater than or equal to 500000000 representing the minimum Unix timestamp that this input requires to be set as the transaction's lock time. | | 0 | 2 |- | Required Height-based Locktime | <tt>PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x12</tt> | None | No key data | <tt><32-bit uiht></tt> | 32 bit unsigned little endian integer less than 500000000 representing the minimum block height that this input requires to be set as the transaction's lock time. | | 0 | 2 |} The new per-output types for PSBT Version 2 are defined as follows: {| ! Name ! <tt><keytype></tt> ! <tt><keydata></tt> ! <tt><keydata></tt> Description ! <tt><valuedata></tt> ! <tt><valuedata></tt> Description ! Versions Requiring Inclusion ! Versions Requiring Exclusion ! Versions Allowing Inclusion |- | Output Amount | <tt>PSBT_OUT_AMOUNT = 0x03</tt> | None | No key data | <tt><64-bit uint></tt> | 64 bit signed little endian integer representing the output's amount in satoshis. | 2 | 0 | 2 |- | Output Script | <tt>PSBT_OUT_SCRIPT = 0x03</tt> | None | No key data | <tt><script></tt> | The script for this output, also known as the scriptPubKey. Must be omitted in PSBTv0. Must be provided in PSBTv2. | 2 | 0 | 2 |} ===Determining Lock Time=== The nLockTime field of a transaction is determined by inspecting the PSBT_GLOBAL_PREFERRED_LOCKTIME and each input's PSBT_IN_REQUIRED_TIME_LOCKTIME and PSBT_IN_REQUIRED_HEIGHT_LOCKTIME fields. If none of the inputs have a PSBT_IN_REQUIRED_TIME_LOCKTIME and PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then PSBT_GLOBAL_PREFERRED_LOCKTIME must be used. If PSBT_GLOBAL_PREFERRED_LOCKTIME is not provided, then it is assumed to be 0. If one or more inuts have a PSBT_IN_REQUIRED_TIME_LOCKTIME or PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then the field chosen is the one which is supported by all of the inputs. This can be determined by looking at all of the inputs which specify a locktime in either of those fields, and choosing the field which is present in all of those inputs. Inputs not specifying a lock time field can take both types of lock times, as can those that specify both. The lock time chosen is then the maximum value of the chosen type of lock time. ===Unique Identification=== PSBTv2s can be uniquely identified by constructing an unsigned transaction given the information provided in the PSBT and computing the transaction ID of that transaction. Since PSBT_IN_SEQUENCE can be changed by Updaters and Combiners, the sequence number in this unsigned transaction must be set to 0 (not final, nor the sequence in PSBT_IN_SEQUENCE). The lock time in this unsigned transaction must be computed as described previously. ==Roles== PSBTv2 introduces new roles and modifies some existing roles. ===Creator=== In PSBTv2, the Creator initializes the PSBT with 0 inputs and 0 outputs. The PSBT version number is set to 2. The transaction version number must be set to at least 2. <ref>'''Why does the transaction version number need to be at least 2?''' The transaction version number is part of the validation rules for some features such as OP_CHECKSEQUENCEVERIFY. Since it is backwards compatible, and there are other ways to disable those features (e.g. through sequence numbers), it is easier to require transactions be able to support these features than to try to negotiate the transaction version number.</ref> The Creator should also PSBT_GLOBAL_PREFERRED_LOCKTIME. If the Creator is not also a Constructor and will be giving the PSBT to others to add inputs and outputs, the PSBT_GLOBAL_TX_MODIFIABLE field must be present and and the Inputs Modifiable and Outputs Modifiable flags set appropriately. If the Creator is a Constructor and no inputs and outputs will be added by other entities, PSBT_GLOBAL_TX_MODIFIABLE may be omitted. ===Constructor=== This Constructor is only present for PSBTv2. Once a Creator initializes the PSBT, a constructor will add inputs and outputs. Before any input or output may be added, the constructor must check the PSBT_GLOBAL_TX_MODIFIABLE field. Inputs may only be added if the Inputs Modifiable flag (first boolean) is True. Outputs may only be added if the Outputs Modifiable flag (second boolean) is True. If PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS is present and its bitvector indicates any inputs use SIGHASH_SINGLE, then the same number of inputs and outputs must be added before that input and its corresponding outputs. When an input or output is added, the corresponding PSBT_GLOBAL_INPUT_COUNT or PSBT_GLOBAL_OUTPUT_COUNT must be incremeted to reflect the number of inputs and outputs in the PSBT. When an input is added, it must have PSBT_IN_PREVIOUS_TXID and PSBT_IN_OUTPUT_INDEX set. When an output is added, it must have PSBT_OUT_VALUE and PSBT_OUT_OUTPUT_SCRIPT set. If the input has a required timelock, Constructors must set the requisite timelock field. If the input has a required time based timelock, then PSBT_IN_REQUIRED_TIME_TIMELOCK. If the input has a required height based timelock, then PSBT_IN_REQUIRED_HEIGHT_TIMELOCK. If an input has both types of timelocks, then both may be set. In some cases, an input that can allow both types, but a particular branch supporting only one type of timelock will be taken, then the type of timelock that will be used can be the only one set. When adding a new input, the new input must be compatible with the timelock types of all existing inputs. Since Bitcoin requires that a transaction uses only one type of timelock, all of the inputs must be able to support the same type of timelock. If the type of timelock is incompatible with the timelock type of the other inputs, then the input must not be added. Since a Constructor may be adding inputs to a PSBT that has inputs with existing signatures, the Constructor must be careful to not invalidate any existing signatures. The PSBT_GLOBAL_TX_MODIFIABLE and PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS fields caches this information so that constructors do not need to inspect every input. If the PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS bit vector indicates an input index uses SIGHASH_SINGLE, the same number of inputs and outputs must be added before that input and its corresponding output. When an input is added, the bit vector, if present, must be expanded to include a bit for this input in the correct position. When there are signatures, in addition to respecting the lock time rules described previously, the newly added inputs must not change the lock time used by the transaction. It cannot cause the lock time to change as that will invalidate all signatures since they all include the lock time regardless of the sighash type. A Constructor may choose to declare that no further inputs and outputs can be added to the transaction by setting the booleans in PSBT_GLOBAL_TX_MODIFIABLE to False or by removing this field entirely. A single entity is likely to be both a Creator and Constructor. ===Updater=== For PSBTv2, an Updater can set the sequence number. ===Signer=== For PSBTv2s, a signer must update the PSBT_GLOBAL_TX_MODIFIABLE and PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS fields after signing inputs so that these fields accurately reflects the state of the PSBT. If the Signer added a signature that does not use SIGHASH_ANYONECANPAY, the Input Modifiable flag must be set to False. If the Signer added a signature that does not use SIGHASH_NONE, the Outputs Modifiable flag must be set to False. If the Signer added a signature that uses SIGHASH_SINGLE, a PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS field must be added if not present, and the corresponding bit in its bit vector must be set to 1. ===Transaction Extractor=== For PSBTv2s, the transaction is constructed using the PSBTv2 fields. The lock time for this transaction is determined as described in the Determining Lock Time section. The Extractor should produce a fully valid, network serialized transaction if all inputs are complete. ==Compatibility== PSBTv2 shares the same gemeric format as PSBTv0 as defined in BIP 174. Parsers for PSBTv0 should be able to deserialize PSBTv2 with only changes to support the new fields. However PSBTv2 is incompatible with PSBTv0, and vice versa due to the use of the PSBT_GLOBAL_VERSION. This incompatibility is intentional so that PSBT_GLOBAL_UNSIGNED_TX could be removed in PSBTv2. However it is possible to convert a PSBTv2 to a PSBTv0 by creating an unsigned transaction from the PSBTv2 fields. ==Test Vectors== TBD ==Rationale== <references/> ==Reference implementation== The reference implementation of the PSBT format is available at https://github.com/achow101/bitcoin/tree/psbt2. On 12/23/20 4:32 PM, Andrew Chow wrote: > Hi All, > > The full modified BIP can be read at > https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki. > > I will open a PR to the BIPs repo soon after further discussion on this. > > > Andrew > > On 12/22/20 3:12 PM, Andrew Chow wrote: >> Hi All, >> >> I have some updates on this after speaking with some people off-list. >> >> Firstly, the version number will be set to 2. In most discussions, this >> proposal was being referred to as PSBT version 2, so it'll be easier and >> clearer to set the version number to 2. >> >> For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, >> there will be 2 of them, one for a time based lock time, and the other >> for height based. These will be: >> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 >> * Key: empty >> * Value: 32 bit unsigned little endian integer greater than or equal >> to 500000000 representing the minimum Unix timestamp that this input >> requires to be set as the transaction's lock time. Must be omitted in >> PSBTv0, and may be omitted in PSBTv2 >> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 >> * Key: empty >> * Value: 32 bit unsigned little endian integer less than 500000000 >> representing the minimum block height that this input requires to be set >> as the transaction's lock time. Must be omitted in PSBTv0, and may be >> omitted in PSBTv2. >> >> Having two lock time fields is necessary due to the behavior where all >> inputs must use the same type of lock time (height or time). Thus if an >> input requires a particular type of lock time, it must set the requisite >> field. Any new inputs being added must be able to accommodate all >> existing inputs' lock time type. This means they either must not have a >> lock time specified (i.e. no OP_CLTV involved), or have branches that >> allow the acceptance of either type. If an input has a lock time type >> that is incompatible with the rest of the transaction, it must not be added. >> >> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback >> option if no input lock time fields are present. If there are input lock >> times, all lock time calculations must ignore it. >> >> Any role which does lock time calculation will first check if there are >> input lock time fields. If there are not, it must then check for a >> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the >> transaction's lock time. If it does not, the lock time is 0. If there >> are input lock time fields, it must choose the type which does not >> invalidate any inputs. The lock time is then determined to be the >> maximum value of all of the lock time fields for the chosen type. >> >> >> Additionally, I would like to add a new global field: >> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 >> * Key: empty >> * Value: A single byte as a boolean. 0 for False, 1 for True. All >> other values ore prohibited. Must be omitted for PSBTv0, may be omitted >> in PSBTv2. >> >> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and >> outputs can be added to the PSBT. This flag may be set to True when >> inputs and outputs are being updated, signed, and finalized. However >> care must be taken when there are existing signatures. If this field is >> omitted or set to False, no further inputs and outputs may be added to >> the PSBT. >> >> Several rules must be followed to ensure that adding additional inputs >> and outputs will not invalidate existing signatures. First, an input or >> output adder must check for any existing signatures in all of the other >> inputs. If there are none, the input or output may be added in any >> position. If there are one or more signatures, each signature's sighash >> type must be examined. Inputs may only be added if all existing >> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all >> existing signatures use SIGHASH_NONE. If an input has a signature using >> SIGHASH_SINGLE, the same number of inputs and outputs must be added >> before that input and it's corresponding output. For all other sighash >> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or >> outputs may be added to the PSBT. Specific exceptions can be made in the >> future for additional sighash types. >> >> Furthermore, these newly added inputs must follow additional lock time >> rules. Because all signatures, regardless of sighash type, sign the >> transaction lock time, newly added inputs when there are existing >> signatures must have the same type of lock time used in the transaction, >> and must be less than or equal to the transaction lock time. It must not >> cause the transaction lock time to change, otherwise the signatures will >> be invalidated. >> >> >> Lastly, to uniquely identify transactions for combiners, a txid can be >> computed from the information present in the PSBT. Internally, combiners >> can create an unsigned transaction given the transaction version, the >> input prevouts, the outputs, and the computed locktime. This can then be >> used to calculate a txid and thus used as a way to identify PSBTs. >> Combiners will need to do this for all version 2 PSBTs in order to avoid >> combining distinct transactions. >> >> >> Andrew Chow >> >> On 12/9/20 5:25 PM, Andrew Chow wrote: >>> Hi All, >>> >>> I would like to propose a new PSBT version that addresses a few >>> deficiencies in the current PSBT v0. As this will be backwards >>> incompatible, a new PSBT version will be used, v1. >>> >>> The primary change is to truly have all input and output data for each >>> in their respective maps. Instead of having to parse an unsigned >>> transaction and lookup some data from there, and other data from the >>> correct map, all of the data for an input will be contained in its map. >>> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. >>> Thus I propose that the following fields be added: >>> >>> Global: >>> * PSBT_GLOBAL_TX_VERSION = 0x02 >>> * Key: empty >>> * Value: 32-bit little endian unsigned integer for the transaction >>> version number. Must be provided in PSBT v1 and omitted in v0. >>> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 >>> * Key: empty >>> * Value: 32 bit little endian unsigned integer for the preferred >>> transaction lock time. Must be omitted in PSBT v0. May be provided in >>> PSBT v1, assumed to be 0 if not provided. >>> * PSBT_GLOBAL_INPUT_COUNT = 0x04 >>> * Key: empty >>> * Value: Compact size unsigned integer. Number of inputs in this >>> PSBT. Must be provided in PSBT v1 and omitted in v0. >>> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 >>> * Key: empty >>> * Value: Compact size unsigned integer. Number of outputs in this >>> PSBT. Must be provided in PSBT v1 and omitted in v0. >>> >>> Input: >>> * PSBT_IN_PREVIOUS_TXID = 0x0e >>> * Key: empty >>> * Value: 32 byte txid of the previous transaction whose output at >>> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and >>> omitted in v0. >>> * PSBT_IN_OUTPUT_INDEX = 0x0f >>> * Key: empty >>> * Value: 32 bit little endian integer for the index of the output >>> being spent. Must be provided in PSBT v1 and omitted in v0. >>> * PSBT_IN_SEQUENCE = 0x0f >>> * Key: empty >>> * Value: 32 bit unsigned little endian integer for the sequence >>> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed >>> to be max sequence (0xffffffff) if not provided. >>> * PSBT_IN_REQUIRED_LOCKTIME = 0x10 >>> * Key: empty >>> * Value: 32 bit unsigned little endian integer for the lock time that >>> this input requires. Must be omitted in PSBT v0. May be provided in PSBT >>> v1, assumed to be 0 if not provided. >>> >>> Output: >>> * PSBT_OUT_VALUE = 0x03 >>> * Key: empty >>> * Value: 64-bit unsigned little endian integer for the output's >>> amount in satoshis. Must be provided in PSBT v1 and omitted in v0. >>> * PSBT_OUT_OUTPUT_SCRIPT = 0x04 >>> * Key: empty >>> * Value: The script for this output. Otherwise known as the >>> scriptPubKey. Must be provided in PSBT v1 and omitted in v0. >>> >>> This change allows for PSBT to be used in the construction of >>> transactions. With these new fields, inputs and outputs can be added as >>> needed. One caveat is that there is no longer a unique transaction >>> identifier so more care must be taken when combining PSBTs. >>> Additionally, adding new inputs and outputs must be done such that >>> signatures are not invalidated. This may be harder to specify. >>> >>> An important thing to note in this proposal are the fields >>> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin >>> transaction only has a single locktime yet a PSBT may have multiple >>> locktimes. To choose the locktime for the transaction, finalizers must >>> choose the maximum of all of the *_LOCKTIME fields. >>> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those >>> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to >>> be set. This field allows finalizers to choose a locktime that is high >>> enough for all inputs without needing to understand the scripts >>> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if >>> no inputs require a particular locktime. >>> >>> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 >>> needs the version number bump to enforce backwards incompatibility. >>> However once the inputs and outputs of a PSBT are decided, a PSBT could >>> be "downgraded" back to v0 by creating the unsigned transaction from the >>> above fields, and then dropping these new fields. >>> >>> If the list finds that these changes are reasonable, I will write a PR >>> to modify BIP 174 to incorporate them. >>> >>> Thanks, >>> Andrew Chow ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2021-01-15 17:28 ` Andrew Chow @ 2021-01-21 19:50 ` Andrew Chow 0 siblings, 0 replies; 17+ messages in thread From: Andrew Chow @ 2021-01-21 19:50 UTC (permalink / raw) To: Bitcoin Protocol Discussion While working on the reference implementation for this, it occurred to me that the Inputs Modifiable flag needs to be more than just a boolean. If there are existing signatures in the PSBT, then any added inputs cannot change the transaction's locktime as all signatures, regardless of sighash type, commit to the locktime. Additionally if an input with a signature is being added, it also needs to set the locktime for the transaction. It also seems like the SIGHASH_SINGLE bitmap is unnecessary. Signers can instead iterate all inputs, check for existing signatures, and extract the sighash byte from those signatures to determine whether any are SIGHASH_SINGLE. This bitmap doesn't seem to provide much benefit and also causes headaches for implementation. So I've decided to remove it. But it is still useful to know that there are SIGHASH_SINGLE inputs and that iteration of the inputs vector will be necessary. It is also useful to know that there are already some signatures in the transaction so the locktime must be preserved. Thus I would like to change PSBT_GLOBAL_TX_MODIFIABLE to include those. I propose making PSBT_GLOBAL_TX_MODIFIABLE an 8 bit unsigned little endian integer that is treated as a bit field. If bit 0 is set, inputs may be added. This will be the Inputs Modifiable flag. If bit 1 is set, outputs may be added. This will be the Outputs Modifiable flag. If bit 2 is set, the transaction contains signatures and locktime must be preserved. This will be the Has Signatures flag. If bit 3 is set, the transaction contains SIGHASH_SINGLE inputs and their index pairings must be preserved. This will be the Has SIGHASH_SINGLE flag. Changing PSBT_GLOBAL_TX_MODIFIABLE to a bitfield like this allows us to include more conditions that need to be considered when adding inputs and outputs. I think these are all of the conditions for now, but with 8 bits, there is still some space for additional conditions in the future. Perhaps it should be changed to be larger if we think there will be more conditions, but I think that is unlikely. Andrew On 1/15/21 12:28 PM, Andrew Chow wrote: > Hi All, > > I've made some reorganization changes to the way that new PSBT versions > should be handled in BIP 174 (see > https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted > as a separate BIP. The full document can be read at > https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I > have also included it in this email. > > I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and > made a few modifications. First, the field will be named > PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and > outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as > a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most > PSBTs to not have to carry around a useless bitmap. > > Andrew > > *** > > <pre> > BIP: PSBTv2 > Layer: Applications > Title: PSBT Version 2 > Author: Andrew Chow <achow101@gmail•com> > Comments-Summary: No comments yet. > Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2 > Status: Draft > Type: Standards Track > Created: 2021-01-14 > License: BSD-2-Clause > </pre> > > ==Introduction== > > ===Abstract=== > > This document proposes a second version of the Partially Signed Bitcoin > Transaction format > described in BIP 174 which allows for inputs and outputs to be added to > the PSBT after creation. > > ===Copyright=== > > This BIP is licensed under the 2-clause BSD license. > > ===Motivation=== > > Partially Signed Bitcoin Transaction Version 0 as described in BIP 174 > is unable to have new > inputs and outputs be added to the transaction. The fixed global > unsigned transaction > cannot be changed which prevents any additional inputs or outputs to be > added. > PSBT Version 2 is intended to rectify this problem. > > An additional benficial side effect is that all information for a given > input or output will be > provided by its <tt><input-map></tt> or <tt><output-map></tt>. With > Version 0, to retrieve > all of the information for an input or output, data would need to be > found in two locations: > the <tt><input-map></tt>/<tt><output-map></tt> and the global unsigned > transaction. PSBT > Version 2 now moves all related information to one place. > > ==Specification== > > PSBT Version 2 (PSBTv2) only specifies new fields and field > inclusion/exclusion requirements. > > <tt>PSBT_GLOBAL_UNSIGNED_TX</tt> must be excluded in PSBTv2. > <tt>PSBT_GLOBAL_VERSION</tt> must be included in PSBTv2 and set to > version number 2<ref>'''What happened to version number 1?''' > Version number 1 is skipped because PSBT Version 0 has been colloquially > referred to as version 1. Originally this BIP was to be > version 1, but because it has been colloquially referred to as version 2 > during its design phrase, it was decided to change the > version number to 2 so that there would not be any confusion</ref>. > > The new global types for PSBT Version 2 are as follows: > > {| > ! Name > ! <tt><keytype></tt> > ! <tt><keydata></tt> > ! <tt><keydata></tt> Description > ! <tt><valuedata></tt> > ! <tt><valuedata></tt> Description > ! Versions Requiring Inclusion > ! Versions Requiring Exclusion > ! Versions Allowing Inclusion > |- > | Transaction Version > | <tt>PSBT_GLOBAL_TX_VERSION = 0x02</tt> > | None > | No key data > | <tt><32-bit uint></tt> > | The 32-bit little endian signed integer representing the version > number of the transaction being created. Note that this is not the same > as the PSBT version number specified by the PSBT_GLOBAL_VERSION field. > | 2 > | 0 > | 2 > |- > | Fallback Locktime > | <tt>PSBT_GLOBAL_FALLBACK_LOCKTIME = 0x03</tt> > | None > | No key data > | <tt><32-bit uint></tt> > | The 32-bit little endian unsigned integer representing the transaction > locktime to use if no inputs specify a required locktime. > | > | 0 > | 2 > |- > | Input Count > | <tt>PSBT_GLOBAL_INPUT_COUNT = 0x04</tt> > | None > | No key data > | <tt><compact size uint></tt> > | Compact size unsigned integer representing the number of inputs in > this PSBT. > | 2 > | 0 > | 2 > |- > | Output Count > | <tt>PSBT_GLOBAL_OUTPUT_COUNT = 0x05</tt> > | None > | No key data > | <tt><compact size uint></tt> > | Compact size unsigned integer representing the number of outputs in > this PSBT. > | 2 > | 0 > | 2 > |- > | Transaction Modifiable Flags > | <tt>PSBT_GLOBAL_TX_MODIFIABLE = 0x06</tt> > | None > | No key data > | <tt><single byte boolean> <single byte boolean></tt> > | A single byte boolean (0 for False, 1 for True) representing whether > inputs can be modified, referred to as the Inputs Modifiable Flag. This > is followed by a single byte boolean representing whether outputs can be > modified, referred to as the Outputs Modifiable Flag. > | > | 0 > | 2 > |- > | SIGHASH_SINGLE Inputs > | <tt>PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS = 0x07</tt> > | None > | No key data > | <tt><bit vector></tt> > | A bit vector representing which input indexes use SIGHASH_SINGLE. If > the bit for an index is set to 1, then the input and output pair at that > index are tied together with SIGHASH_SINGLE and must be moved together. > | > | 0 > | 2 > |} > > The new per-input types for PSBT Version 2 are defined as follows: > > {| > ! Name > ! <tt><keytype></tt> > ! <tt><keydata></tt> > ! <tt><keydata></tt> Description > ! <tt><valuedata></tt> > ! <tt><valuedata></tt> Description > ! Versions Requiring Inclusion > ! Versions Requiring Exclusion > ! Versions Allowing Inclusion > |- > | Previous TXID > | <tt>PSBT_IN_PREVIOUS_TXID = 0x0e</tt> > | None > | No key data > | <tt><txid></tt> > | 32 byte txid of the previous transaction whose output at > PSBT_IN_OUTPUT_INDEX is being spent. > | 2 > | 0 > | 2 > |- > | Spent Output Index > | <tt>PSBT_IN_OUTPUT_INDEX = 0x0f</tt> > | None > | No key data > | <tt><32-bit uint></tt> > | 32 bit little endian integer representing the index of the output > being spent in the transaction with the txid of PSBT_IN_PREVIOUS_TXID. > | 2 > | 0 > | 2 > |- > | Sequence Number > | <tt>PSBT_IN_SEQUENCE = 0x10</tt> > | None > | No key data > | <tt><32-bit uint></tt> > | The 32 bit unsigned little endian integer for the sequence number of > this input. If omitted, the sequence number is assumed to be the final > sequence number (0xffffffff). > | > | 0 > | 2 > |- > | Required Time-based Locktime > | <tt>PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x11</tt> > | None > | No key data > | <tt><32-bit uint></tt> > | 32 bit unsigned little endian integer greater than or equal to > 500000000 representing the minimum Unix timestamp that this input > requires to be set as the transaction's lock time. > | > | 0 > | 2 > |- > | Required Height-based Locktime > | <tt>PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x12</tt> > | None > | No key data > | <tt><32-bit uiht></tt> > | 32 bit unsigned little endian integer less than 500000000 representing > the minimum block height that this input requires to be set as the > transaction's lock time. > | > | 0 > | 2 > |} > > The new per-output types for PSBT Version 2 are defined as follows: > > {| > ! Name > ! <tt><keytype></tt> > ! <tt><keydata></tt> > ! <tt><keydata></tt> Description > ! <tt><valuedata></tt> > ! <tt><valuedata></tt> Description > ! Versions Requiring Inclusion > ! Versions Requiring Exclusion > ! Versions Allowing Inclusion > |- > | Output Amount > | <tt>PSBT_OUT_AMOUNT = 0x03</tt> > | None > | No key data > | <tt><64-bit uint></tt> > | 64 bit signed little endian integer representing the output's amount > in satoshis. > | 2 > | 0 > | 2 > |- > | Output Script > | <tt>PSBT_OUT_SCRIPT = 0x03</tt> > | None > | No key data > | <tt><script></tt> > | The script for this output, also known as the scriptPubKey. Must be > omitted in PSBTv0. Must be provided in PSBTv2. > | 2 > | 0 > | 2 > |} > > ===Determining Lock Time=== > > The nLockTime field of a transaction is determined by inspecting the > PSBT_GLOBAL_PREFERRED_LOCKTIME and each input's > PSBT_IN_REQUIRED_TIME_LOCKTIME and PSBT_IN_REQUIRED_HEIGHT_LOCKTIME fields. > If none of the inputs have a PSBT_IN_REQUIRED_TIME_LOCKTIME and > PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then PSBT_GLOBAL_PREFERRED_LOCKTIME > must be used. > If PSBT_GLOBAL_PREFERRED_LOCKTIME is not provided, then it is assumed to > be 0. > > If one or more inuts have a PSBT_IN_REQUIRED_TIME_LOCKTIME or > PSBT_IN_REQUIRED_HEIGHT_LOCKTIME, then the field chosen is the one which > is supported by all of the inputs. > This can be determined by looking at all of the inputs which specify a > locktime in either of those fields, and choosing the field which is > present in all of those inputs. > Inputs not specifying a lock time field can take both types of lock > times, as can those that specify both. > The lock time chosen is then the maximum value of the chosen type of > lock time. > > ===Unique Identification=== > > PSBTv2s can be uniquely identified by constructing an unsigned > transaction given the information provided in the PSBT and computing the > transaction ID of that transaction. > Since PSBT_IN_SEQUENCE can be changed by Updaters and Combiners, the > sequence number in this unsigned transaction must be set to 0 (not > final, nor the sequence in PSBT_IN_SEQUENCE). > The lock time in this unsigned transaction must be computed as described > previously. > > ==Roles== > > PSBTv2 introduces new roles and modifies some existing roles. > > ===Creator=== > > In PSBTv2, the Creator initializes the PSBT with 0 inputs and 0 outputs. > The PSBT version number is set to 2. The transaction version number must > be set to at least 2. <ref>'''Why does the transaction version number > need to be at least 2?''' The transaction version number is part of the > validation rules for some features such as OP_CHECKSEQUENCEVERIFY. Since > it is backwards compatible, and there are other ways to disable those > features (e.g. through sequence numbers), it is easier to require > transactions be able to support these features than to try to negotiate > the transaction version number.</ref> > The Creator should also PSBT_GLOBAL_PREFERRED_LOCKTIME. > If the Creator is not also a Constructor and will be giving the PSBT to > others to add inputs and outputs, the PSBT_GLOBAL_TX_MODIFIABLE field > must be present and and the Inputs Modifiable and Outputs Modifiable > flags set appropriately. > If the Creator is a Constructor and no inputs and outputs will be added > by other entities, PSBT_GLOBAL_TX_MODIFIABLE may be omitted. > > ===Constructor=== > > This Constructor is only present for PSBTv2. > Once a Creator initializes the PSBT, a constructor will add inputs and > outputs. > Before any input or output may be added, the constructor must check the > PSBT_GLOBAL_TX_MODIFIABLE field. > Inputs may only be added if the Inputs Modifiable flag (first boolean) > is True. > Outputs may only be added if the Outputs Modifiable flag (second > boolean) is True. > If PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS is present and its bitvector > indicates any inputs use SIGHASH_SINGLE, then the same number of inputs > and outputs must be added before that input and its corresponding outputs. > > When an input or output is added, the corresponding > PSBT_GLOBAL_INPUT_COUNT or PSBT_GLOBAL_OUTPUT_COUNT must be incremeted > to reflect the number of inputs and outputs in the PSBT. > When an input is added, it must have PSBT_IN_PREVIOUS_TXID and > PSBT_IN_OUTPUT_INDEX set. > When an output is added, it must have PSBT_OUT_VALUE and > PSBT_OUT_OUTPUT_SCRIPT set. > If the input has a required timelock, Constructors must set the > requisite timelock field. > If the input has a required time based timelock, then > PSBT_IN_REQUIRED_TIME_TIMELOCK. > If the input has a required height based timelock, then > PSBT_IN_REQUIRED_HEIGHT_TIMELOCK. > If an input has both types of timelocks, then both may be set. > In some cases, an input that can allow both types, but a particular > branch supporting only one type of timelock will be taken, then the type > of timelock that will be used can be the only one set. > > When adding a new input, the new input must be compatible with the > timelock types of all existing inputs. > Since Bitcoin requires that a transaction uses only one type of > timelock, all of the inputs must be able to support the same type of > timelock. > If the type of timelock is incompatible with the timelock type of the > other inputs, then the input must not be added. > > Since a Constructor may be adding inputs to a PSBT that has inputs with > existing signatures, the Constructor must be careful to not invalidate > any existing signatures. > The PSBT_GLOBAL_TX_MODIFIABLE and PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS > fields caches this information so that constructors do not need to > inspect every input. > If the PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS bit vector indicates an input > index uses SIGHASH_SINGLE, the same number of inputs and outputs must be > added before that input and its corresponding output. > When an input is added, the bit vector, if present, must be expanded to > include a bit for this input in the correct position. > When there are signatures, in addition to respecting the lock time rules > described previously, the newly added inputs must not change the lock > time used by the transaction. > It cannot cause the lock time to change as that will invalidate all > signatures since they all include the lock time regardless of the > sighash type. > > A Constructor may choose to declare that no further inputs and outputs > can be added to the transaction by setting the booleans in > PSBT_GLOBAL_TX_MODIFIABLE to False or by removing this field entirely. > > A single entity is likely to be both a Creator and Constructor. > > ===Updater=== > > For PSBTv2, an Updater can set the sequence number. > > ===Signer=== > > For PSBTv2s, a signer must update the PSBT_GLOBAL_TX_MODIFIABLE and > PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS fields after signing inputs so > that these fields accurately reflects the state of the PSBT. > If the Signer added a signature that does not use SIGHASH_ANYONECANPAY, > the Input Modifiable flag must be set to False. > If the Signer added a signature that does not use SIGHASH_NONE, the > Outputs Modifiable flag must be set to False. > If the Signer added a signature that uses SIGHASH_SINGLE, a > PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS field must be added if not present, > and the corresponding bit in its bit vector must be set to 1. > > ===Transaction Extractor=== > > For PSBTv2s, the transaction is constructed using the PSBTv2 fields. > The lock time for this transaction is determined as described in the > Determining Lock Time section. > The Extractor should produce a fully valid, network serialized > transaction if all inputs are complete. > > ==Compatibility== > > PSBTv2 shares the same gemeric format as PSBTv0 as defined in BIP 174. > Parsers for PSBTv0 should > be able to deserialize PSBTv2 with only changes to support the new fields. > > However PSBTv2 is incompatible with PSBTv0, and vice versa due to the > use of the PSBT_GLOBAL_VERSION. > This incompatibility is intentional so that PSBT_GLOBAL_UNSIGNED_TX > could be removed in PSBTv2. > However it is possible to convert a PSBTv2 to a PSBTv0 by creating an > unsigned > transaction from the PSBTv2 fields. > > ==Test Vectors== > > TBD > > ==Rationale== > > <references/> > > ==Reference implementation== > > The reference implementation of the PSBT format is available at > https://github.com/achow101/bitcoin/tree/psbt2. > > > On 12/23/20 4:32 PM, Andrew Chow wrote: >> Hi All, >> >> The full modified BIP can be read at >> https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki. >> >> I will open a PR to the BIPs repo soon after further discussion on this. >> >> >> Andrew >> >> On 12/22/20 3:12 PM, Andrew Chow wrote: >>> Hi All, >>> >>> I have some updates on this after speaking with some people off-list. >>> >>> Firstly, the version number will be set to 2. In most discussions, this >>> proposal was being referred to as PSBT version 2, so it'll be easier and >>> clearer to set the version number to 2. >>> >>> For lock times, instead of a single PSBT_IN_REQUIRED_LOCKTIME field, >>> there will be 2 of them, one for a time based lock time, and the other >>> for height based. These will be: >>> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10 >>> * Key: empty >>> * Value: 32 bit unsigned little endian integer greater than or equal >>> to 500000000 representing the minimum Unix timestamp that this input >>> requires to be set as the transaction's lock time. Must be omitted in >>> PSBTv0, and may be omitted in PSBTv2 >>> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11 >>> * Key: empty >>> * Value: 32 bit unsigned little endian integer less than 500000000 >>> representing the minimum block height that this input requires to be set >>> as the transaction's lock time. Must be omitted in PSBTv0, and may be >>> omitted in PSBTv2. >>> >>> Having two lock time fields is necessary due to the behavior where all >>> inputs must use the same type of lock time (height or time). Thus if an >>> input requires a particular type of lock time, it must set the requisite >>> field. Any new inputs being added must be able to accommodate all >>> existing inputs' lock time type. This means they either must not have a >>> lock time specified (i.e. no OP_CLTV involved), or have branches that >>> allow the acceptance of either type. If an input has a lock time type >>> that is incompatible with the rest of the transaction, it must not be added. >>> >>> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback >>> option if no input lock time fields are present. If there are input lock >>> times, all lock time calculations must ignore it. >>> >>> Any role which does lock time calculation will first check if there are >>> input lock time fields. If there are not, it must then check for a >>> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the >>> transaction's lock time. If it does not, the lock time is 0. If there >>> are input lock time fields, it must choose the type which does not >>> invalidate any inputs. The lock time is then determined to be the >>> maximum value of all of the lock time fields for the chosen type. >>> >>> >>> Additionally, I would like to add a new global field: >>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 >>> * Key: empty >>> * Value: A single byte as a boolean. 0 for False, 1 for True. All >>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted >>> in PSBTv2. >>> >>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and >>> outputs can be added to the PSBT. This flag may be set to True when >>> inputs and outputs are being updated, signed, and finalized. However >>> care must be taken when there are existing signatures. If this field is >>> omitted or set to False, no further inputs and outputs may be added to >>> the PSBT. >>> >>> Several rules must be followed to ensure that adding additional inputs >>> and outputs will not invalidate existing signatures. First, an input or >>> output adder must check for any existing signatures in all of the other >>> inputs. If there are none, the input or output may be added in any >>> position. If there are one or more signatures, each signature's sighash >>> type must be examined. Inputs may only be added if all existing >>> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all >>> existing signatures use SIGHASH_NONE. If an input has a signature using >>> SIGHASH_SINGLE, the same number of inputs and outputs must be added >>> before that input and it's corresponding output. For all other sighash >>> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or >>> outputs may be added to the PSBT. Specific exceptions can be made in the >>> future for additional sighash types. >>> >>> Furthermore, these newly added inputs must follow additional lock time >>> rules. Because all signatures, regardless of sighash type, sign the >>> transaction lock time, newly added inputs when there are existing >>> signatures must have the same type of lock time used in the transaction, >>> and must be less than or equal to the transaction lock time. It must not >>> cause the transaction lock time to change, otherwise the signatures will >>> be invalidated. >>> >>> >>> Lastly, to uniquely identify transactions for combiners, a txid can be >>> computed from the information present in the PSBT. Internally, combiners >>> can create an unsigned transaction given the transaction version, the >>> input prevouts, the outputs, and the computed locktime. This can then be >>> used to calculate a txid and thus used as a way to identify PSBTs. >>> Combiners will need to do this for all version 2 PSBTs in order to avoid >>> combining distinct transactions. >>> >>> >>> Andrew Chow >>> >>> On 12/9/20 5:25 PM, Andrew Chow wrote: >>>> Hi All, >>>> >>>> I would like to propose a new PSBT version that addresses a few >>>> deficiencies in the current PSBT v0. As this will be backwards >>>> incompatible, a new PSBT version will be used, v1. >>>> >>>> The primary change is to truly have all input and output data for each >>>> in their respective maps. Instead of having to parse an unsigned >>>> transaction and lookup some data from there, and other data from the >>>> correct map, all of the data for an input will be contained in its map. >>>> Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. >>>> Thus I propose that the following fields be added: >>>> >>>> Global: >>>> * PSBT_GLOBAL_TX_VERSION = 0x02 >>>> * Key: empty >>>> * Value: 32-bit little endian unsigned integer for the transaction >>>> version number. Must be provided in PSBT v1 and omitted in v0. >>>> * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 >>>> * Key: empty >>>> * Value: 32 bit little endian unsigned integer for the preferred >>>> transaction lock time. Must be omitted in PSBT v0. May be provided in >>>> PSBT v1, assumed to be 0 if not provided. >>>> * PSBT_GLOBAL_INPUT_COUNT = 0x04 >>>> * Key: empty >>>> * Value: Compact size unsigned integer. Number of inputs in this >>>> PSBT. Must be provided in PSBT v1 and omitted in v0. >>>> * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 >>>> * Key: empty >>>> * Value: Compact size unsigned integer. Number of outputs in this >>>> PSBT. Must be provided in PSBT v1 and omitted in v0. >>>> >>>> Input: >>>> * PSBT_IN_PREVIOUS_TXID = 0x0e >>>> * Key: empty >>>> * Value: 32 byte txid of the previous transaction whose output at >>>> PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and >>>> omitted in v0. >>>> * PSBT_IN_OUTPUT_INDEX = 0x0f >>>> * Key: empty >>>> * Value: 32 bit little endian integer for the index of the output >>>> being spent. Must be provided in PSBT v1 and omitted in v0. >>>> * PSBT_IN_SEQUENCE = 0x0f >>>> * Key: empty >>>> * Value: 32 bit unsigned little endian integer for the sequence >>>> number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed >>>> to be max sequence (0xffffffff) if not provided. >>>> * PSBT_IN_REQUIRED_LOCKTIME = 0x10 >>>> * Key: empty >>>> * Value: 32 bit unsigned little endian integer for the lock time that >>>> this input requires. Must be omitted in PSBT v0. May be provided in PSBT >>>> v1, assumed to be 0 if not provided. >>>> >>>> Output: >>>> * PSBT_OUT_VALUE = 0x03 >>>> * Key: empty >>>> * Value: 64-bit unsigned little endian integer for the output's >>>> amount in satoshis. Must be provided in PSBT v1 and omitted in v0. >>>> * PSBT_OUT_OUTPUT_SCRIPT = 0x04 >>>> * Key: empty >>>> * Value: The script for this output. Otherwise known as the >>>> scriptPubKey. Must be provided in PSBT v1 and omitted in v0. >>>> >>>> This change allows for PSBT to be used in the construction of >>>> transactions. With these new fields, inputs and outputs can be added as >>>> needed. One caveat is that there is no longer a unique transaction >>>> identifier so more care must be taken when combining PSBTs. >>>> Additionally, adding new inputs and outputs must be done such that >>>> signatures are not invalidated. This may be harder to specify. >>>> >>>> An important thing to note in this proposal are the fields >>>> PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin >>>> transaction only has a single locktime yet a PSBT may have multiple >>>> locktimes. To choose the locktime for the transaction, finalizers must >>>> choose the maximum of all of the *_LOCKTIME fields. >>>> PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those >>>> involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to >>>> be set. This field allows finalizers to choose a locktime that is high >>>> enough for all inputs without needing to understand the scripts >>>> involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if >>>> no inputs require a particular locktime. >>>> >>>> As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 >>>> needs the version number bump to enforce backwards incompatibility. >>>> However once the inputs and outputs of a PSBT are decided, a PSBT could >>>> be "downgraded" back to v0 by creating the unsigned transaction from the >>>> above fields, and then dropping these new fields. >>>> >>>> If the list finds that these changes are reasonable, I will write a PR >>>> to modify BIP 174 to incorporate them. >>>> >>>> Thanks, >>>> Andrew Chow ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-22 20:12 ` Andrew Chow 2020-12-23 3:30 ` fiatjaf 2020-12-23 21:32 ` Andrew Chow @ 2021-01-06 23:26 ` Rusty Russell 2021-01-06 23:48 ` Andrew Chow 2 siblings, 1 reply; 17+ messages in thread From: Rusty Russell @ 2021-01-06 23:26 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion Hi Andrew et al, Very excited to see this progress; thanks for doing all the work! Sorry for the delayed feedback, I didn't get to this before the break. > Additionally, I would like to add a new global field: > * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 > * Key: empty > * Value: A single byte as a boolean. 0 for False, 1 for True. All > other values ore prohibited. Must be omitted for PSBTv0, may be omitted > in PSBTv2. > > PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and > outputs can be added to the PSBT. This flag may be set to True when > inputs and outputs are being updated, signed, and finalized. However > care must be taken when there are existing signatures. If this field is > omitted or set to False, no further inputs and outputs may be added to > the PSBT. I wonder if this can be flagged simply by omitting the (AFAICT redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT? What are the purposes of those fields? For our uses, there would be no signatures at this stage; it's simply a subdivision of the Creator role. This role would be terminated by removing the under-construction marker. For this, it could be clear that such an under-construction PSBT SHOULD NOT be signed. Otherwise, if an explicit marker is required, I would omit the value and simply use its existence to as a flag. Having two "false" values is simply asking for trouble. Thanks! Rusty. PS. Perhaps we should change the name to PBT (Partial Bitcoin Transaction) now, since it's more than just signing... ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2021-01-06 23:26 ` Rusty Russell @ 2021-01-06 23:48 ` Andrew Chow 2021-01-08 0:40 ` Rusty Russell 0 siblings, 1 reply; 17+ messages in thread From: Andrew Chow @ 2021-01-06 23:48 UTC (permalink / raw) To: Rusty Russell, Bitcoin Protocol Discussion Hi Rusty, On 1/6/21 6:26 PM, Rusty Russell wrote: > Hi Andrew et al, > > Very excited to see this progress; thanks for doing all the > work! Sorry for the delayed feedback, I didn't get to this before the > break. > >> Additionally, I would like to add a new global field: >> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 >> * Key: empty >> * Value: A single byte as a boolean. 0 for False, 1 for True. All >> other values ore prohibited. Must be omitted for PSBTv0, may be omitted >> in PSBTv2. >> >> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and >> outputs can be added to the PSBT. This flag may be set to True when >> inputs and outputs are being updated, signed, and finalized. However >> care must be taken when there are existing signatures. If this field is >> omitted or set to False, no further inputs and outputs may be added to >> the PSBT. > I wonder if this can be flagged simply by omitting the (AFAICT > redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT? What > are the purposes of those fields? The purpose of those fields is to know how many input and output maps there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine whether a map is an input map or an output map. So the counts are there to allow that. > For our uses, there would be no signatures at this stage; it's simply a > subdivision of the Creator role. This role would be terminated by > removing the under-construction marker. For this, it could be clear > that such an under-construction PSBT SHOULD NOT be signed. There are some protocols where signed inputs are added to transactions. > Otherwise, if an explicit marker is required, I would omit the value and > simply use its existence to as a flag. Having two "false" values is > simply asking for trouble. Seems reasonable. Andrew > Thanks! > Rusty. > PS. Perhaps we should change the name to PBT (Partial Bitcoin > Transaction) now, since it's more than just signing... ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2021-01-06 23:48 ` Andrew Chow @ 2021-01-08 0:40 ` Rusty Russell 2021-01-14 17:07 ` Andrew Chow 0 siblings, 1 reply; 17+ messages in thread From: Rusty Russell @ 2021-01-08 0:40 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion Andrew Chow <achow101-lists@achow101•com> writes: > Hi Rusty, > > On 1/6/21 6:26 PM, Rusty Russell wrote: >> Hi Andrew et al, >> >> Very excited to see this progress; thanks for doing all the >> work! Sorry for the delayed feedback, I didn't get to this before the >> break. >> >>> Additionally, I would like to add a new global field: >>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 >>> * Key: empty >>> * Value: A single byte as a boolean. 0 for False, 1 for True. All >>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted >>> in PSBTv2. >>> >>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and >>> outputs can be added to the PSBT. This flag may be set to True when >>> inputs and outputs are being updated, signed, and finalized. However >>> care must be taken when there are existing signatures. If this field is >>> omitted or set to False, no further inputs and outputs may be added to >>> the PSBT. >> I wonder if this can be flagged simply by omitting the (AFAICT >> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT? What >> are the purposes of those fields? > The purpose of those fields is to know how many input and output maps > there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine > whether a map is an input map or an output map. So the counts are there > to allow that. Ah, yeah, you need at least the number of input maps :( It's generally preferable to have sections be self-describing; internally if you have a function which takes all the input maps you should be able to trivially tell if you're handed the output maps by mistake. Similarly, it would have been nice to have an input map be a distinctly marked type from global or output maps. Nonetheless, that's a bigger change. You could just require a double-00 terminator between the global, input and output sections though. >> For our uses, there would be no signatures at this stage; it's simply a >> subdivision of the Creator role. This role would be terminated by >> removing the under-construction marker. For this, it could be clear >> that such an under-construction PSBT SHOULD NOT be signed. > > There are some protocols where signed inputs are added to transactions. Sure, but you can't solve every problem. We've now created the possibility that a PSBT is "under construction" but can't be modified, *and* a very invasive requirement to determine that. I disagree with Andrew's goal here: > 1. PSBT provides no way to modify the set of inputs or outputs after the > Creator role is done. It's simpler if, "the under-construction PSBT can be used within the Creator role, which can now have sub-roles". If you really want to allow this (and I think we need to explore concrete examples to justify this complexity!), better to add data to PSBT_GLOBAL_UNDER_CONSTRUCTION: 1. a flag to indicate whether inputs are modifiable. 2. a flag to indicate whether outputs are modifiable. 3. a bitmap of what inputs are SIGHASH_SINGLE. If you add a signature which is not SIGHASH_NONE, you clear the "outputs modifiable" flag. If you add a signature which is not SIGHASH_ANYONECANPAY, you clear the "inputs modifiable" flag. If you clear both flags, you remove the PSBT_GLOBAL_UNDER_CONSTRUCTION altogether. You similarly set the bitmap depending on whether all sigs are SIGHASH_SINGLE. Cheers, Rusty. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2021-01-08 0:40 ` Rusty Russell @ 2021-01-14 17:07 ` Andrew Chow 0 siblings, 0 replies; 17+ messages in thread From: Andrew Chow @ 2021-01-14 17:07 UTC (permalink / raw) To: Rusty Russell, Bitcoin Protocol Discussion On 1/7/21 7:40 PM, Rusty Russell wrote: > Andrew Chow <achow101-lists@achow101•com> writes: >> Hi Rusty, >> >> On 1/6/21 6:26 PM, Rusty Russell wrote: >>> Hi Andrew et al, >>> >>> Very excited to see this progress; thanks for doing all the >>> work! Sorry for the delayed feedback, I didn't get to this before the >>> break. >>> >>>> Additionally, I would like to add a new global field: >>>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05 >>>> * Key: empty >>>> * Value: A single byte as a boolean. 0 for False, 1 for True. All >>>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted >>>> in PSBTv2. >>>> >>>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and >>>> outputs can be added to the PSBT. This flag may be set to True when >>>> inputs and outputs are being updated, signed, and finalized. However >>>> care must be taken when there are existing signatures. If this field is >>>> omitted or set to False, no further inputs and outputs may be added to >>>> the PSBT. >>> I wonder if this can be flagged simply by omitting the (AFAICT >>> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT? What >>> are the purposes of those fields? >> The purpose of those fields is to know how many input and output maps >> there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine >> whether a map is an input map or an output map. So the counts are there >> to allow that. > Ah, yeah, you need at least the number of input maps :( > > It's generally preferable to have sections be self-describing; > internally if you have a function which takes all the input maps you > should be able to trivially tell if you're handed the output maps by > mistake. Similarly, it would have been nice to have an input map be a > distinctly marked type from global or output maps. > > Nonetheless, that's a bigger change. You could just require a double-00 > terminator between the global, input and output sections though. Changing that is a bigger change and I'd rather keep this as minimal as possible. Having only new fields is simpler. >>> For our uses, there would be no signatures at this stage; it's simply a >>> subdivision of the Creator role. This role would be terminated by >>> removing the under-construction marker. For this, it could be clear >>> that such an under-construction PSBT SHOULD NOT be signed. >> There are some protocols where signed inputs are added to transactions. > Sure, but you can't solve every problem. We've now created the > possibility that a PSBT is "under construction" but can't be modified, > *and* a very invasive requirement to determine that. > > I disagree with Andrew's goal here: > >> 1. PSBT provides no way to modify the set of inputs or outputs after the >> Creator role is done. > It's simpler if, "the under-construction PSBT can be used within the > Creator role, which can now have sub-roles". > > If you really want to allow this (and I think we need to explore > concrete examples to justify this complexity!), better to add data to > PSBT_GLOBAL_UNDER_CONSTRUCTION: > 1. a flag to indicate whether inputs are modifiable. > 2. a flag to indicate whether outputs are modifiable. > 3. a bitmap of what inputs are SIGHASH_SINGLE. > > If you add a signature which is not SIGHASH_NONE, you clear the "outputs > modifiable" flag. If you add a signature which is not > SIGHASH_ANYONECANPAY, you clear the "inputs modifiable" flag. If you > clear both flags, you remove the PSBT_GLOBAL_UNDER_CONSTRUCTION > altogether. You similarly set the bitmap depending on whether all sigs > are SIGHASH_SINGLE. I think we do need to support adding signed inputs and adding inputs to signed transactions. Someone had asked for this feature before. Additionally Nicolas Dorier informed me that his NTumbleBit project does those things. In that case, I will include your suggestions for PSBT_GLOBAL_UNDER_CONSTRUCTION in the BIP. Andrew Chow > > Cheers, > Rusty. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2020-12-09 22:25 [bitcoin-dev] New PSBT version proposal Andrew Chow ` (2 preceding siblings ...) 2020-12-22 20:12 ` Andrew Chow @ 2021-03-10 0:20 ` Lloyd Fournier 2021-04-05 0:35 ` Lloyd Fournier 3 siblings, 1 reply; 17+ messages in thread From: Lloyd Fournier @ 2021-03-10 0:20 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 5238 bytes --] Hi Andrew & all, I've been working with PSBTs for a little while now. FWIW I agree with the change of removing the global tx and having the input/output data stored together in the new unified structures. One thing I've been wondering about is how output descriptors could fit into PSBTs. They are useful since they allow you to determine the maximum satisfaction weight for inputs so you can properly align fees as things get added. I haven't seen any discussion about including them in this revision. Is it simply a matter of time before they make it into a subsequent PSBT spec or is there something I'm missing conceptually? Cheers, LL On Thu, 10 Dec 2020 at 09:33, Andrew Chow via bitcoin-dev < bitcoin-dev@lists•linuxfoundation.org> wrote: > Hi All, > > I would like to propose a new PSBT version that addresses a few > deficiencies in the current PSBT v0. As this will be backwards > incompatible, a new PSBT version will be used, v1. > > The primary change is to truly have all input and output data for each > in their respective maps. Instead of having to parse an unsigned > transaction and lookup some data from there, and other data from the > correct map, all of the data for an input will be contained in its map. > Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. > Thus I propose that the following fields be added: > > Global: > * PSBT_GLOBAL_TX_VERSION = 0x02 > * Key: empty > * Value: 32-bit little endian unsigned integer for the transaction > version number. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03 > * Key: empty > * Value: 32 bit little endian unsigned integer for the preferred > transaction lock time. Must be omitted in PSBT v0. May be provided in > PSBT v1, assumed to be 0 if not provided. > * PSBT_GLOBAL_INPUT_COUNT = 0x04 > * Key: empty > * Value: Compact size unsigned integer. Number of inputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > * PSBT_GLOBAL_OUTPUT_COUNT = 0x05 > * Key: empty > * Value: Compact size unsigned integer. Number of outputs in this > PSBT. Must be provided in PSBT v1 and omitted in v0. > > Input: > * PSBT_IN_PREVIOUS_TXID = 0x0e > * Key: empty > * Value: 32 byte txid of the previous transaction whose output at > PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and > omitted in v0. > * PSBT_IN_OUTPUT_INDEX = 0x0f > * Key: empty > * Value: 32 bit little endian integer for the index of the output > being spent. Must be provided in PSBT v1 and omitted in v0. > * PSBT_IN_SEQUENCE = 0x0f > * Key: empty > * Value: 32 bit unsigned little endian integer for the sequence > number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed > to be max sequence (0xffffffff) if not provided. > * PSBT_IN_REQUIRED_LOCKTIME = 0x10 > * Key: empty > * Value: 32 bit unsigned little endian integer for the lock time that > this input requires. Must be omitted in PSBT v0. May be provided in PSBT > v1, assumed to be 0 if not provided. > > Output: > * PSBT_OUT_VALUE = 0x03 > * Key: empty > * Value: 64-bit unsigned little endian integer for the output's > amount in satoshis. Must be provided in PSBT v1 and omitted in v0. > * PSBT_OUT_OUTPUT_SCRIPT = 0x04 > * Key: empty > * Value: The script for this output. Otherwise known as the > scriptPubKey. Must be provided in PSBT v1 and omitted in v0. > > This change allows for PSBT to be used in the construction of > transactions. With these new fields, inputs and outputs can be added as > needed. One caveat is that there is no longer a unique transaction > identifier so more care must be taken when combining PSBTs. > Additionally, adding new inputs and outputs must be done such that > signatures are not invalidated. This may be harder to specify. > > An important thing to note in this proposal are the fields > PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin > transaction only has a single locktime yet a PSBT may have multiple > locktimes. To choose the locktime for the transaction, finalizers must > choose the maximum of all of the *_LOCKTIME fields. > PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those > involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to > be set. This field allows finalizers to choose a locktime that is high > enough for all inputs without needing to understand the scripts > involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if > no inputs require a particular locktime. > > As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 > needs the version number bump to enforce backwards incompatibility. > However once the inputs and outputs of a PSBT are decided, a PSBT could > be "downgraded" back to v0 by creating the unsigned transaction from the > above fields, and then dropping these new fields. > > If the list finds that these changes are reasonable, I will write a PR > to modify BIP 174 to incorporate them. > > Thanks, > Andrew Chow > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 6248 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [bitcoin-dev] New PSBT version proposal 2021-03-10 0:20 ` Lloyd Fournier @ 2021-04-05 0:35 ` Lloyd Fournier 0 siblings, 0 replies; 17+ messages in thread From: Lloyd Fournier @ 2021-04-05 0:35 UTC (permalink / raw) To: Andrew Chow, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 1210 bytes --] On Wed, 10 Mar 2021 at 11:20, Lloyd Fournier <lloyd.fourn@gmail•com> wrote: > Hi Andrew & all, > > I've been working with PSBTs for a little while now. FWIW I agree with the > change of removing the global tx and having the input/output data stored > together in the new unified structures. > > One thing I've been wondering about is how output descriptors could fit > into PSBTs. They are useful since they allow you to determine the maximum > satisfaction weight for inputs so you can properly align fees as things get > added. I haven't seen any discussion about including them in this revision. > Is it simply a matter of time before they make it into a subsequent PSBT > spec or is there something I'm missing conceptually? > Sipa replied to me off list some time ago and explained what I was missing. PSBTs have all the information you could want from a descriptor already. For example the maximum satisfaction weight can be determined from the witness/redeem script (I had forgot these fields existed). Therefore descriptors are more useful in higher level applications while PSBTs are useful for communicating with signing devices. Therefore there is no reason for PSBTs to support descriptors. LL [-- Attachment #2: Type: text/html, Size: 1684 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2021-04-05 0:35 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-09 22:25 [bitcoin-dev] New PSBT version proposal Andrew Chow 2020-12-10 11:28 ` Sanket Kanjalkar 2020-12-16 17:44 ` Andrew Poelstra 2020-12-22 20:12 ` Andrew Chow 2020-12-23 3:30 ` fiatjaf 2020-12-23 15:22 ` Andrew Poelstra 2020-12-23 21:30 ` Andrew Chow 2021-01-02 6:34 ` Jeremy 2020-12-23 21:32 ` Andrew Chow 2021-01-15 17:28 ` Andrew Chow 2021-01-21 19:50 ` Andrew Chow 2021-01-06 23:26 ` Rusty Russell 2021-01-06 23:48 ` Andrew Chow 2021-01-08 0:40 ` Rusty Russell 2021-01-14 17:07 ` Andrew Chow 2021-03-10 0:20 ` Lloyd Fournier 2021-04-05 0:35 ` Lloyd Fournier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox