[bitcoin-dev] Two questions about segwit implementation

[bitcoin-dev] Two questions about segwit implementation

[Aymeric Vitte]

I realized recently that my segwit implementation was not correct,
basically some time ago, wrongly reading the specs (and misleaded by
what follows), I thought that scriptsig would go into witness data as it
was, but that's not the case, op_pushdata is replaced by varlen

Now reading correctly the specs, they seem to be not totally correct,
then the first question is: why OP_0 is 00 in witness data and not 0100?
Does this apply to other op_codes? This does not look logical at all

The second question is: why for non segwit inputs there is a 00 length
in segwit data, what is the rational for that? It should just be nothing
since you don't need this to reconciliate things

Re: [bitcoin-dev] Two questions about segwit implementation

[Johnson Lau]

> On 26 May 2019, at 7:56 AM, Aymeric Vitte via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> I realized recently that my segwit implementation was not correct,
> basically some time ago, wrongly reading the specs (and misleaded by
> what follows), I thought that scriptsig would go into witness data as it
> was, but that's not the case, op_pushdata is replaced by varlen
> 

Witness is not script. There is no op_pushdata or any other opcodes.

Witness is a stack. For each input, the witness starts with a CCompactSize for the number of stack elements for this input. Each stack element in turns starts with a CCompactSize for the size of this element, followed by the actual data


> Now reading correctly the specs, they seem to be not totally correct,
> then the first question is: why OP_0 is 00 in witness data and not 0100?
> Does this apply to other op_codes? This does not look logical at all
> 

A “00” element means the size of this element is zero. Since it’s zero size, no data is followed. This will create an empty element on the stack. It’s effectively same as OP_0 (Again, witness is not script)

A “0100” element means the element size is one, and the data for this element is “00”. So it will leave an 1-byte element on the stack.


> The second question is: why for non segwit inputs there is a 00 length
> in segwit data, what is the rational for that? It should just be nothing
> since you don't need this to reconciliate things

The “00” here means "this input has no witness stack element”. You need this even for non segwit inputs, because there is no way to tell whether an input is segwit-enabled or not, until you look up the UTXO, which might not be always available. Transaction serialization couldn’t rely on contextual information.

However, if all inputs have no stack element, the spec requires you to always use the non-segwit serialization.

> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Two questions about segwit implementation

[Aymeric Vitte]

OK, thanks, understood for OP_0 but still for the 00 number of witness
data for non segwit inputs the one that is doing the transaction knows
which inputs are segwit or not, then parsing the transaction you can
associate the correct input to the correct witness data, without the
need of 00, so I must be missing the use case

Le 26/05/2019 à 16:33, Johnson Lau a écrit :
>> On 26 May 2019, at 7:56 AM, Aymeric Vitte via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> I realized recently that my segwit implementation was not correct,
>> basically some time ago, wrongly reading the specs (and misleaded by
>> what follows), I thought that scriptsig would go into witness data as it
>> was, but that's not the case, op_pushdata is replaced by varlen
>>
> Witness is not script. There is no op_pushdata or any other opcodes.
>
> Witness is a stack. For each input, the witness starts with a CCompactSize for the number of stack elements for this input. Each stack element in turns starts with a CCompactSize for the size of this element, followed by the actual data
>
>
>> Now reading correctly the specs, they seem to be not totally correct,
>> then the first question is: why OP_0 is 00 in witness data and not 0100?
>> Does this apply to other op_codes? This does not look logical at all
>>
> A “00” element means the size of this element is zero. Since it’s zero size, no data is followed. This will create an empty element on the stack. It’s effectively same as OP_0 (Again, witness is not script)
>
> A “0100” element means the element size is one, and the data for this element is “00”. So it will leave an 1-byte element on the stack.
>
>
>> The second question is: why for non segwit inputs there is a 00 length
>> in segwit data, what is the rational for that? It should just be nothing
>> since you don't need this to reconciliate things
> The “00” here means "this input has no witness stack element”. You need this even for non segwit inputs, because there is no way to tell whether an input is segwit-enabled or not, until you look up the UTXO, which might not be always available. Transaction serialization couldn’t rely on contextual information.
>
> However, if all inputs have no stack element, the spec requires you to always use the non-segwit serialization.
>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-- 
Move your coins by yourself (browser version): https://peersm.com/wallet
Bitcoin transactions made simple: https://github.com/Ayms/bitcoin-transactions
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms

Re: [bitcoin-dev] Two questions about segwit implementation

[Johnson Lau]

This is not how it works. While the transaction creator may know which inputs are segwit, the validators have no way to tell until they look up the UTXO set.

In a transaction, all information about an input the validators have is the 36-byte outpoint (txid + index). Just by looking at the outpoint, there is no way to tell whether it is segwit-enabled or not. So there needs to be a way to tell the validator that “the witness for this input is empty”, and it is the “00”.

> On 27 May 2019, at 12:18 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
> 
> ……. for the 00 number of witness
> data for non segwit inputs the one that is doing the transaction knows
> which inputs are segwit or not, then parsing the transaction you can
> associate the correct input to the correct witness data, without the
> need of 00, so I must be missing the use case

Re: [bitcoin-dev] Two questions about segwit implementation

[Aymeric Vitte]

I did not phrase correctly in fact, what I meant is: if the validator
sees empty or witness script in scriptSig, then this is a segwit input,
and doing this one by one the validator can associate the correct segwit
data to the correct segwit input, so 00 does not look to be needed

Le 26/05/2019 à 18:28, Johnson Lau a écrit :
> This is not how it works. While the transaction creator may know which inputs are segwit, the validators have no way to tell until they look up the UTXO set.
>
> In a transaction, all information about an input the validators have is the 36-byte outpoint (txid + index). Just by looking at the outpoint, there is no way to tell whether it is segwit-enabled or not. So there needs to be a way to tell the validator that “the witness for this input is empty”, and it is the “00”.
>
>> On 27 May 2019, at 12:18 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
>>
>> ……. for the 00 number of witness
>> data for non segwit inputs the one that is doing the transaction knows
>> which inputs are segwit or not, then parsing the transaction you can
>> associate the correct input to the correct witness data, without the
>> need of 00, so I must be missing the use case
>

Re: [bitcoin-dev] Two questions about segwit implementation

[Johnson Lau]

Empty scriptSig doesn’t imply segwit input: if the previous scriptPubKey is OP_1 (which does not allow witness), it could still be spent with an empty scriptSig

Similarly, a scriptSig looking like a spend of P2SH-segwit doesn’t imply segwit input: if the previous scriptPubKey is empty, it could be spent with a push of any non-zero value.

> On 27 May 2019, at 1:09 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
> 
> I did not phrase correctly in fact, what I meant is: if the validator
> sees empty or witness script in scriptSig, then this is a segwit input,
> and doing this one by one the validator can associate the correct segwit
> data to the correct segwit input, so 00 does not look to be needed
> 
> Le 26/05/2019 à 18:28, Johnson Lau a écrit :
>> This is not how it works. While the transaction creator may know which inputs are segwit, the validators have no way to tell until they look up the UTXO set.
>> 
>> In a transaction, all information about an input the validators have is the 36-byte outpoint (txid + index). Just by looking at the outpoint, there is no way to tell whether it is segwit-enabled or not. So there needs to be a way to tell the validator that “the witness for this input is empty”, and it is the “00”.
>> 
>>> On 27 May 2019, at 12:18 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
>>> 
>>> ……. for the 00 number of witness
>>> data for non segwit inputs the one that is doing the transaction knows
>>> which inputs are segwit or not, then parsing the transaction you can
>>> associate the correct input to the correct witness data, without the
>>> need of 00, so I must be missing the use case
>> 

Re: [bitcoin-dev] Two questions about segwit implementation

[Aymeric Vitte]

Well, OK, then back to non standard stuff and bitcoin considers that an
OP_1 or empty scriptpubkey is something that can exist, sipa does not
like my questions on this list but this is a bit frightening in fact to
see that after 10 years an OP_1 scriptpubkey or empty one can be a use
case, thanks Thomas also, where all of this is clearly documented so
people don't bother the list not to produce approximative stuff remains
mysterious

Le 26/05/2019 à 19:24, Johnson Lau a écrit :
> Empty scriptSig doesn’t imply segwit input: if the previous scriptPubKey is OP_1 (which does not allow witness), it could still be spent with an empty scriptSig
>
> Similarly, a scriptSig looking like a spend of P2SH-segwit doesn’t imply segwit input: if the previous scriptPubKey is empty, it could be spent with a push of any non-zero value.
>
>> On 27 May 2019, at 1:09 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
>>
>> I did not phrase correctly in fact, what I meant is: if the validator
>> sees empty or witness script in scriptSig, then this is a segwit input,
>> and doing this one by one the validator can associate the correct segwit
>> data to the correct segwit input, so 00 does not look to be needed
>>
>> Le 26/05/2019 à 18:28, Johnson Lau a écrit :
>>> This is not how it works. While the transaction creator may know which inputs are segwit, the validators have no way to tell until they look up the UTXO set.
>>>
>>> In a transaction, all information about an input the validators have is the 36-byte outpoint (txid + index). Just by looking at the outpoint, there is no way to tell whether it is segwit-enabled or not. So there needs to be a way to tell the validator that “the witness for this input is empty”, and it is the “00”.
>>>
>>>> On 27 May 2019, at 12:18 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
>>>>
>>>> ……. for the 00 number of witness
>>>> data for non segwit inputs the one that is doing the transaction knows
>>>> which inputs are segwit or not, then parsing the transaction you can
>>>> associate the correct input to the correct witness data, without the
>>>> need of 00, so I must be missing the use case
>

Re: [bitcoin-dev] Two questions about segwit implementation

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

On Sun, May 26, 2019, 07:07 Aymeric Vitte via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> I realized recently that my segwit implementation was not correct,
> basically some time ago, wrongly reading the specs (and misleaded by
> what follows), I thought that scriptsig would go into witness data as it
> was, but that's not the case, op_pushdata is replaced by varlen
>
> Now reading correctly the specs, they seem to be not totally correct,
> then the first question is: why OP_0 is 00 in witness data and not 0100?
> Does this apply to other op_codes? This does not look logical at all
>
> The second question is: why for non segwit inputs there is a 00 length
> in segwit data, what is the rational for that? It should just be nothing
> since you don't need this to reconciliate things
>

This is a question that belongs on https://bitcoin.stackexchange.com, not
this list.

Cheers,

-- 
Pieter

[-- Attachment #2: Type: text/html, Size: 1651 bytes --]

Re: [bitcoin-dev] Two questions about segwit implementation

[Thomas Kerin]

The scriptSig when evaluated populates the stack so opcodes can operate
on them. A witness is essentially a list of data elements, quite similar
to the script stack (the witness is passed in as the script stack in fact)

OP_0 when evaluated pushes a _zero length_ value onto the stack, hence
the 00 (the varlen) in the witness serialization. OP_1 (51 in decimal)
pushes 0x01 to the stack, so when serialized would be 0101.

It may help to consider the entire witness structure as
vector<vector<data element>> and it's length must equal the number of
inputs - so a non-segwit input must have a zero sized witness.

On 5/26/19 12:56 AM, Aymeric Vitte via bitcoin-dev wrote:
> I realized recently that my segwit implementation was not correct,
> basically some time ago, wrongly reading the specs (and misleaded by
> what follows), I thought that scriptsig would go into witness data as it
> was, but that's not the case, op_pushdata is replaced by varlen
>
> Now reading correctly the specs, they seem to be not totally correct,
> then the first question is: why OP_0 is 00 in witness data and not 0100?
> Does this apply to other op_codes? This does not look logical at all
>
> The second question is: why for non segwit inputs there is a 00 length
> in segwit data, what is the rational for that? It should just be nothing
> since you don't need this to reconciliate things
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Two questions about segwit implementation

[Kostas Karasavvas]

[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]

On Sun, May 26, 2019 at 5:07 PM Aymeric Vitte via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> I realized recently that my segwit implementation was not correct,
> basically some time ago, wrongly reading the specs (and misleaded by
> what follows), I thought that scriptsig would go into witness data as it
> was, but that's not the case, op_pushdata is replaced by varlen
>
> Now reading correctly the specs, they seem to be not totally correct,
> then the first question is: why OP_0 is 00 in witness data and not 0100?
> Does this apply to other op_codes? This does not look logical at all
>
>
op_pushdata allows for unsigned integers and thus it uses the extra byte to
specify that.

A varint (varlen) is unsigned.  0 and anything <= 252 is one byte length,
so 00.


> The second question is: why for non segwit inputs there is a 00 length
> in segwit data, what is the rational for that? It should just be nothing
> since you don't need this to reconciliate things
>
>
Because you don't have the number of witnesses as you have for
inputs/outputs. The witness data length is assumed to be the same as the
inputs length. And then for non-segwit inputs you need the 00 to specify
that it is empty.

Hope that helps.

[-- Attachment #2: Type: text/html, Size: 1963 bytes --]