[bitcoin-dev] Test cases for Taproot signature message

[bitcoin-dev] Test cases for Taproot signature message

[Giacomo Caironi]

[-- Attachment #1: Type: text/plain, Size: 1477 bytes --]

Hi,
recently I have worked on a python implementation of bitcoin signature
messages, and I have found that there was way better documentation about
Segwit signature message than Taproot.

1) Segwit signature message got its own BIP, completed with test cases
regarding only that specific function; Taproot on the other hand has the
signature message function defined in BIP 341 and the test vectors in a
different BIP (341). This is confusing. Shouldn't we create a different BIP
only for Taproot signature message exactly like Segwit?

2) The test vectors for Taproot have no documentation and, most
importantly, they are not atomic, in the sense that they do not target a
specific part of the taproot code but all of it. This may not be a very big
problem, but for signature verification it is. Because there are hashes
involved, we can't really debug why a signature message doesn't pass
validation, either it is valid or it is not. BIP 143 in this case is really
good, because it provides hash preimages, so it is possible to debug the
function and see where something went wrong. Because of this, writing the
Segwit signature hash function took a fraction of the time compared to
Taproot.

If this idea is accepted I will be more than happy to write the test cases
for Taproot.

BTW this is the first time I contribute to Bitcoin, let me know if I was
rude or did something wrong. Moreover english is not my first language, so
I apologize if I wrote something awful above

[-- Attachment #2: Type: text/html, Size: 1619 bytes --]

Re: [bitcoin-dev] Test cases for Taproot signature message

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]

On Thursday, September 16th, 2021 at 5:36 PM, Giacomo Caironi via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi,
> recently I have worked on a python implementation of bitcoin signature messages, and I have found that there was way better documentation about Segwit signature message than Taproot.
>
> 1) Segwit signature message got its own BIP, completed with test cases regarding only that specific function; Taproot on the other hand has the signature message function defined in BIP 341 and the test vectors in a different BIP (341). This is confusing. Shouldn't we create a different BIP only for Taproot signature message exactly like Segwit?

I'm not entirely sure what you mean; you're saying BIP 341 twice.

Still, you're right overall - there is no separate BIP for the signature message function. The reason is that the message function is different for BIP341 and BIP342. BIP 341 defines a basic common message function, which is then built up for BIP 341 key path spending, and for BIP 342 tapscript spending. This common part could have been a separate BIP, but that'd still not be a very clean separation. I'm not very inclined to support changing that at this point, given the state of deployment the BIPs have, but that doesn't mean the documentation/vectors can't be improved in the existing documents.

> 2) The test vectors for Taproot have no documentation and, most importantly, they are not atomic, in the sense that they do not target a specific part of the taproot code but all of it. This may not be a very big problem, but for signature verification it is. Because there are hashes involved, we can't really debug why a signature message doesn't pass validation, either it is valid or it is not. BIP 143 in this case is really good, because it provides hash preimages, so it is possible to debug the function and see where something went wrong. Because of this, writing the Segwit signature hash function took a fraction of the time compared to Taproot.

You're right. The existing tests are really intended for verifying an implementation against (and for making sure future code changes don't break anything). They have much higher coverage than the segwit tests had. But they aren't useful as documentation; the code that generates them (https://github.com/bitcoin/bitcoin/blob/v22.0/test/functional/feature_taproot.py#L605L1122) is probably better at that even, but still pretty dense.

> If this idea is accepted I will be more than happy to write the test cases for Taproot.

If you're interested in writing test vectors that are more aimed at helping debugging issues, by all means, do. You've already brought up the sighash code as an example. Another idea, primarily aimed at developers of signing code, is test vectors for certain P2TR scriptPubKeys, derived from certain internal keys and script trees. I'm happy to help to integrate such in Bitcoin Core and the BIP(s).

Thanks!

Cheers,

--
Pieter

[-- Attachment #2: Type: text/html, Size: 3675 bytes --]

Re: [bitcoin-dev] Test cases for Taproot signature message

[Riccardo Casatta]

[-- Attachment #1: Type: text/plain, Size: 2350 bytes --]

Hi Giacomo,

I wrote the rust implementation of bitcoin signature messages and to
double-check I created some test vectors you can see at
https://github.com/rust-bitcoin/rust-bitcoin/blob/b7f984972ad6cb4942827c2b7c401f590588cdcf/src/util/sighash.rs#L689-L799.
These vectors have been created printing intermediate results from
https://github.com/bitcoin/bitcoin/blob/6401de0133e32a641ed9e78a85b3aa337c75d190/test/functional/feature_taproot.py

Il giorno gio 16 set 2021 alle ore 23:40 Giacomo Caironi via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> ha scritto:

> Hi,
> recently I have worked on a python implementation of bitcoin signature
> messages, and I have found that there was way better documentation about
> Segwit signature message than Taproot.
>
> 1) Segwit signature message got its own BIP, completed with test cases
> regarding only that specific function; Taproot on the other hand has the
> signature message function defined in BIP 341 and the test vectors in a
> different BIP (341). This is confusing. Shouldn't we create a different BIP
> only for Taproot signature message exactly like Segwit?
>
> 2) The test vectors for Taproot have no documentation and, most
> importantly, they are not atomic, in the sense that they do not target a
> specific part of the taproot code but all of it. This may not be a very big
> problem, but for signature verification it is. Because there are hashes
> involved, we can't really debug why a signature message doesn't pass
> validation, either it is valid or it is not. BIP 143 in this case is really
> good, because it provides hash preimages, so it is possible to debug the
> function and see where something went wrong. Because of this, writing the
> Segwit signature hash function took a fraction of the time compared to
> Taproot.
>
> If this idea is accepted I will be more than happy to write the test cases
> for Taproot.
>
> BTW this is the first time I contribute to Bitcoin, let me know if I was
> rude or did something wrong. Moreover english is not my first language, so
> I apologize if I wrote something awful above
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>


-- 
Riccardo Casatta - @RCasatta <https://twitter.com/RCasatta>

[-- Attachment #2: Type: text/html, Size: 3495 bytes --]

Re: [bitcoin-dev] Test cases for Taproot signature message

[Giacomo Caironi]

[-- Attachment #1: Type: text/plain, Size: 3631 bytes --]

Ok I have created three test cases, you can find them here
<https://gist.github.com/giacomocaironi/e41a45195b2ac6863ec46e8f86324757>.
They cover most of the SigMsg function but they don't cover the ext_flag,
so they are only for taproot key path; but if you want to test for script
paths you have to implement more than this function so you would use the
official test vector.
Could someone please take a look at them? I think that they are right but I
am not too sure

Il giorno ven 17 set 2021 alle ore 00:30 Pieter Wuille <
bitcoin-dev@wuille.net> ha scritto:

> On Thursday, September 16th, 2021 at 5:36 PM, Giacomo Caironi via
> bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Hi,
> recently I have worked on a python implementation of bitcoin signature
> messages, and I have found that there was way better documentation about
> Segwit signature message than Taproot.
>
> 1) Segwit signature message got its own BIP, completed with test cases
> regarding only that specific function; Taproot on the other hand has the
> signature message function defined in BIP 341 and the test vectors in a
> different BIP (341). This is confusing. Shouldn't we create a different BIP
> only for Taproot signature message exactly like Segwit?
>
>
> I'm not entirely sure what you mean; you're saying BIP 341 twice.
>
> Still, you're right overall - there is no separate BIP for the signature
> message function. The reason is that the message function is different for
> BIP341 and BIP342. BIP 341 defines a basic common message function, which
> is then built up for BIP 341 key path spending, and for BIP 342 tapscript
> spending. This common part could have been a separate BIP, but that'd still
> not be a very clean separation. I'm not very inclined to support changing
> that at this point, given the state of deployment the BIPs have, but that
> doesn't mean the documentation/vectors can't be improved in the existing
> documents.
>
> 2) The test vectors for Taproot have no documentation and, most
> importantly, they are not atomic, in the sense that they do not target a
> specific part of the taproot code but all of it. This may not be a very big
> problem, but for signature verification it is. Because there are hashes
> involved, we can't really debug why a signature message doesn't pass
> validation, either it is valid or it is not. BIP 143 in this case is really
> good, because it provides hash preimages, so it is possible to debug the
> function and see where something went wrong. Because of this, writing the
> Segwit signature hash function took a fraction of the time compared to
> Taproot.
>
>
> You're right. The existing tests are really intended for verifying an
> implementation against (and for making sure future code changes don't break
> anything). They have much higher coverage than the segwit tests had. But
> they aren't useful as documentation; the code that generates them (
> https://github.com/bitcoin/bitcoin/blob/v22.0/test/functional/feature_taproot.py#L605L1122)
> is probably better at that even, but still pretty dense.
>
> If this idea is accepted I will be more than happy to write the test cases
> for Taproot.
>
>
> If you're interested in writing test vectors that are more aimed at
> helping debugging issues, by all means, do. You've already brought up the
> sighash code as an example. Another idea, primarily aimed at developers of
> signing code, is test vectors for certain P2TR scriptPubKeys, derived from
> certain internal keys and script trees. I'm happy to help to integrate such
> in Bitcoin Core and the BIP(s).
>
> Thanks!
>
> Cheers,
>
> --
> Pieter
>
>

[-- Attachment #2: Type: text/html, Size: 4639 bytes --]

Re: [bitcoin-dev] Test cases for Taproot signature message

[Giacomo Caironi]

[-- Attachment #1: Type: text/plain, Size: 3897 bytes --]

The related pull request is now open
https://github.com/bitcoin/bips/pull/1191

Il giorno sab 18 set 2021 alle ore 13:32 Giacomo Caironi <
giacomo.caironi@gmail.com> ha scritto:

> Ok I have created three test cases, you can find them here
> <https://gist.github.com/giacomocaironi/e41a45195b2ac6863ec46e8f86324757>.
> They cover most of the SigMsg function but they don't cover the ext_flag,
> so they are only for taproot key path; but if you want to test for script
> paths you have to implement more than this function so you would use the
> official test vector.
> Could someone please take a look at them? I think that they are right but
> I am not too sure
>
> Il giorno ven 17 set 2021 alle ore 00:30 Pieter Wuille <
> bitcoin-dev@wuille.net> ha scritto:
>
>> On Thursday, September 16th, 2021 at 5:36 PM, Giacomo Caironi via
>> bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> Hi,
>> recently I have worked on a python implementation of bitcoin signature
>> messages, and I have found that there was way better documentation about
>> Segwit signature message than Taproot.
>>
>> 1) Segwit signature message got its own BIP, completed with test cases
>> regarding only that specific function; Taproot on the other hand has the
>> signature message function defined in BIP 341 and the test vectors in a
>> different BIP (341). This is confusing. Shouldn't we create a different BIP
>> only for Taproot signature message exactly like Segwit?
>>
>>
>> I'm not entirely sure what you mean; you're saying BIP 341 twice.
>>
>> Still, you're right overall - there is no separate BIP for the signature
>> message function. The reason is that the message function is different for
>> BIP341 and BIP342. BIP 341 defines a basic common message function, which
>> is then built up for BIP 341 key path spending, and for BIP 342 tapscript
>> spending. This common part could have been a separate BIP, but that'd still
>> not be a very clean separation. I'm not very inclined to support changing
>> that at this point, given the state of deployment the BIPs have, but that
>> doesn't mean the documentation/vectors can't be improved in the existing
>> documents.
>>
>> 2) The test vectors for Taproot have no documentation and, most
>> importantly, they are not atomic, in the sense that they do not target a
>> specific part of the taproot code but all of it. This may not be a very big
>> problem, but for signature verification it is. Because there are hashes
>> involved, we can't really debug why a signature message doesn't pass
>> validation, either it is valid or it is not. BIP 143 in this case is really
>> good, because it provides hash preimages, so it is possible to debug the
>> function and see where something went wrong. Because of this, writing the
>> Segwit signature hash function took a fraction of the time compared to
>> Taproot.
>>
>>
>> You're right. The existing tests are really intended for verifying an
>> implementation against (and for making sure future code changes don't break
>> anything). They have much higher coverage than the segwit tests had. But
>> they aren't useful as documentation; the code that generates them (
>> https://github.com/bitcoin/bitcoin/blob/v22.0/test/functional/feature_taproot.py#L605L1122)
>> is probably better at that even, but still pretty dense.
>>
>> If this idea is accepted I will be more than happy to write the test
>> cases for Taproot.
>>
>>
>> If you're interested in writing test vectors that are more aimed at
>> helping debugging issues, by all means, do. You've already brought up the
>> sighash code as an example. Another idea, primarily aimed at developers of
>> signing code, is test vectors for certain P2TR scriptPubKeys, derived from
>> certain internal keys and script trees. I'm happy to help to integrate such
>> in Bitcoin Core and the BIP(s).
>>
>> Thanks!
>>
>> Cheers,
>>
>> --
>> Pieter
>>
>>

[-- Attachment #2: Type: text/html, Size: 5175 bytes --]