[Bitcoin-development] 75%/95% threshold for transaction versions

[Bitcoin-development] 75%/95% threshold for transaction versions

[s7r]

Hi,

Would it be wise to add a consensus rule like the one we have for blocks,

(if > 75% from last 1000 blocks are version 'n' mark version 'n' as
standard for blocks and if > 95% from the last 1000 blocks are version
'n' mark previous block versions as invalid)

but for transaction versions? In simple terms, if > 75% from all the
transactions in the latest 1000 blocks are version 'n', mark all
previous transaction versions as non-standard and if > 95% from all the
transactions in the latest 1000 blocks are version 'n' mark all previous
transaction versions as invalid.

At this moment, the standard in consensus is v1, but nothing is enforced
in the network related to transaction versions.

Regarding BIP62, as it can be read here [0] it is said that it requires
v2 transactions. It is also said that transaction version 2 will be
skipped and jump directly to v3, for an even version for transactions
and blocks (?). Might as well add the rule for invalidating previous
transaction versions if the majority updates - could this break anything
or affect functionality in any way?

BIP62 adds a newer transaction version which is optional and does not
mark previous v1 as non-standard or invalid. This means bitcoin core
will treat both v1 and v2/v3 transactions as standard and relay/mine
them with the same priority, regardless of the tx version?


Thanks.

[0]
https://bitcoin.stackexchange.com/questions/35904/how-much-of-bip-62-dealing-with-malleability-has-been-implemented

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 2412 bytes --]

If I had a time locked signed transaction where I threw away the key, this
would potentially invalidate my transaction.

What is the point of such a rule?

On Wed, Apr 15, 2015 at 6:43 PM, s7r <s7r@sky-ip.org> wrote:

> Hi,
>
> Would it be wise to add a consensus rule like the one we have for blocks,
>
> (if > 75% from last 1000 blocks are version 'n' mark version 'n' as
> standard for blocks and if > 95% from the last 1000 blocks are version
> 'n' mark previous block versions as invalid)
>
> but for transaction versions? In simple terms, if > 75% from all the
> transactions in the latest 1000 blocks are version 'n', mark all
> previous transaction versions as non-standard and if > 95% from all the
> transactions in the latest 1000 blocks are version 'n' mark all previous
> transaction versions as invalid.
>
> At this moment, the standard in consensus is v1, but nothing is enforced
> in the network related to transaction versions.
>
> Regarding BIP62, as it can be read here [0] it is said that it requires
> v2 transactions. It is also said that transaction version 2 will be
> skipped and jump directly to v3, for an even version for transactions
> and blocks (?). Might as well add the rule for invalidating previous
> transaction versions if the majority updates - could this break anything
> or affect functionality in any way?
>
> BIP62 adds a newer transaction version which is optional and does not
> mark previous v1 as non-standard or invalid. This means bitcoin core
> will treat both v1 and v2/v3 transactions as standard and relay/mine
> them with the same priority, regardless of the tx version?
>
>
> Thanks.
>
> [0]
>
> https://bitcoin.stackexchange.com/questions/35904/how-much-of-bip-62-dealing-with-malleability-has-been-implemented
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 3319 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

On Apr 16, 2015 1:46 AM, "s7r" <s7r@sky-ip.org> wrote:
> but for transaction versions? In simple terms, if > 75% from all the
> transactions in the latest 1000 blocks are version 'n', mark all
> previous transaction versions as non-standard and if > 95% from all the
> transactions in the latest 1000 blocks are version 'n' mark all previous
> transaction versions as invalid.

What problem are you trying to solve?

The reason why BIP62 (as specified, it is just a draft) does not make v1
transactions invalid is because it is opt-in. The creator of a transaction
needs to agree to protect it from malleability, and this subjects him to
extra rules in the creation.

Forcing v3 transactions would require every piece of wallet software to be
changed.

-- 
Pieter

[-- Attachment #2: Type: text/html, Size: 981 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[s7r]

Hi Pieter,

Thanks for your reply. I agree. Allen has a good point in the previous
email too, so the suggestion might not fix anything and complicate things.

The problem I am trying to solve is making all transactions
non-malleable by default. I guess there is a very good reason why BIP62
will not touch v1 anyway.

I am trying to build a bitcoin contract which will relay on 3 things:
- coinjoin / txes with inputs from multiple users which are signed by
all users after they are merged together (every user is sure his coins
will not be spent without the other users to spend anything, as per
agreed contract);
- pre-signed txes with nLockTime 'n' weeks. These txes will be signed
before the inputs being spent are broadcasted/confirmed, using the txid
provided by the user before broadcasting it. Malleability hurts here.
- P2SH

In simple terms, how malleable transactions really are in the network at
this moment? Who can alter a txid without invalidating the tx? Just the
parties who sign it? The miners? Anyone in the network? This is a little
bit unclear to me.

Another thing I would like to confirm, the 3 pieces of the bitcoin
protocol mentioned above will be supported in _any_ future transaction
version or block version, regardless what changes are made or features
added to bitcoin core? The contract needs to be built and left unchanged
for a very very long period of time...


On 4/16/2015 8:22 AM, Pieter Wuille wrote:
> 
> On Apr 16, 2015 1:46 AM, "s7r" <s7r@sky-ip.org <mailto:s7r@sky-ip.org>>
> wrote:
>> but for transaction versions? In simple terms, if > 75% from all the
>> transactions in the latest 1000 blocks are version 'n', mark all
>> previous transaction versions as non-standard and if > 95% from all the
>> transactions in the latest 1000 blocks are version 'n' mark all previous
>> transaction versions as invalid.
> 
> What problem are you trying to solve?
> 
> The reason why BIP62 (as specified, it is just a draft) does not make v1
> transactions invalid is because it is opt-in. The creator of a
> transaction needs to agree to protect it from malleability, and this
> subjects him to extra rules in the creation.
> 
> Forcing v3 transactions would require every piece of wallet software to
> be changed.
> 
> -- 
> Pieter
> 

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

At this moment anyone can alter the txid. Assume transactions are 100%
malleable.
On Apr 16, 2015 9:13 AM, "s7r" <s7r@sky-ip.org> wrote:

> Hi Pieter,
>
> Thanks for your reply. I agree. Allen has a good point in the previous
> email too, so the suggestion might not fix anything and complicate things.
>
> The problem I am trying to solve is making all transactions
> non-malleable by default. I guess there is a very good reason why BIP62
> will not touch v1 anyway.
>
> I am trying to build a bitcoin contract which will relay on 3 things:
> - coinjoin / txes with inputs from multiple users which are signed by
> all users after they are merged together (every user is sure his coins
> will not be spent without the other users to spend anything, as per
> agreed contract);
> - pre-signed txes with nLockTime 'n' weeks. These txes will be signed
> before the inputs being spent are broadcasted/confirmed, using the txid
> provided by the user before broadcasting it. Malleability hurts here.
> - P2SH
>
> In simple terms, how malleable transactions really are in the network at
> this moment? Who can alter a txid without invalidating the tx? Just the
> parties who sign it? The miners? Anyone in the network? This is a little
> bit unclear to me.
>
> Another thing I would like to confirm, the 3 pieces of the bitcoin
> protocol mentioned above will be supported in _any_ future transaction
> version or block version, regardless what changes are made or features
> added to bitcoin core? The contract needs to be built and left unchanged
> for a very very long period of time...
>
>
> On 4/16/2015 8:22 AM, Pieter Wuille wrote:
> >
> > On Apr 16, 2015 1:46 AM, "s7r" <s7r@sky-ip.org <mailto:s7r@sky-ip.org>>
> > wrote:
> >> but for transaction versions? In simple terms, if > 75% from all the
> >> transactions in the latest 1000 blocks are version 'n', mark all
> >> previous transaction versions as non-standard and if > 95% from all the
> >> transactions in the latest 1000 blocks are version 'n' mark all previous
> >> transaction versions as invalid.
> >
> > What problem are you trying to solve?
> >
> > The reason why BIP62 (as specified, it is just a draft) does not make v1
> > transactions invalid is because it is opt-in. The creator of a
> > transaction needs to agree to protect it from malleability, and this
> > subjects him to extra rules in the creation.
> >
> > Forcing v3 transactions would require every piece of wallet software to
> > be changed.
> >
> > --
> > Pieter
> >
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4046 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[s7r]

On 4/16/2015 8:34 PM, Mark Friedenbach wrote:
> At this moment anyone can alter the txid. Assume transactions are 100%
> malleable.
> 

Anyone can alter the txid - more details needed. The number of altered
txids in practice is not so high in order to make us believe anyone can
do it easily. It is obvious that all current bitcoin transactions are
malleable, but not by anyone and not that easy. At least I like to think so.

From your answer I understand that right now if I create a transaction
(tx1) and broadcast it, you can alter its txid at your will, without any
mining power and/or access to my private keys so I would end up not
recognizing my own transaction and probably my change too (if my systems
rely hardly on txid)?

> On Apr 16, 2015 9:13 AM, "s7r" <s7r@sky-ip.org <mailto:s7r@sky-ip.org>>
> wrote:
> 
>     Hi Pieter,
> 
>     Thanks for your reply. I agree. Allen has a good point in the previous
>     email too, so the suggestion might not fix anything and complicate
>     things.
> 
>     The problem I am trying to solve is making all transactions
>     non-malleable by default. I guess there is a very good reason why BIP62
>     will not touch v1 anyway.
> 
>     I am trying to build a bitcoin contract which will relay on 3 things:
>     - coinjoin / txes with inputs from multiple users which are signed by
>     all users after they are merged together (every user is sure his coins
>     will not be spent without the other users to spend anything, as per
>     agreed contract);
>     - pre-signed txes with nLockTime 'n' weeks. These txes will be signed
>     before the inputs being spent are broadcasted/confirmed, using the txid
>     provided by the user before broadcasting it. Malleability hurts here.
>     - P2SH
> 
>     In simple terms, how malleable transactions really are in the network at
>     this moment? Who can alter a txid without invalidating the tx? Just the
>     parties who sign it? The miners? Anyone in the network? This is a little
>     bit unclear to me.
> 
>     Another thing I would like to confirm, the 3 pieces of the bitcoin
>     protocol mentioned above will be supported in _any_ future transaction
>     version or block version, regardless what changes are made or features
>     added to bitcoin core? The contract needs to be built and left unchanged
>     for a very very long period of time...
> 
> 
>     On 4/16/2015 8:22 AM, Pieter Wuille wrote:
>     >
>     > On Apr 16, 2015 1:46 AM, "s7r" <s7r@sky-ip.org
>     <mailto:s7r@sky-ip.org> <mailto:s7r@sky-ip.org <mailto:s7r@sky-ip.org>>>
>     > wrote:
>     >> but for transaction versions? In simple terms, if > 75% from all the
>     >> transactions in the latest 1000 blocks are version 'n', mark all
>     >> previous transaction versions as non-standard and if > 95% from
>     all the
>     >> transactions in the latest 1000 blocks are version 'n' mark all
>     previous
>     >> transaction versions as invalid.
>     >
>     > What problem are you trying to solve?
>     >
>     > The reason why BIP62 (as specified, it is just a draft) does not
>     make v1
>     > transactions invalid is because it is opt-in. The creator of a
>     > transaction needs to agree to protect it from malleability, and this
>     > subjects him to extra rules in the creation.
>     >
>     > Forcing v3 transactions would require every piece of wallet
>     software to
>     > be changed.
>     >
>     > --
>     > Pieter
>     >
> 
>     ------------------------------------------------------------------------------
>     BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>     Develop your own process in accordance with the BPMN 2 standard
>     Learn Process modeling best practices with Bonita BPM through live
>     exercises
>     http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>     event?utm_
>     source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>     _______________________________________________
>     Bitcoin-development mailing list
>     Bitcoin-development@lists.sourceforge.net
>     <mailto:Bitcoin-development@lists.sourceforge.net>
>     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Pieter Wuille]

[-- Attachment #1: Type: text/plain, Size: 1423 bytes --]

> Anyone can alter the txid - more details needed. The number of altered
> txids in practice is not so high in order to make us believe anyone can
> do it easily. It is obvious that all current bitcoin transactions are
> malleable, but not by anyone and not that easy. At least I like to think
so.

Don't assume that because it does not (frequently) happen, that it cannot
happen. Large amounts of malleated transactions have happened in the past.
Especially if you build a system depends on non-malleability for its
security, you may at some point have an attacker who has financial gain
from malleation.

> >From your answer I understand that right now if I create a transaction
> (tx1) and broadcast it, you can alter its txid at your will, without any
> mining power and/or access to my private keys so I would end up not
> recognizing my own transaction and probably my change too (if my systems
> rely hardly on txid)?

In theory, yes, anyone can alter the txid without invalidating it, without
mining power and without access to the sender's private keys.

All it requires is seeing a transaction on the network, doing a trivial
modification to it, and rebroadcasting it quickly. If the modifies version
gets mined, you're out of luck. Having mining power helps of course.

After BIP62, you will, as a sender, optionally be able to protect others
from malleating. You're always able to re-sign yourself.

-- 
Pieter

[-- Attachment #2: Type: text/html, Size: 1639 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[s7r]

Understood. That is unfortunate, but not the end of the world. If you
could please give feedback also to these last comments / questions:

How far are we at this moment from BIP62? Can an user send a
non-malleable tx now, if enforces some additional rules?

As for the security of the system, it does not fully rely on txids being
non malleable, but see this quote from my previous email:

[QUOTE]
I am trying to build a bitcoin contract which will relay on 3 things:
- coinjoin / txes with inputs from multiple users which are signed by
all users after they are merged together (every user is sure his coins
will not be spent without the other users to spend anything, as per
agreed contract);
- pre-signed txes with nLockTime 'n' weeks. These txes will be signed
before the inputs being spent are broadcasted/confirmed, using the txid
provided by the user before broadcasting it. Malleability hurts here.
- P2SH

Another thing I would like to confirm, the 3 pieces of the bitcoin
protocol mentioned above will be supported in _any_ future transaction
version or block version, regardless what changes are made or features
added to bitcoin core? The contract needs to be built and left unchanged
for a very very long period of time...
[/END QUOTE]

Can you comment on the quote please?

So, basically transaction malleability could affect the system in the
way that a pre-signed tx which offers the insurance and which is sent to
the user before the user sends the coins (spending user's coins back to
him after a certain period of time) could be invalidated. The insurance
tx signature will still be good, but invalid overall since the input
(txid) being spent does not exist (was altered / modified). The coins
won't be stolen or lost, but a new tx needs to be signed with the
altered (new) txid, for the system to work.

So, an user creates a transaction TX1 sending the coins to the server
but does not broadcast it. Instead, he provides the txid of TX1 to the
server. Server generates another transaction TX2 which spends TX1 back
to the user, with an nLockTime. User checks and if everything ok
broadcasts TX1. In case the txid of TX1 will be altered/modified, TX2
will become invalid (since it will be spending an inexistent input), and
the server will need to re-create and sign TX2 with the new
(altered/modified) txid of TX1, as per agreed contract. Should the
server disappear after user broadcasts TX1 and before the
altered/modified txid of TX1 gets confirmed, user's coins are forever
locked. It is true that no third party can benefit from this type of
attack, only the user will result with coins locked, but it is something
which could be used by competition to make a service useless / annoying
/ too complicated or less safe to use.

How could I mitigate this?

Thanks you for your time and help.

On 4/17/2015 12:02 PM, Pieter Wuille wrote:
>> Anyone can alter the txid - more details needed. The number of altered
>> txids in practice is not so high in order to make us believe anyone can
>> do it easily. It is obvious that all current bitcoin transactions are
>> malleable, but not by anyone and not that easy. At least I like to
> think so.
> 
> Don't assume that because it does not (frequently) happen, that it
> cannot happen. Large amounts of malleated transactions have happened in
> the past. Especially if you build a system depends on non-malleability
> for its security, you may at some point have an attacker who has
> financial gain from malleation.
> 
>> >From your answer I understand that right now if I create a transaction
>> (tx1) and broadcast it, you can alter its txid at your will, without any
>> mining power and/or access to my private keys so I would end up not
>> recognizing my own transaction and probably my change too (if my systems
>> rely hardly on txid)?
> 
> In theory, yes, anyone can alter the txid without invalidating it,
> without mining power and without access to the sender's private keys.
> 
> All it requires is seeing a transaction on the network, doing a trivial
> modification to it, and rebroadcasting it quickly. If the modifies
> version gets mined, you're out of luck. Having mining power helps of course.
> 
> After BIP62, you will, as a sender, optionally be able to protect others
> from malleating. You're always able to re-sign yourself.
> 
> -- 
> Pieter
> 

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Jorge Timón]

s7r you may be interested in this video explaining several aspects of
malleability: https://www.youtube.com/watch?v=jyDE-aFqJTs
It is pre BIP62, but I believe it is very relevant and will hopefully
clear some of your doubts.
The signer of TX1 will always be able to change the signature and thus
the tx ID.

On Sat, Apr 18, 2015 at 4:49 PM, s7r <s7r@sky-ip.org> wrote:
> Understood. That is unfortunate, but not the end of the world. If you
> could please give feedback also to these last comments / questions:
>
> How far are we at this moment from BIP62? Can an user send a
> non-malleable tx now, if enforces some additional rules?
>
> As for the security of the system, it does not fully rely on txids being
> non malleable, but see this quote from my previous email:
>
> [QUOTE]
> I am trying to build a bitcoin contract which will relay on 3 things:
> - coinjoin / txes with inputs from multiple users which are signed by
> all users after they are merged together (every user is sure his coins
> will not be spent without the other users to spend anything, as per
> agreed contract);
> - pre-signed txes with nLockTime 'n' weeks. These txes will be signed
> before the inputs being spent are broadcasted/confirmed, using the txid
> provided by the user before broadcasting it. Malleability hurts here.
> - P2SH
>
> Another thing I would like to confirm, the 3 pieces of the bitcoin
> protocol mentioned above will be supported in _any_ future transaction
> version or block version, regardless what changes are made or features
> added to bitcoin core? The contract needs to be built and left unchanged
> for a very very long period of time...
> [/END QUOTE]
>
> Can you comment on the quote please?
>
> So, basically transaction malleability could affect the system in the
> way that a pre-signed tx which offers the insurance and which is sent to
> the user before the user sends the coins (spending user's coins back to
> him after a certain period of time) could be invalidated. The insurance
> tx signature will still be good, but invalid overall since the input
> (txid) being spent does not exist (was altered / modified). The coins
> won't be stolen or lost, but a new tx needs to be signed with the
> altered (new) txid, for the system to work.
>
> So, an user creates a transaction TX1 sending the coins to the server
> but does not broadcast it. Instead, he provides the txid of TX1 to the
> server. Server generates another transaction TX2 which spends TX1 back
> to the user, with an nLockTime. User checks and if everything ok
> broadcasts TX1. In case the txid of TX1 will be altered/modified, TX2
> will become invalid (since it will be spending an inexistent input), and
> the server will need to re-create and sign TX2 with the new
> (altered/modified) txid of TX1, as per agreed contract. Should the
> server disappear after user broadcasts TX1 and before the
> altered/modified txid of TX1 gets confirmed, user's coins are forever
> locked. It is true that no third party can benefit from this type of
> attack, only the user will result with coins locked, but it is something
> which could be used by competition to make a service useless / annoying
> / too complicated or less safe to use.
>
> How could I mitigate this?
>
> Thanks you for your time and help.
>
> On 4/17/2015 12:02 PM, Pieter Wuille wrote:
>>> Anyone can alter the txid - more details needed. The number of altered
>>> txids in practice is not so high in order to make us believe anyone can
>>> do it easily. It is obvious that all current bitcoin transactions are
>>> malleable, but not by anyone and not that easy. At least I like to
>> think so.
>>
>> Don't assume that because it does not (frequently) happen, that it
>> cannot happen. Large amounts of malleated transactions have happened in
>> the past. Especially if you build a system depends on non-malleability
>> for its security, you may at some point have an attacker who has
>> financial gain from malleation.
>>
>>> >From your answer I understand that right now if I create a transaction
>>> (tx1) and broadcast it, you can alter its txid at your will, without any
>>> mining power and/or access to my private keys so I would end up not
>>> recognizing my own transaction and probably my change too (if my systems
>>> rely hardly on txid)?
>>
>> In theory, yes, anyone can alter the txid without invalidating it,
>> without mining power and without access to the sender's private keys.
>>
>> All it requires is seeing a transaction on the network, doing a trivial
>> modification to it, and rebroadcasting it quickly. If the modifies
>> version gets mined, you're out of luck. Having mining power helps of course.
>>
>> After BIP62, you will, as a sender, optionally be able to protect others
>> from malleating. You're always able to re-sign yourself.
>>
>> --
>> Pieter
>>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Jorge Timón]

Oh, no, sorry, it also covers bip62.

On Fri, Apr 24, 2015 at 10:55 AM, Jorge Timón <jtimon@jtimon.cc> wrote:
> s7r you may be interested in this video explaining several aspects of
> malleability: https://www.youtube.com/watch?v=jyDE-aFqJTs
> It is pre BIP62, but I believe it is very relevant and will hopefully
> clear some of your doubts.
> The signer of TX1 will always be able to change the signature and thus
> the tx ID.
>
> On Sat, Apr 18, 2015 at 4:49 PM, s7r <s7r@sky-ip.org> wrote:
>> Understood. That is unfortunate, but not the end of the world. If you
>> could please give feedback also to these last comments / questions:
>>
>> How far are we at this moment from BIP62? Can an user send a
>> non-malleable tx now, if enforces some additional rules?
>>
>> As for the security of the system, it does not fully rely on txids being
>> non malleable, but see this quote from my previous email:
>>
>> [QUOTE]
>> I am trying to build a bitcoin contract which will relay on 3 things:
>> - coinjoin / txes with inputs from multiple users which are signed by
>> all users after they are merged together (every user is sure his coins
>> will not be spent without the other users to spend anything, as per
>> agreed contract);
>> - pre-signed txes with nLockTime 'n' weeks. These txes will be signed
>> before the inputs being spent are broadcasted/confirmed, using the txid
>> provided by the user before broadcasting it. Malleability hurts here.
>> - P2SH
>>
>> Another thing I would like to confirm, the 3 pieces of the bitcoin
>> protocol mentioned above will be supported in _any_ future transaction
>> version or block version, regardless what changes are made or features
>> added to bitcoin core? The contract needs to be built and left unchanged
>> for a very very long period of time...
>> [/END QUOTE]
>>
>> Can you comment on the quote please?
>>
>> So, basically transaction malleability could affect the system in the
>> way that a pre-signed tx which offers the insurance and which is sent to
>> the user before the user sends the coins (spending user's coins back to
>> him after a certain period of time) could be invalidated. The insurance
>> tx signature will still be good, but invalid overall since the input
>> (txid) being spent does not exist (was altered / modified). The coins
>> won't be stolen or lost, but a new tx needs to be signed with the
>> altered (new) txid, for the system to work.
>>
>> So, an user creates a transaction TX1 sending the coins to the server
>> but does not broadcast it. Instead, he provides the txid of TX1 to the
>> server. Server generates another transaction TX2 which spends TX1 back
>> to the user, with an nLockTime. User checks and if everything ok
>> broadcasts TX1. In case the txid of TX1 will be altered/modified, TX2
>> will become invalid (since it will be spending an inexistent input), and
>> the server will need to re-create and sign TX2 with the new
>> (altered/modified) txid of TX1, as per agreed contract. Should the
>> server disappear after user broadcasts TX1 and before the
>> altered/modified txid of TX1 gets confirmed, user's coins are forever
>> locked. It is true that no third party can benefit from this type of
>> attack, only the user will result with coins locked, but it is something
>> which could be used by competition to make a service useless / annoying
>> / too complicated or less safe to use.
>>
>> How could I mitigate this?
>>
>> Thanks you for your time and help.
>>
>> On 4/17/2015 12:02 PM, Pieter Wuille wrote:
>>>> Anyone can alter the txid - more details needed. The number of altered
>>>> txids in practice is not so high in order to make us believe anyone can
>>>> do it easily. It is obvious that all current bitcoin transactions are
>>>> malleable, but not by anyone and not that easy. At least I like to
>>> think so.
>>>
>>> Don't assume that because it does not (frequently) happen, that it
>>> cannot happen. Large amounts of malleated transactions have happened in
>>> the past. Especially if you build a system depends on non-malleability
>>> for its security, you may at some point have an attacker who has
>>> financial gain from malleation.
>>>
>>>> >From your answer I understand that right now if I create a transaction
>>>> (tx1) and broadcast it, you can alter its txid at your will, without any
>>>> mining power and/or access to my private keys so I would end up not
>>>> recognizing my own transaction and probably my change too (if my systems
>>>> rely hardly on txid)?
>>>
>>> In theory, yes, anyone can alter the txid without invalidating it,
>>> without mining power and without access to the sender's private keys.
>>>
>>> All it requires is seeing a transaction on the network, doing a trivial
>>> modification to it, and rebroadcasting it quickly. If the modifies
>>> version gets mined, you're out of luck. Having mining power helps of course.
>>>
>>> After BIP62, you will, as a sender, optionally be able to protect others
>>> from malleating. You're always able to re-sign yourself.
>>>
>>> --
>>> Pieter
>>>
>>
>> ------------------------------------------------------------------------------
>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> Develop your own process in accordance with the BPMN 2 standard
>> Learn Process modeling best practices with Bonita BPM through live exercises
>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[William Swanson]

On Thu, Apr 16, 2015 at 9:12 AM, s7r <s7r@sky-ip.org> wrote:
> Thanks for your reply. I agree. Allen has a good point in the previous
> email too, so the suggestion might not fix anything and complicate things.

The BIP 62 approach to malleability isn't the only option. Another
approach is to sign the transaction in such a way that the input
txid's are allowed to change without invalidating the signatures. That
way, if malleability happens, you just adjust you transaction to match
and re-broadcast. That proposal is here:

https://github.com/scmorse/bitcoin-misc/blob/master/sighash_proposal.md

The "Build your own nHashType" thread on this mailing list contains
the discussion.

I personally prefer this solution, since it nails the problem
completely with one simple and obvious change. The BIP 62 approach is
more like a game of wac-a-mole.

-William

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Gregory Maxwell]

On Fri, Apr 24, 2015 at 7:58 PM, William Swanson <swansontec@gmail.com> wrote:
> On Thu, Apr 16, 2015 at 9:12 AM, s7r <s7r@sky-ip.org> wrote:
>> Thanks for your reply. I agree. Allen has a good point in the previous
>> email too, so the suggestion might not fix anything and complicate things.
>
> The BIP 62 approach to malleability isn't the only option. Another
> approach is to sign the transaction in such a way that the input
> txid's are allowed to change without invalidating the signatures. That
> way, if malleability happens, you just adjust you transaction to match
> and re-broadcast. That proposal is here:

This is not a free choice. There are several concerns, from mild to
severe, that arise when you do not sign enough.

In particular not covering the ID allows for transaction replay which
can result in monetary losses far more severe than any possible
mishandling of malleability could result in. Byzantine attackers can
costlessly replay your old transactions any time anyone reuses an
address, even accidentally (which cannot be easily prevented since
they can race).

Other fun effects also show up like being able to backwards compute
signatures to result in a kind of limited covenant- coins which can
only be spent a particular way which has some implications for
fungibility. (See here for a discussion in general of covenants:
https://bitcointalk.org/index.php?topic=278122.0)

There are no free lunches;  the proposal linked to there is itself a
game of wack-a-mole with assorted masking flags; many of which we have
no notion of if they're useful for any particular application(s); and
it doesn't provide tools to address the replay issue; and in order to
'improve' malleability via that mechanism you must always mask out the
inputs completely; meaning you'd always be exposed to replay and not
just in specialized 'contract' applications where "there won't be
address reuse" could be a strong assumption enforced by the
application.

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Stephen Morse]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

Hi William,

I personally prefer this solution, since it nails the problem
> completely with one simple and obvious change. The BIP 62 approach is
> more like a game of wac-a-mole.
>

The two are complementary, not competing. BIP62 prevents *non-signers* from
mutating the transactions, which is very important. The 'Build your own
nHashType' proposal enables chained transactions even in the face of
*signers* mutating the transaction. I believe that integrating both will
lead to the best defense against transaction malleability, and will enable
more complicated uses of chained transactions (such as micropayment
channels).

Best,
Stephen

[-- Attachment #2: Type: text/html, Size: 969 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Stephen Morse]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Hi Gregory,

In particular not covering the ID allows for transaction replay which
> can result in monetary losses far more severe than any possible
> mishandling of malleability could result in. Byzantine attackers can
> costlessly replay your old transactions any time anyone reuses an
> address, even accidentally (which cannot be easily prevented since
> they can race).
>

With the SIGHASH_WITHOUT_PREV_VALUE flag, signatures have to explicitly
specify that they are to be signed without the previous UTXO's
value/amount. This means that, at worst, replay attacks can send the money
to the same place it was sent before (which in many cases is likely not be
a loss of funds), and only if the amount sent to the reused address is the
exact same as it was before. I don't think this is worse than an attacker
being able to mutate their transaction and extort a merchant who accepts
zero-conf transactions. Anyway, not signing the input ID wouldn't exactly
be the norm, there would be a defined set of flags for standard use cases.
Not signing the input TXID would only be used in specialized cases, such as
setting up micropayment channels.


> There are no free lunches;  the proposal linked to there is itself a
> game of wack-a-mole with assorted masking flags;


I agree that it is also a bit of wac-a-mole, but the defined space of
issues is possibly more limited here. There are only X number of things
that can be signed/not signed in a transaction, and the 'Build your own
nHashType' proposal enables you to fully specify which of those are being
signed. If you don't want to get burned by not fully signing your
transactions, then don't use the non-standard sighash flags.

many of which we have
> no notion of if they're useful for any particular application(s);


A few of the flags, indeed, may not ever be useful. But we can't predict
the future, and I think it's better to build in a more flexible solution
now than to wish we had more flexible nHashTypes later.

To the original point of this thread, hopefully the suggested proposal
won't be necessary as wallets will upgrade to use version 3 transactions
and the rules associated with them over time.

Best,
Stephen

[-- Attachment #2: Type: text/html, Size: 3016 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[s7r]

Thank you all for your comments. The youtube video was indeed very
educative and nice to watch.

It's true that malleability is not the end of the world, but it is
annoying for contracts and micropayment channels, especially refunds
spending the fund tx before it is even in the blockchain, relying solely
on its txid.

BIP62 is good for preventing 3rd parties (non signers) to mutate txids,
but cannot do anything against 2nd parties (signers). I think we can
solve both by using NORMALIZEDTXID - wouldn't this be simpler and easier
to implement? Why are we talking about P3SH when we can just upgrade
P2SH to support additional OP codes? I saw that there have been talks
about a hard fork for increasing the block size, might as well take the
opportunity and fix this for good, by implementing BIP62, NORMALIZEDTXID
as well as BIP65. Couldn't all these be part of P2SH?

On 4/25/2015 6:40 PM, Stephen Morse wrote:
> Hi Gregory,
> 
>     In particular not covering the ID allows for transaction replay which
>     can result in monetary losses far more severe than any possible
>     mishandling of malleability could result in. Byzantine attackers can
>     costlessly replay your old transactions any time anyone reuses an
>     address, even accidentally (which cannot be easily prevented since
>     they can race).
> 
> 
> With the SIGHASH_WITHOUT_PREV_VALUE flag, signatures have to explicitly
> specify that they are to be signed without the previous UTXO's
> value/amount. This means that, at worst, replay attacks can send the
> money to the same place it was sent before (which in many cases is
> likely not be a loss of funds), and only if the amount sent to the
> reused address is the exact same as it was before. I don't think this is
> worse than an attacker being able to mutate their transaction and extort
> a merchant who accepts zero-conf transactions. Anyway, not signing the
> input ID wouldn't exactly be the norm, there would be a defined set of
> flags for standard use cases. Not signing the input TXID would only be
> used in specialized cases, such as setting up micropayment channels. 
>  
> 
>     There are no free lunches;  the proposal linked to there is itself a
>     game of wack-a-mole with assorted masking flags; 
> 
> 
> I agree that it is also a bit of wac-a-mole, but the defined space of
> issues is possibly more limited here. There are only X number of things
> that can be signed/not signed in a transaction, and the 'Build your own
> nHashType' proposal enables you to fully specify which of those are
> being signed. If you don't want to get burned by not fully signing your
> transactions, then don't use the non-standard sighash flags.
> 
>     many of which we have
>     no notion of if they're useful for any particular application(s); 
> 
> 
> A few of the flags, indeed, may not ever be useful. But we can't predict
> the future, and I think it's better to build in a more flexible solution
> now than to wish we had more flexible nHashTypes later.
> 
> To the original point of this thread, hopefully the suggested proposal
> won't be necessary as wallets will upgrade to use version 3 transactions
> and the rules associated with them over time. 
> 
> Best,
> Stephen
> 
> 
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud 
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 
> 
> 
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Joseph Poon]

On Sun, Apr 26, 2015 at 03:01:10AM +0300, s7r wrote:
> It's true that malleability is not the end of the world, but it is
> annoying for contracts and micropayment channels, especially refunds
> spending the fund tx before it is even in the blockchain, relying
> solely on its txid.

Agreed, needing the transaction to be signed & broadcastable before the
refunds can be generated is similar to paying for a contract before the
terms have been decided.

>  I think we can solve both by using NORMALIZEDTXID - wouldn't this be
>  simpler and easier to implement? 

The current problem is that SIGHASH_NORMALIZED_TXID as presently
discussed implies stripping the sigScript, which is not sufficient for
the Lightning Network.

The currently discussed SIGHASH_NORMALIZED_TXID does not permit chained
transactions 2 levels deep, which is necessary for Lightning as well.
The path from the Commitment -> HTLC -> Refund requires up to 3 levels
deep of transactions. 

Suppose TxA -> TxB -> TxC -> TxD. All outputs are 2-of-2 multisig. TxA
has already entered into the blockchain, the rest have not yet been
broadcast. If TxB spends from TxA, it doesn't need new sighash flags, it
just does a plain SIGHASH_ALL. However, TxC needs
SIGHASH_NORMALIZED_TXID due to malleability risks.
SIGHASH_NORMALIZED_TXID works for TxC because the sigScript can change,
but because TxA's txid has already entered the blockchain, the parent's
input txids cannot change (with high degrees of certainty).

However, with TxD, the txid of TxB may be different, which will result
in an invalid transaction if SIGHASH_NORMALIZED_TXID only strips the
sigScript when obtaining the normalized txid of TxC. The reason for this
is TxC's input txid of TxB has changed (TxC's input 0 txid of TxB)!

Therefore, a functional SIGHASH_NORMALIZED which permits chained
transactions requires the parent transaction's sigScript *AND* txid to
be stripped when determining the parent's normalized txid. Similar to
OP_CHECKSIG, a part of the normalized TXID includes each input's
scriptPubKey, e.g. TxC's normalized TXID includes TxB's scriptPubKey
output which it is spending, so when TxD signs TxC's normalized TXID, it
includes TxB's output (this is a cheap way of increasing uniqueness but
is not an absolute necessity if it's too difficult). All this data
should be immediately available when validating the transaction and
appending it to the UTXO set.

If the txid and sigScript are removed when building the normalized input
txid as part of the spend/signature, it should be possible for chained
transactions to work. However, this isn't absolute security against
replay attacks. If there are two spends with all inputs having the same
values *AND* the same scriptPubKeys per input, then it can be replayed.
The odds of this occurring seems like a sort of uncanny valley of risks;
it's low enough that it shouldn't ever happen which may result in a lack
of documentation, so when it does happen it'll be a big surprise. So,
even if this "safer" method becomes a softfork, perhaps great care
should be taken before making this a default method of spending when the
sighash flag is not an absolute necessity (i.e. "don't do it!" I'm all
in favor of giving this a scary name so developers won't inadvertently
think "hey, normalization sounds like a good thing to do").

That said, it should cover an overwhelming majority of potential
replays, it's nearly impossible to create a "duplicate" replayable tx of
someone *else's* send, since the poteintally "replayable" transaction
signs the sigScript of the redeemed output.

As a side note, SIGHASH_NORMALIZED does not permit spending from any
transaction, which is desirable for the Lightning Network (HTLCs may
persist in new Commitment Transactions). However, this is merely a "nice
to have" and is not an absolute necessity, there is no significant loss
of functionality, merely some slight slowdown from significantly more
signatures. For Lightning in particular, the effect would probably be
batching Commitment Transactions (e.g. 1 mass update per second per
channel), with the only major discernable penalty is an order of
magnitude greater storage of signatures.

Additionally, I think it was Mark Friedenbach who brought up that
SIGHASH_NORMALIZED creates significant complexities with the need for an
additional hash with every UTXO (almost doubling the UTXO set size), and
with nodes which already have UTXO pruning enabled, it'll require
downloading the entire blockchain. I'm not sure if this problem is
insurmountable or not, but if a normalized sighash becomes the most
ideal candidate for a malleability soft-fork, then sooner may be better
than later as more nodes start using the pruning patch.


> Why are we talking about P3SH when we can just upgrade
> P2SH to support additional OP codes? 

Assuming you mean the current P2SH scriptPubKey format, it's not
possible to do so while making it a soft fork. If you use OP_EQUAL,
current nodes will treat "P3SH" transactions as P2SH ones.

I'm in favor of keeping P3SH conservative. It's possible to have your
cake and eat it too, by enabling script versions within P3SH.

If you create P3SH as:

OP_DUP <20-byte hash> OP_EQUALVERIFY

The redeemScript has the first byte as a version number, and there is
also an OP_TRUE pushed right before the redeemScript. The scriptSig
would look something like:

<sigs...> OP_TRUE <3 redeemScript>

When executing the script, the last item on the stack verifies against
the hash, then the redeemScript is copied/read, the 3 is popped off
(first byte unsigned int), the OP_TRUE is popped off the stack, and the
script then executes P3SH "version 3" (again, it is the first byte, NOT
an opcode). Any non-known version will return everything as true and not
continue with execution of the script, to permit future soft-forks. The
OP_TRUE is to ensure there is a OP_TRUE left on the stack just in case
for older nodes as this is an EQUALVERIFY.

This works because the address, 20-byte hash, has the 3 version number
as part of the hash, so it is the recipient who determines the version
number. For future soft-forks, it's incredibly flexible, just make the
version byte to 4. Prior addresses work the same, and it's not possible
to accidentally send it using different scripting versions. Perhaps this
can make things upgradeable enough that a malleability sighash flag can
go in sooner rather than later.

-- 
Joseph Poon

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Joseph Poon]

On Sat, Apr 25, 2015 at 11:51:37PM -0700, Joseph Poon wrote:
> signs the sigScript of the redeemed output.

Err, typo, I meant:
... signs the *scriptPubKey* of the redeemed output.

-- 
Joseph Poon

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1925 bytes --]

On Sat, Apr 25, 2015 at 10:32:36AM -0400, Stephen Morse wrote:
> Hi William,
> 
> I personally prefer this solution, since it nails the problem
> > completely with one simple and obvious change. The BIP 62 approach is
> > more like a game of wac-a-mole.
> >
> 
> The two are complementary, not competing. BIP62 prevents *non-signers* from
> mutating the transactions, which is very important.

I strongly disagree.

There are exactly two cases where mutation matters to normal wallets:

1) Spending unconfirmed change. This can be more efficiently done by
   double-spending the first tx with a second that pays both recipients.

2) Large reorganizations. Making mutation impossible makes it more
   likely that after a large reorg all previously confirmed transactions
   will make it back to the blockchain succesfully.

Meanwhile, the "whack-a-mole" aspect of BIP62 is worrying - it's very
likely we'll miss a case. Even right now there are edge cases without
good solutions, like how in a multisig environment any of the key
holders can mutate transactions. Building wallets that make strong
assumptions about malleability and fail if those assumptions turn out to
be wrong is poor engineering.

> The 'Build your own
> nHashType' proposal enables chained transactions even in the face of
> *signers* mutating the transaction. I believe that integrating both will
> lead to the best defense against transaction malleability, and will enable
> more complicated uses of chained transactions (such as micropayment
> channels).

While I think there are better ways to do 'Build your own nHashType'
than what was recently proposed, I strongly agree that for protocols
that really, truly, need malleability resistance it's far better to use
a purpose-built signature hashing algorithm.

-- 
'peter'[:-1]@petertodd.org
00000000000000000e7980aab9c096c46e7f34c43a661c5cb2ea71525ebb8af7

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [Bitcoin-development] 75%/95% threshold for transaction versions

[Oleg Andreev]

> On 27 Apr 2015, at 21:21, Peter Todd <pete@petertodd.org> wrote:
> 
> Even right now there are edge cases without
> good solutions, like how in a multisig environment any of the key
> holders can mutate transactions.

Can't we add requirement for RFC6979 signatures to mitigate this? Of course, multiple signers can still mutate transaction by choosing a different set (but not the order, thankfully) of signatures. Or when a single signer has multiple participating keys.

In some interesting to me scenarios mutation by signer is not critical: it is mutation by non-signers that creates a problem. Do you know of any edge cases when non-signers can mutate transactions which are not covered by BIP62? What would be a more robust approach than "whack-a-mole" to work around mutability? (Normalized tx ids?)