[Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

Hello all,

it was something I planned to do since a long time, but with the
recent related issues popping up, I finally got around to writing a
BIP about how we can get rid of transaction malleability over time.

The proposed document is here: https://gist.github.com/sipa/8907691

I expect most rules to not be controversial. Maybe rules 1 and 3, as
they require modifications to wallet software (Bitcoin Core 0.9 and
BitcoinJ already implement it, though) and potentially invalidate some
script functionality. However, these new rules remain optional and
controlled by an nVersion increase.

Comments please!

-- 
Pieter

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2108 bytes --]

On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
> Hello all,
> 
> it was something I planned to do since a long time, but with the
> recent related issues popping up, I finally got around to writing a
> BIP about how we can get rid of transaction malleability over time.
> 
> The proposed document is here: https://gist.github.com/sipa/8907691
> 
> I expect most rules to not be controversial. Maybe rules 1 and 3, as
> they require modifications to wallet software (Bitcoin Core 0.9 and
> BitcoinJ already implement it, though) and potentially invalidate some
> script functionality. However, these new rules remain optional and
> controlled by an nVersion increase.
> 
> Comments please!

You should probably add making CHECKMULTISIG require the dummy value to
be exactly equal to OP_FALSE; verifying that in the transaction itself is
laborious. A more subtle example is we may want both CHECKSIG and
CHECKMULTISIG to fail the transaction if the signature is invalid but
not exactly equal to OP_FALSE; some transaction forms are significantly
more compact if you can have failed signatures, but that's a source of
malleability. (are there counter examples people can think of?)


But as I said on IRC, I'm a bit hesitant to bake in assumptions about
malleability when we have no solid idea if ECC signatures are or are not
malleable on a fundemental level; if "whack-a-mole" anti-malleability is
all we've got it could be ugly if a break is found. Similarly, we may
find we missed something, or some needed change makes the malleability
rules difficult to work with for some new script type that is required.

I'd rather see a new CHECKSIG mode for the case where malleability
absolutely must be eliminated - certain multi-party protocols - and fix
wallet software instead. (the malleability problems people see are
closely related to inability to handle double-spends and reorgs) But I
can easily see that being an impossible goal engineering wise...

-- 
'peter'[:-1]@petertodd.org
0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 685 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Luke-Jr]

On Sunday, February 09, 2014 11:33:02 PM Pieter Wuille wrote:
> The proposed document is here: https://gist.github.com/sipa/8907691

Rule 3 & 4 are already enforced.

AFAIK nVersion==3 transactions are not currently considered non-standard?

Luke

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Rune Kjær Svendsen]

Instead of trying to remove the possibility of transaction
malleability, would it make sense to define a new, "canonical
transaction hash/ID" (cTxID), which would be a hash of the part of the
transaction data which we know is not malleable, and have clients use
this cTxID internally, thus making the traditional transaction hash
irrelevant for a client to function correctly?

We already have a non-malleable transaction hash: the hash that is
signed, ie. the transaction with each scriptSig replaced by the
scriptPubKey it redeems. This could be the cTxID.

Or is this simply a too fundamental change to the way bitcoin-qt (and
all other clients) work in order to be feasible?

As far as I can see, it completely solves the issue of not having a
canonical ID for a transaction, but it also increases the
computational requirements for a node. For one, as far as I can see,
it requires the node to index all transactions, because in order to
calculate a cTxID, it would be necessary to fetch all transactions
referred to by the transaction in question, in order to pull in the
scriptPubKeys that are redeemed.


On Mon, Feb 10, 2014 at 4:00 AM, Peter Todd <pete@petertodd.org> wrote:
> On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
>> Hello all,
>>
>> it was something I planned to do since a long time, but with the
>> recent related issues popping up, I finally got around to writing a
>> BIP about how we can get rid of transaction malleability over time.
>>
>> The proposed document is here: https://gist.github.com/sipa/8907691
>>
>> I expect most rules to not be controversial. Maybe rules 1 and 3, as
>> they require modifications to wallet software (Bitcoin Core 0.9 and
>> BitcoinJ already implement it, though) and potentially invalidate some
>> script functionality. However, these new rules remain optional and
>> controlled by an nVersion increase.
>>
>> Comments please!
>
> You should probably add making CHECKMULTISIG require the dummy value to
> be exactly equal to OP_FALSE; verifying that in the transaction itself is
> laborious. A more subtle example is we may want both CHECKSIG and
> CHECKMULTISIG to fail the transaction if the signature is invalid but
> not exactly equal to OP_FALSE; some transaction forms are significantly
> more compact if you can have failed signatures, but that's a source of
> malleability. (are there counter examples people can think of?)
>
>
> But as I said on IRC, I'm a bit hesitant to bake in assumptions about
> malleability when we have no solid idea if ECC signatures are or are not
> malleable on a fundemental level; if "whack-a-mole" anti-malleability is
> all we've got it could be ugly if a break is found. Similarly, we may
> find we missed something, or some needed change makes the malleability
> rules difficult to work with for some new script type that is required.
>
> I'd rather see a new CHECKSIG mode for the case where malleability
> absolutely must be eliminated - certain multi-party protocols - and fix
> wallet software instead. (the malleability problems people see are
> closely related to inability to handle double-spends and reorgs) But I
> can easily see that being an impossible goal engineering wise...
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Alan Reiner]

[-- Attachment #1: Type: text/plain, Size: 5464 bytes --]

I think the solution is simply to encourage Bitcoin software developers to
design their software to use this static ID, instead of the full
transaction hash.    If MtGox had talked those IDs instead of the TX ID,
their software would've correctly identified the mutated transactions and
there would be  no problem.

Armory is slightly different, since it doesn't deal with the same stuff as
exchanges do.  But it didn't have any problems with malleability because it
doesn't track anything by ID, it only pays attention to whether inputs and
outputs are related to your wallets.  It's not necessarily hard to do it
this way, people just have to be aware of it.

-Alan

Sent from my overpriced smartphone
On Feb 12, 2014 10:15 AM, "Rune Kjær Svendsen" <runesvend@gmail.com> wrote:

> Instead of trying to remove the possibility of transaction
> malleability, would it make sense to define a new, "canonical
> transaction hash/ID" (cTxID), which would be a hash of the part of the
> transaction data which we know is not malleable, and have clients use
> this cTxID internally, thus making the traditional transaction hash
> irrelevant for a client to function correctly?
>
> We already have a non-malleable transaction hash: the hash that is
> signed, ie. the transaction with each scriptSig replaced by the
> scriptPubKey it redeems. This could be the cTxID.
>
> Or is this simply a too fundamental change to the way bitcoin-qt (and
> all other clients) work in order to be feasible?
>
> As far as I can see, it completely solves the issue of not having a
> canonical ID for a transaction, but it also increases the
> computational requirements for a node. For one, as far as I can see,
> it requires the node to index all transactions, because in order to
> calculate a cTxID, it would be necessary to fetch all transactions
> referred to by the transaction in question, in order to pull in the
> scriptPubKeys that are redeemed.
>
>
> On Mon, Feb 10, 2014 at 4:00 AM, Peter Todd <pete@petertodd.org> wrote:
> > On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
> >> Hello all,
> >>
> >> it was something I planned to do since a long time, but with the
> >> recent related issues popping up, I finally got around to writing a
> >> BIP about how we can get rid of transaction malleability over time.
> >>
> >> The proposed document is here: https://gist.github.com/sipa/8907691
> >>
> >> I expect most rules to not be controversial. Maybe rules 1 and 3, as
> >> they require modifications to wallet software (Bitcoin Core 0.9 and
> >> BitcoinJ already implement it, though) and potentially invalidate some
> >> script functionality. However, these new rules remain optional and
> >> controlled by an nVersion increase.
> >>
> >> Comments please!
> >
> > You should probably add making CHECKMULTISIG require the dummy value to
> > be exactly equal to OP_FALSE; verifying that in the transaction itself is
> > laborious. A more subtle example is we may want both CHECKSIG and
> > CHECKMULTISIG to fail the transaction if the signature is invalid but
> > not exactly equal to OP_FALSE; some transaction forms are significantly
> > more compact if you can have failed signatures, but that's a source of
> > malleability. (are there counter examples people can think of?)
> >
> >
> > But as I said on IRC, I'm a bit hesitant to bake in assumptions about
> > malleability when we have no solid idea if ECC signatures are or are not
> > malleable on a fundemental level; if "whack-a-mole" anti-malleability is
> > all we've got it could be ugly if a break is found. Similarly, we may
> > find we missed something, or some needed change makes the malleability
> > rules difficult to work with for some new script type that is required.
> >
> > I'd rather see a new CHECKSIG mode for the case where malleability
> > absolutely must be eliminated - certain multi-party protocols - and fix
> > wallet software instead. (the malleability problems people see are
> > closely related to inability to handle double-spends and reorgs) But I
> > can easily see that being an impossible goal engineering wise...
> >
> > --
> > 'peter'[:-1]@petertodd.org
> > 0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1
> >
> >
> ------------------------------------------------------------------------------
> > Managing the Performance of Cloud-Based Applications
> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> > Read the Whitepaper.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> >
>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 6899 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 8173 bytes --]

While that solution does work for many use cases, it does make it much
harder to do anything needing chained transactions.  Granted, this is the
short term solution for current implementations, but having a transaction
identifier that does not change does open up other use cases.

For example, Alice wants to send coins to a multisignature address with
Bob, such that both parties are required to spend the coins.  Alice also
requires for Bob to send coins to this address as well before they will
proceed.  Alice cannot guarantee that Bob will cooperate (and vice versa),
so before she broadcasts the transaction to send to A+B, she sends Bob a
transaction that spends her incoming transaction back to herself, but has a
time lock of far into the future.  Bob signs this, returns it to Alice, and
she broadcasts her funding transaction.  At this point, Bob disappears,
loses his key, or just decides to spite Alice and her coins are locked.
 Since she has a refund transaction, she can broadcast it in a month and
get her coins back.  Except her funding transaction has been modified such
that the txhash is different, so her refund is now invalid.  She would need
Bob to issue a new refund as soon as her funding transaction hits the
blockchain if it is modified, which defeats the point of the trustless
refund transaction.

Longer term it would be more ideal have a canonical identifier for the
transaction before it even gets to the chain to support these use cases,
even if wallets are able to properly identify the status of it's
transactions.  Obviously this is a difficult problem to solve and cannot be
implemented without breaking changes, but it would be a nice goal to be
able to completely remove malleability.  There are other important use
cases where having a unique identifier just for internal accounting is
insufficient.

-Allen


On Wed, Feb 12, 2014 at 10:22 AM, Alan Reiner <etotheipi@gmail.com> wrote:

> I think the solution is simply to encourage Bitcoin software developers to
> design their software to use this static ID, instead of the full
> transaction hash.    If MtGox had talked those IDs instead of the TX ID,
> their software would've correctly identified the mutated transactions and
> there would be  no problem.
>
> Armory is slightly different, since it doesn't deal with the same stuff as
> exchanges do.  But it didn't have any problems with malleability because it
> doesn't track anything by ID, it only pays attention to whether inputs and
> outputs are related to your wallets.  It's not necessarily hard to do it
> this way, people just have to be aware of it.
>
> -Alan
>
> Sent from my overpriced smartphone
> On Feb 12, 2014 10:15 AM, "Rune Kjær Svendsen" <runesvend@gmail.com>
> wrote:
>
>> Instead of trying to remove the possibility of transaction
>> malleability, would it make sense to define a new, "canonical
>> transaction hash/ID" (cTxID), which would be a hash of the part of the
>> transaction data which we know is not malleable, and have clients use
>> this cTxID internally, thus making the traditional transaction hash
>> irrelevant for a client to function correctly?
>>
>> We already have a non-malleable transaction hash: the hash that is
>> signed, ie. the transaction with each scriptSig replaced by the
>> scriptPubKey it redeems. This could be the cTxID.
>>
>> Or is this simply a too fundamental change to the way bitcoin-qt (and
>> all other clients) work in order to be feasible?
>>
>> As far as I can see, it completely solves the issue of not having a
>> canonical ID for a transaction, but it also increases the
>> computational requirements for a node. For one, as far as I can see,
>> it requires the node to index all transactions, because in order to
>> calculate a cTxID, it would be necessary to fetch all transactions
>> referred to by the transaction in question, in order to pull in the
>> scriptPubKeys that are redeemed.
>>
>>
>> On Mon, Feb 10, 2014 at 4:00 AM, Peter Todd <pete@petertodd.org> wrote:
>> > On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
>> >> Hello all,
>> >>
>> >> it was something I planned to do since a long time, but with the
>> >> recent related issues popping up, I finally got around to writing a
>> >> BIP about how we can get rid of transaction malleability over time.
>> >>
>> >> The proposed document is here: https://gist.github.com/sipa/8907691
>> >>
>> >> I expect most rules to not be controversial. Maybe rules 1 and 3, as
>> >> they require modifications to wallet software (Bitcoin Core 0.9 and
>> >> BitcoinJ already implement it, though) and potentially invalidate some
>> >> script functionality. However, these new rules remain optional and
>> >> controlled by an nVersion increase.
>> >>
>> >> Comments please!
>> >
>> > You should probably add making CHECKMULTISIG require the dummy value to
>> > be exactly equal to OP_FALSE; verifying that in the transaction itself
>> is
>> > laborious. A more subtle example is we may want both CHECKSIG and
>> > CHECKMULTISIG to fail the transaction if the signature is invalid but
>> > not exactly equal to OP_FALSE; some transaction forms are significantly
>> > more compact if you can have failed signatures, but that's a source of
>> > malleability. (are there counter examples people can think of?)
>> >
>> >
>> > But as I said on IRC, I'm a bit hesitant to bake in assumptions about
>> > malleability when we have no solid idea if ECC signatures are or are not
>> > malleable on a fundemental level; if "whack-a-mole" anti-malleability is
>> > all we've got it could be ugly if a break is found. Similarly, we may
>> > find we missed something, or some needed change makes the malleability
>> > rules difficult to work with for some new script type that is required.
>> >
>> > I'd rather see a new CHECKSIG mode for the case where malleability
>> > absolutely must be eliminated - certain multi-party protocols - and fix
>> > wallet software instead. (the malleability problems people see are
>> > closely related to inability to handle double-spends and reorgs) But I
>> > can easily see that being an impossible goal engineering wise...
>> >
>> > --
>> > 'peter'[:-1]@petertodd.org
>> > 0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Managing the Performance of Cloud-Based Applications
>> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> > Read the Whitepaper.
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Bitcoin-development mailing list
>> > Bitcoin-development@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Android apps run on BlackBerry 10
>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>> Get your Android app in front of a whole new audience.  Start now.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 10230 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Alan Reiner]

[-- Attachment #1: Type: text/plain, Size: 8943 bytes --]

Agreed.  I'm not suggesting that malleability shouldn't be fixed or isn't a
problem.  I would love to be able to leverage chained TX for Bitcoin
contracts.  But that in its current state it doesn't have to be complicated
to deal with  it.

Changing the protocol to use these static IDs is a pretty fundamental
change that would never happen in Bitcoin.   But they can still be useful
at the application level to mitigate these issues.

Sent from my overpriced smartphone
On Feb 12, 2014 11:38 AM, "Allen Piscitello" <allen.piscitello@gmail.com>
wrote:

> While that solution does work for many use cases, it does make it much
> harder to do anything needing chained transactions.  Granted, this is the
> short term solution for current implementations, but having a transaction
> identifier that does not change does open up other use cases.
>
> For example, Alice wants to send coins to a multisignature address with
> Bob, such that both parties are required to spend the coins.  Alice also
> requires for Bob to send coins to this address as well before they will
> proceed.  Alice cannot guarantee that Bob will cooperate (and vice versa),
> so before she broadcasts the transaction to send to A+B, she sends Bob a
> transaction that spends her incoming transaction back to herself, but has a
> time lock of far into the future.  Bob signs this, returns it to Alice, and
> she broadcasts her funding transaction.  At this point, Bob disappears,
> loses his key, or just decides to spite Alice and her coins are locked.
>  Since she has a refund transaction, she can broadcast it in a month and
> get her coins back.  Except her funding transaction has been modified such
> that the txhash is different, so her refund is now invalid.  She would need
> Bob to issue a new refund as soon as her funding transaction hits the
> blockchain if it is modified, which defeats the point of the trustless
> refund transaction.
>
> Longer term it would be more ideal have a canonical identifier for the
> transaction before it even gets to the chain to support these use cases,
> even if wallets are able to properly identify the status of it's
> transactions.  Obviously this is a difficult problem to solve and cannot be
> implemented without breaking changes, but it would be a nice goal to be
> able to completely remove malleability.  There are other important use
> cases where having a unique identifier just for internal accounting is
> insufficient.
>
> -Allen
>
>
> On Wed, Feb 12, 2014 at 10:22 AM, Alan Reiner <etotheipi@gmail.com> wrote:
>
>> I think the solution is simply to encourage Bitcoin software developers
>> to design their software to use this static ID, instead of the full
>> transaction hash.    If MtGox had talked those IDs instead of the TX ID,
>> their software would've correctly identified the mutated transactions and
>> there would be  no problem.
>>
>> Armory is slightly different, since it doesn't deal with the same stuff
>> as exchanges do.  But it didn't have any problems with malleability because
>> it doesn't track anything by ID, it only pays attention to whether inputs
>> and outputs are related to your wallets.  It's not necessarily hard to do
>> it this way, people just have to be aware of it.
>>
>> -Alan
>>
>> Sent from my overpriced smartphone
>> On Feb 12, 2014 10:15 AM, "Rune Kjær Svendsen" <runesvend@gmail.com>
>> wrote:
>>
>>> Instead of trying to remove the possibility of transaction
>>> malleability, would it make sense to define a new, "canonical
>>> transaction hash/ID" (cTxID), which would be a hash of the part of the
>>> transaction data which we know is not malleable, and have clients use
>>> this cTxID internally, thus making the traditional transaction hash
>>> irrelevant for a client to function correctly?
>>>
>>> We already have a non-malleable transaction hash: the hash that is
>>> signed, ie. the transaction with each scriptSig replaced by the
>>> scriptPubKey it redeems. This could be the cTxID.
>>>
>>> Or is this simply a too fundamental change to the way bitcoin-qt (and
>>> all other clients) work in order to be feasible?
>>>
>>> As far as I can see, it completely solves the issue of not having a
>>> canonical ID for a transaction, but it also increases the
>>> computational requirements for a node. For one, as far as I can see,
>>> it requires the node to index all transactions, because in order to
>>> calculate a cTxID, it would be necessary to fetch all transactions
>>> referred to by the transaction in question, in order to pull in the
>>> scriptPubKeys that are redeemed.
>>>
>>>
>>> On Mon, Feb 10, 2014 at 4:00 AM, Peter Todd <pete@petertodd.org> wrote:
>>> > On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
>>> >> Hello all,
>>> >>
>>> >> it was something I planned to do since a long time, but with the
>>> >> recent related issues popping up, I finally got around to writing a
>>> >> BIP about how we can get rid of transaction malleability over time.
>>> >>
>>> >> The proposed document is here: https://gist.github.com/sipa/8907691
>>> >>
>>> >> I expect most rules to not be controversial. Maybe rules 1 and 3, as
>>> >> they require modifications to wallet software (Bitcoin Core 0.9 and
>>> >> BitcoinJ already implement it, though) and potentially invalidate some
>>> >> script functionality. However, these new rules remain optional and
>>> >> controlled by an nVersion increase.
>>> >>
>>> >> Comments please!
>>> >
>>> > You should probably add making CHECKMULTISIG require the dummy value to
>>> > be exactly equal to OP_FALSE; verifying that in the transaction itself
>>> is
>>> > laborious. A more subtle example is we may want both CHECKSIG and
>>> > CHECKMULTISIG to fail the transaction if the signature is invalid but
>>> > not exactly equal to OP_FALSE; some transaction forms are significantly
>>> > more compact if you can have failed signatures, but that's a source of
>>> > malleability. (are there counter examples people can think of?)
>>> >
>>> >
>>> > But as I said on IRC, I'm a bit hesitant to bake in assumptions about
>>> > malleability when we have no solid idea if ECC signatures are or are
>>> not
>>> > malleable on a fundemental level; if "whack-a-mole" anti-malleability
>>> is
>>> > all we've got it could be ugly if a break is found. Similarly, we may
>>> > find we missed something, or some needed change makes the malleability
>>> > rules difficult to work with for some new script type that is required.
>>> >
>>> > I'd rather see a new CHECKSIG mode for the case where malleability
>>> > absolutely must be eliminated - certain multi-party protocols - and fix
>>> > wallet software instead. (the malleability problems people see are
>>> > closely related to inability to handle double-spends and reorgs) But I
>>> > can easily see that being an impossible goal engineering wise...
>>> >
>>> > --
>>> > 'peter'[:-1]@petertodd.org
>>> > 0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Managing the Performance of Cloud-Based Applications
>>> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>> > Read the Whitepaper.
>>> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
>>> > _______________________________________________
>>> > Bitcoin-development mailing list
>>> > Bitcoin-development@lists.sourceforge.net
>>> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Android apps run on BlackBerry 10
>>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>> Get your Android app in front of a whole new audience.  Start now.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Android apps run on BlackBerry 10
>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>> Get your Android app in front of a whole new audience.  Start now.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 11097 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pavol Rusnak]

On 02/10/2014 12:33 AM, Pieter Wuille wrote:
> The proposed document is here: https://gist.github.com/sipa/8907691

If we are bumping nVersion, how about dropping DER encoding completely
and using just 64 bytes directly for signature?

Also using 2 different variable integer encodings (in addition to what
DER already does) is very confusing.

-- 
Best Regards / S pozdravom,

Pavol Rusnak <stick@gk2.sk>

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Jeff Garzik]

On Wed, Feb 12, 2014 at 10:12 AM, Rune Kjær Svendsen
<runesvend@gmail.com> wrote:
> Instead of trying to remove the possibility of transaction
> malleability, would it make sense to define a new, "canonical
> transaction hash/ID" (cTxID),

Yes, that is one proposal:  https://github.com/sipa/bitcoin/commits/normtxid

But it is not a complete solution for all transaction types.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

It's also not necessary for wallet software - it's really just for
human consumption.

A wallet can easily detect inputs being respent in another
transaction. You don't need a static hash for that (which wouldn't
need with all hash types, non-malleability double spends, ...).

On Wed, Feb 12, 2014 at 6:13 PM, Jeff Garzik <jgarzik@bitpay.com> wrote:
> On Wed, Feb 12, 2014 at 10:12 AM, Rune Kjær Svendsen
> <runesvend@gmail.com> wrote:
>> Instead of trying to remove the possibility of transaction
>> malleability, would it make sense to define a new, "canonical
>> transaction hash/ID" (cTxID),
>
> Yes, that is one proposal:  https://github.com/sipa/bitcoin/commits/normtxid
>
> But it is not a complete solution for all transaction types.
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc.      https://bitpay.com/
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

On Wed, Feb 12, 2014 at 5:56 PM, Pavol Rusnak <stick@gk2.sk> wrote:
> On 02/10/2014 12:33 AM, Pieter Wuille wrote:
>> The proposed document is here: https://gist.github.com/sipa/8907691
>
> If we are bumping nVersion, how about dropping DER encoding completely
> and using just 64 bytes directly for signature?

That would be a hard fork. Certainly something to be discussed if we
ever introduce a version-2 scripting language, but that's a long-term
thing.

-- 
Pieter

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Wed, Feb 12, 2014 at 7:12 AM, Rune Kjær Svendsen <runesvend@gmail.com> wrote:
> Instead of trying to remove the possibility of transaction
> malleability, would it make sense to define a new, "canonical
> transaction hash/ID" (cTxID), which would be a hash of the part of the
> transaction data which we know is not malleable, and have clients use
> this cTxID internally, thus making the traditional transaction hash
> irrelevant for a client to function correctly?

This is fine and good. But it only scratches the surface of the
problems created by malleability, especially for fancier transaction
protocols.

Mutation allows you to invalidate a chain of unconfirmed transaction
by mutating the parent. This breaks any protocol which depends on
creating a precomputed nlocked time refund transaction.

So a canonical ID can be used to prevent some buggy behavior it
doesn't actually fix the problem. Fortunately the non-fixed parts
aren't too critical today.

On Wed, Feb 12, 2014 at 8:22 AM, Alan Reiner <etotheipi@gmail.com> wrote:
> I think the solution is simply to encourage Bitcoin software developers to
> design their software to use this static ID, instead of the full transaction
> hash.    If MtGox had talked those IDs instead of the TX ID, their software
> would've correctly identified the mutated transactions and there would be
> no problem.

This is incorrect.  MtGox was automatically issuing replacement
transactions resulting in double payments.

When you attempt to replace/reissue/cancel a transaction you __MUST__
double-spend the original transaction. If the original transaction has
not been conflicted then it is possible someone will pull the original
transaction out of a hat and both your replacement and the original
will be confirmed.  It is not safe at any time to look to see if the
original has been confirmed yet, and if not reissue— not because
mutation may mean you're looking in the wrong place— but because the
state of the world could change nano-seconds after you looked.

If you do double-spend the original then there is no chance that both
will go through, you'll have atomic exclusion and only one transaction
or the other will be confirmed.

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Alan Reiner]

[-- Attachment #1: Type: text/plain, Size: 3498 bytes --]

We're talking about two slightly different things.  If their system had
tracked by inputs and outputs (or some kind of static ID) , their system
wouldn't have been issuing refunds/replacements/cancellations in the first
place.

I agree with you that the reissuing code should also guarantee that both TX
can't be valid... But really their system should do both.   Without the I/O
based tracking their bookkeeping will be off, regardless of the reissuing
code,  because they can't properly associate outgoing transactions with
customer accounts/actions.

Sent from my overpriced smartphone
On Feb 12, 2014 1:06 PM, "Gregory Maxwell" <gmaxwell@gmail.com> wrote:

On Wed, Feb 12, 2014 at 7:12 AM, Rune Kjær Svendsen <runesvend@gmail.com>
wrote:
> Instead of trying to remove the possibility of transaction
> malleability, would it make sense to define a new, "canonical
> transaction hash/ID" (cTxID), which would be a hash of the part of the
> transaction data which we know is not malleable, and have clients use
> this cTxID internally, thus making the traditional transaction hash
> irrelevant for a client to function correctly?

This is fine and good. But it only scratches the surface of the
problems created by malleability, especially for fancier transaction
protocols.

Mutation allows you to invalidate a chain of unconfirmed transaction
by mutating the parent. This breaks any protocol which depends on
creating a precomputed nlocked time refund transaction.

So a canonical ID can be used to prevent some buggy behavior it
doesn't actually fix the problem. Fortunately the non-fixed parts
aren't too critical today.

On Wed, Feb 12, 2014 at 8:22 AM, Alan Reiner <etotheipi@gmail.com> wrote:
> I think the solution is simply to encourage Bitcoin software developers to
> design their software to use this static ID, instead of the full
transaction
> hash.    If MtGox had talked those IDs instead of the TX ID, their
software
> would've correctly identified the mutated transactions and there would be
> no problem.

This is incorrect.  MtGox was automatically issuing replacement
transactions resulting in double payments.

When you attempt to replace/reissue/cancel a transaction you __MUST__
double-spend the original transaction. If the original transaction has
not been conflicted then it is possible someone will pull the original
transaction out of a hat and both your replacement and the original
will be confirmed.  It is not safe at any time to look to see if the
original has been confirmed yet, and if not reissue-- not because
mutation may mean you're looking in the wrong place-- but because the
state of the world could change nano-seconds after you looked.

If you do double-spend the original then there is no chance that both
will go through, you'll have atomic exclusion and only one transaction
or the other will be confirmed.

------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience.  Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[-- Attachment #2: Type: text/html, Size: 4567 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Mark Friedenbach]

On 02/12/2014 08:44 AM, Alan Reiner wrote:
> Changing the protocol to use these static IDs is a pretty fundamental
> change that would never happen in Bitcoin.   But they can still be
> useful at the application level to mitigate these issues.

Not to mention that it would be potentially very insecure to have
consensus depend on data (scriptSigs) which are not hashed in the Merkle
structure of a block.

Not that anyone on this list has suggested such a change, but I've seen
it raised multiple times on the forum....

Mark

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Luke-Jr]

On Wednesday, February 12, 2014 8:27:52 PM Mark Friedenbach wrote:
> On 02/12/2014 08:44 AM, Alan Reiner wrote:
> > Changing the protocol to use these static IDs is a pretty fundamental
> > change that would never happen in Bitcoin.   But they can still be
> > useful at the application level to mitigate these issues.
> 
> Not to mention that it would be potentially very insecure to have
> consensus depend on data (scriptSigs) which are not hashed in the Merkle
> structure of a block.
> 
> Not that anyone on this list has suggested such a change, but I've seen
> it raised multiple times on the forum....

This would be a problem if it was used in the merkle tree, but I'm pretty sure 
using it for input selection would be pretty safe. One could even avoid the 
index by simply using the hashScript as the sole input value; then even 
CoinJoins would be safe without breaking chains of transactions (although this 
would break address reuse entirely - but I don't see that as a problem in a 
theoretical world). One of those things that an altcoin could improve upon 
Bitcoin with... ;)

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Alex Morcos]

[-- Attachment #1: Type: text/plain, Size: 2283 bytes --]

I apologize if this has been discussed many times before.

As a long term solution to malleable transactions, wouldn't it be possible
to modify the signatures to be of the entire transaction.  Why do you have
to zero out the inputs?  I can see that this would be a hard fork, and
maybe it would be somewhat tricky to extract signatures first (since you
can sign everything except the signatures), but it would seem to me that
this is an important enough change to consider making.








On Wed, Feb 12, 2014 at 5:52 PM, Luke-Jr <luke@dashjr.org> wrote:

> On Wednesday, February 12, 2014 8:27:52 PM Mark Friedenbach wrote:
> > On 02/12/2014 08:44 AM, Alan Reiner wrote:
> > > Changing the protocol to use these static IDs is a pretty fundamental
> > > change that would never happen in Bitcoin.   But they can still be
> > > useful at the application level to mitigate these issues.
> >
> > Not to mention that it would be potentially very insecure to have
> > consensus depend on data (scriptSigs) which are not hashed in the Merkle
> > structure of a block.
> >
> > Not that anyone on this list has suggested such a change, but I've seen
> > it raised multiple times on the forum....
>
> This would be a problem if it was used in the merkle tree, but I'm pretty
> sure
> using it for input selection would be pretty safe. One could even avoid the
> index by simply using the hashScript as the sole input value; then even
> CoinJoins would be safe without breaking chains of transactions (although
> this
> would break address reuse entirely - but I don't see that as a problem in a
> theoretical world). One of those things that an altcoin could improve upon
> Bitcoin with... ;)
>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 3187 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Wed, Feb 12, 2014 at 4:39 PM, Alex Morcos <morcos@gmail.com> wrote:
> I apologize if this has been discussed many times before.

It has been, but there are probably many people like you who have not
bothered researching who may also be curious.

> As a long term solution to malleable transactions, wouldn't it be possible
> to modify the signatures to be of the entire transaction.  Why do you have
> to zero out the inputs?  I can see that this would be a hard fork, and maybe
> it would be somewhat tricky to extract signatures first (since you can sign
> everything except the signatures), but it would seem to me that this is an
> important enough change to consider making.

Because doing so would be both unnecessary and ineffective.

Unnecessary because we can very likely eliminate malleability without
changing what is signed. It will take time, but we have been
incrementally moving towards that, e.g. v0.8 made many kinds of
non-canonical encoding non-standard.

Ineffective— at least as you describe it— because the signatures
_themselves_ are malleable.

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Michael Gronager]

[-- Attachment #1: Type: text/plain, Size: 3012 bytes --]

Why introduce a new transaction version for this purpose ? Wouldn't it be more elegant to simply let:

1. the next bitcoin version "prettify" all relayed transactions as deterministic transactions fulfilling the scheme 1-6 effectively blocking any malleability attack? If miners would upgrade then all transactions in blocks would have a deterministic hash. 

2. In a version later one could block relay of non deterministic transactions, as well as the acceptance of blocks with non-confirming transactions.

To non-standard conforming clients this "prettify" change of hash would be seen as a constant malleability attack, but given the "prettify" code it is to fix any client into producing only conforming transactions, just by running the transaction through it before broadcast.

There is a possible fork risk in step 2. above - if a majority of miners still havn't upgraded to 1 when 2 is introduced. We could monitor % non conforming transaction in a block and only introduce 2. once that number is sufficiently small for a certain duration - criteria:
* Switch on forcing of unmalleable transactions in blocks when there has been only conforming transactions for 1000 blocks.


On Feb 13, 2014, at 1:47 AM, Gregory Maxwell <gmaxwell@gmail.com> wrote:

> On Wed, Feb 12, 2014 at 4:39 PM, Alex Morcos <morcos@gmail.com> wrote:
>> I apologize if this has been discussed many times before.
> 
> It has been, but there are probably many people like you who have not
> bothered researching who may also be curious.
> 
>> As a long term solution to malleable transactions, wouldn't it be possible
>> to modify the signatures to be of the entire transaction.  Why do you have
>> to zero out the inputs?  I can see that this would be a hard fork, and maybe
>> it would be somewhat tricky to extract signatures first (since you can sign
>> everything except the signatures), but it would seem to me that this is an
>> important enough change to consider making.
> 
> Because doing so would be both unnecessary and ineffective.
> 
> Unnecessary because we can very likely eliminate malleability without
> changing what is signed. It will take time, but we have been
> incrementally moving towards that, e.g. v0.8 made many kinds of
> non-canonical encoding non-standard.
> 
> Ineffective— at least as you describe it— because the signatures
> _themselves_ are malleable.
> 
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

On Wed, Feb 19, 2014 at 3:11 PM, Michael Gronager <gronager@mac.com> wrote:
> Why introduce a new transaction version for this purpose ? Wouldn't it be more elegant to simply let:
>
> 1. the next bitcoin version "prettify" all relayed transactions as deterministic transactions fulfilling the scheme 1-6 effectively blocking any malleability attack? If miners would upgrade then all transactions in blocks would have a deterministic hash.

I consider actively mutating other's transactions worse than not
relaying them. If we want people to make their software deal with
malleability, either will work.

Regarding deterministic hash: that's impossible. Some signature hash
types are inherently (and intentionally) malleable. I don't think we
should pretend to want to change that. The purpose is making
non-malleability a choice the sender of a transaction can make.

Most of the rules actually are enforced by IsStandard already now.
Only #1 and #7 aren't. #1 affects the majority of all transactions, so
changing it right now would be painful. #7 only affects multisig.

> 2. In a version later one could block relay of non deterministic transactions, as well as the acceptance of blocks with non-confirming transactions.
>
> To non-standard conforming clients this "prettify" change of hash would be seen as a constant malleability attack, but given the "prettify" code it is to fix any client into producing only conforming transactions, just by running the transaction through it before broadcast.
>
> There is a possible fork risk in step 2. above - if a majority of miners still havn't upgraded to 1 when 2 is introduced. We could monitor % non conforming transaction in a block and only introduce 2. once that number is sufficiently small for a certain duration - criteria:
> * Switch on forcing of unmalleable transactions in blocks when there has been only conforming transactions for 1000 blocks.

The problem in making these rules into consensus rule (affecting
tx/block validity) is that some rules (in particular #3) may not be
wanted by everyone, as they effectively limit the possibilities of the
script language further. As it is ultimately only about protecting
senders who care about non-malleability, introducing a new transaction
version is a very neat way of accomplishing that. The new block
version number is only there to coordinate the rollout, and choosing
an automatic forking point.

-- 
Pieter

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Jeremy Spilman]

[-- Attachment #1: Type: text/plain, Size: 2194 bytes --]


> Longer term it would be more ideal have a canonical identifier for the  
> transaction before it even gets to the chain to support these use cases,  
> even if >wallets are able to properly identify the status of it's  
> transactions.  
> -Allen
>
>

One possible work-around to get back trusted transaction chaining for  
payment channels and locked refunds from multi-sig would be to make the  
initial transaction include zero fee, and depend on child-pays-for-parent  
in order to get the first and follow-on transactions into a block. This of  
course only works for protocols where the parties don't need the initial  
funding transaction to actually hit the blockchain right away.

But this relies on two assumptions; 1) that miners won't include a  
zero-fee transaction in the blockchain, and 2) that miners actually  
implement child-pays-for-parent. It's definitely not the same security  
as-if you had immutable txid, but it's something to consider.

1) Mutants may cause wallet spam and difficulty calculating balance (and  
wallets will evolve to deal with it)
2) Mutants cause DoS because they can interfere with your own transaction  
chains, which for example makes batch off-line processing much more  
difficult
3) Mutants introduce a 3rd party attacker into any two-party protocol that  
relies on chains

There's a lot to digest in the 'v3' transaction/block proposal. It sounds  
like there may be some uncertainty over whether we can *prove* that v3  
transactions in v3 blocks would actually be guaranteed immutable with  
these changes?

If we cannot fully prove a Tx is immutable, then is it actually worth  
taking steps to make it seem immutable, or is that just a false sense of  
security in the cases where chained transactions were actually expected to  
be reliable? Under that thinking, maybe it's best to accept mutants as a  
fact of life, and only consider protocols and techniques that cannot be  
broken by mutants.

In what cases does reducing the sources of malleability, but not  
necessarily eliminating from a security proof perspective, actually help?  
Basically, if we don't know that we will succeed, isn't there really no  
point in trying?

[-- Attachment #2.1: Type: text/html, Size: 2679 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Michael Gronager]

[-- Attachment #1: Type: text/plain, Size: 3434 bytes --]

Twisting your words a bit I read:

* you want to support relay of transactions that can be changed on the fly, but you consider it wrong to modify them.
* #3 is already not forwarded, but you still find it relevant to support it.

Rational use cases of #3 will be pretty hard to find given the fact that they can be changed on the fly. We are down to inclusion in blocks by miners for special purposes - or did I miss out something?

I think that we could guarantee fewer incidents by making version 1 transactions unmalleable and then optionally introduce a version 3 that supported the malleability feature. That way most existing problematic implementations would be fixed and no doors were closed for people experimenting with other stuff - tx v 3 would probably then be called experimental transactions.

/M


On Feb 19, 2014, at 3:38 PM, Pieter Wuille <pieter.wuille@gmail.com> wrote:

> On Wed, Feb 19, 2014 at 3:11 PM, Michael Gronager <gronager@mac.com> wrote:
>> Why introduce a new transaction version for this purpose ? Wouldn't it be more elegant to simply let:
>> 
>> 1. the next bitcoin version "prettify" all relayed transactions as deterministic transactions fulfilling the scheme 1-6 effectively blocking any malleability attack? If miners would upgrade then all transactions in blocks would have a deterministic hash.
> 
> I consider actively mutating other's transactions worse than not
> relaying them. If we want people to make their software deal with
> malleability, either will work.
> 
> Regarding deterministic hash: that's impossible. Some signature hash
> types are inherently (and intentionally) malleable. I don't think we
> should pretend to want to change that. The purpose is making
> non-malleability a choice the sender of a transaction can make.
> 
> Most of the rules actually are enforced by IsStandard already now.
> Only #1 and #7 aren't. #1 affects the majority of all transactions, so
> changing it right now would be painful. #7 only affects multisig.
> 
>> 2. In a version later one could block relay of non deterministic transactions, as well as the acceptance of blocks with non-confirming transactions.
>> 
>> To non-standard conforming clients this "prettify" change of hash would be seen as a constant malleability attack, but given the "prettify" code it is to fix any client into producing only conforming transactions, just by running the transaction through it before broadcast.
>> 
>> There is a possible fork risk in step 2. above - if a majority of miners still havn't upgraded to 1 when 2 is introduced. We could monitor % non conforming transaction in a block and only introduce 2. once that number is sufficiently small for a certain duration - criteria:
>> * Switch on forcing of unmalleable transactions in blocks when there has been only conforming transactions for 1000 blocks.
> 
> The problem in making these rules into consensus rule (affecting
> tx/block validity) is that some rules (in particular #3) may not be
> wanted by everyone, as they effectively limit the possibilities of the
> script language further. As it is ultimately only about protecting
> senders who care about non-malleability, introducing a new transaction
> version is a very neat way of accomplishing that. The new block
> version number is only there to coordinate the rollout, and choosing
> an automatic forking point.
> 
> -- 
> Pieter


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Wed, Feb 19, 2014 at 12:28 PM, Michael Gronager <gronager@mac.com> wrote:
> Twisting your words a bit I read:
>
> * you want to support relay of transactions that can be changed on the fly, but you consider it wrong to modify them.
> * #3 is already not forwarded, but you still find it relevant to support it.
>
> Rational use cases of #3 will be pretty hard to find given the fact that they can be changed on the fly. We are down to inclusion in blocks by miners for special purposes - or did I miss out something?

You did. See the other sighash flags.

> I think that we could guarantee fewer incidents by making version 1 transactions unmalleable and then optionally introduce a version 3 that supported the malleability feature. That way most existing problematic implementations would be fixed and no doors were closed for people experimenting with other stuff - tx v 3 would probably then be called experimental transactions.

In exchange you make the behavior basically impossible do deploy
without first blocking all ongoing transactions. This seems foolish.
All signers need to be updated to change their behavior to be
anti-malleability compatible, they can change their version at the
same time... and leave things actually working for the things which
can't be easily updated.

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Peter Todd]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

While we might be able to get away with a retroactive change in meaning right now in the future that won't be so easy. There are lots if proposed applications for nLockTime-using protocols that depend on transactions (or parts of transactions) being possible to mine as is. Making existing transactions impossible to mine in the future will break those types of applications. We might as well use this as a learning experience for what a version bump would look like infrastructures wise.

Note how the above is a particularly bad example of gmaxwell's generic "don't break things" objection. Equally, remember that lots of infrastructure *does* handle malleability just fine already.

On February 19, 2014 3:28:24 PM EST, Michael Gronager <gronager@mac.com> wrote:
>Twisting your words a bit I read:
>
>* you want to support relay of transactions that can be changed on the
>fly, but you consider it wrong to modify them.
>* #3 is already not forwarded, but you still find it relevant to
>support it.
>
>Rational use cases of #3 will be pretty hard to find given the fact
>that they can be changed on the fly. We are down to inclusion in blocks
>by miners for special purposes - or did I miss out something?
>
>I think that we could guarantee fewer incidents by making version 1
>transactions unmalleable and then optionally introduce a version 3 that
>supported the malleability feature. That way most existing problematic
>implementations would be fixed and no doors were closed for people
>experimenting with other stuff - tx v 3 would probably then be called
>experimental transactions.
>
>/M
>
>
>On Feb 19, 2014, at 3:38 PM, Pieter Wuille <pieter.wuille@gmail.com>
>wrote:
>
>> On Wed, Feb 19, 2014 at 3:11 PM, Michael Gronager <gronager@mac.com>
>wrote:
>>> Why introduce a new transaction version for this purpose ? Wouldn't
>it be more elegant to simply let:
>>>
>>> 1. the next bitcoin version "prettify" all relayed transactions as
>deterministic transactions fulfilling the scheme 1-6 effectively
>blocking any malleability attack? If miners would upgrade then all
>transactions in blocks would have a deterministic hash.
>>
>> I consider actively mutating other's transactions worse than not
>> relaying them. If we want people to make their software deal with
>> malleability, either will work.
>>
>> Regarding deterministic hash: that's impossible. Some signature hash
>> types are inherently (and intentionally) malleable. I don't think we
>> should pretend to want to change that. The purpose is making
>> non-malleability a choice the sender of a transaction can make.
>>
>> Most of the rules actually are enforced by IsStandard already now.
>> Only #1 and #7 aren't. #1 affects the majority of all transactions,
>so
>> changing it right now would be painful. #7 only affects multisig.
>>
>>> 2. In a version later one could block relay of non deterministic
>transactions, as well as the acceptance of blocks with non-confirming
>transactions.
>>>
>>> To non-standard conforming clients this "prettify" change of hash
>would be seen as a constant malleability attack, but given the
>"prettify" code it is to fix any client into producing only conforming
>transactions, just by running the transaction through it before
>broadcast.
>>>
>>> There is a possible fork risk in step 2. above - if a majority of
>miners still havn't upgraded to 1 when 2 is introduced. We could
>monitor % non conforming transaction in a block and only introduce 2.
>once that number is sufficiently small for a certain duration -
>criteria:
>>> * Switch on forcing of unmalleable transactions in blocks when there
>has been only conforming transactions for 1000 blocks.
>>
>> The problem in making these rules into consensus rule (affecting
>> tx/block validity) is that some rules (in particular #3) may not be
>> wanted by everyone, as they effectively limit the possibilities of
>the
>> script language further. As it is ultimately only about protecting
>> senders who care about non-malleability, introducing a new
>transaction
>> version is a very neat way of accomplishing that. The new block
>> version number is only there to coordinate the rollout, and choosing
>> an automatic forking point.
>>
>> --
>> Pieter
>
>
>
>------------------------------------------------------------------------
>
>------------------------------------------------------------------------------
>Managing the Performance of Cloud-Based Applications
>Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>Read the Whitepaper.
>http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Bitcoin-development mailing list
>Bitcoin-development@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/bitcoin-development
-----BEGIN PGP SIGNATURE-----
Version: APG v1.0.9

iQFQBAEBCAA6BQJTBRjcMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhbuuCADHHZvCbWNR+hj3lq2u
Xjr8POSsMWk4XorvLftgXSzAzypr7n0BP7+fmz/v0J98XfeOHxf8NHB2VXzFMCzI
mstYyFC+gdsPf9eIMoN2S9EB9d4Lh1Y7Zv5BGqopuHCUIVMpzk2QDaFlLe+gW8Ai
p4Yv/jGib8ym1ahJ24nZ89l7Psa+uXDw8N2VX5PcyDNVRwzuXwa0h2Kix/gt8uJb
RV5Sj3duxUE6mOGN07j6lPu9VcrtD0ydvAO3DoEJqkBqjhbC33h05H96KPQKuGcg
5DOKXUV5ChW5CF3DH5HN/LdduLgbTevtLbkBhdLKo+z5GKaU7Qpc5i6dIeAKl3uA
KCQE
=DiAE
-----END PGP SIGNATURE-----

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

dOn Wed, Feb 19, 2014 at 12:49 PM, Peter Todd <pete@petertodd.org> wrote:
> While we might be able to get away with a retroactive change in meaning right now in the future that won't be so easy. There are lots if proposed applications for nLockTime-using protocols that depend on transactions (or parts of transactions) being possible to mine as is. Making existing transactions impossible to mine in the future will break those types of applications. We might as well use this as a learning experience for what a version bump would look like infrastructures wise.

For some reason it took me a couple reads to get this so I thought I'd
restate it in a more blunt form.

There may exist people today who have send funds to addresses,
authored nlocktime releases, and destroyed the key the funds are at
now in order to achieve a timelock.  This might be a foolish thing to
do, but it's the kind of thing that you have to worry about when
potentially breaking existing transactions.

(This kind of us is, fwiw, another example of why ANYONE_CAN_PAY is useful).

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com> wrote:
> I think that we could guarantee fewer incidents by making version 1 transactions unmalleable and then optionally introduce a version 3 that supported the malleability feature. That way most existing problematic implementations would be fixed and no doors were closed for people experimenting with other stuff - tx v 3 would probably then be called experimental transactions.

Just to be clear: this change is not directly intended to avoid
"incidents". It will take way too long to deploy this. Software should
deal with malleability. This is a longer-term solution intended to
provide non-malleability guarantees for clients that a) are upgraded
to use them  b) willing to restrict their functionality. As there are
several intended use cases for malleable transactions (the sighash
flags pretty directly are a way to signify what malleabilities are
*wanted*), this is not about outlawing malleability.

While we could right now make all these rules non-standard, and
schedule a soft fork in a year or so to make them illegal, it would
mean removing potential functionality that can only be re-enabled
through a hard fork. This is significantly harder, so we should think
about it very well in advance.

About new transaction and block versions: this allows implementing and
automatically scheduling a softfork without waiting for wallets to
upgrade. The non-DER signature change was discussed for over two
years, and implemented almost a year ago, and we still notice wallets
that don't support it. We can't expect every wallet to be instantly
modified (what about hardware wallets like the Trezor, for example?
they may not just be able to be upgraded). Nor is it necessary: if
your software only spends confirmed change, and tracks all debits
correctly, there is no need.

-- 
Pieter

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Natanael]

[-- Attachment #1: Type: text/plain, Size: 3356 bytes --]

Regarding chains of transactions intended to be published at once together,
wouldn't it be easier to add a "only-mine-with-child flag"?

That way the parent transactions aren't actually valid unless spent
together with the transaction that depends on it, and only the original
will have a child referencing it.

Then malleability is not an issue at all for transaction chains if you only
need to broadcast your full transaction chain once, and don't need to
extend it in two or more occasions, *after* broadcasting subchains to the
network, from the same set of pregenerated transactions.

If you need to broadcast pregenerated subchains separately, then you need
the last child in the chain to be non-malleable.

This would require all miners to start to respect it at once in order to
avoid forking the network.

- Sent from my phone
Den 19 feb 2014 22:13 skrev "Pieter Wuille" <pieter.wuille@gmail.com>:

> On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com>
> wrote:
> > I think that we could guarantee fewer incidents by making version 1
> transactions unmalleable and then optionally introduce a version 3 that
> supported the malleability feature. That way most existing problematic
> implementations would be fixed and no doors were closed for people
> experimenting with other stuff - tx v 3 would probably then be called
> experimental transactions.
>
> Just to be clear: this change is not directly intended to avoid
> "incidents". It will take way too long to deploy this. Software should
> deal with malleability. This is a longer-term solution intended to
> provide non-malleability guarantees for clients that a) are upgraded
> to use them  b) willing to restrict their functionality. As there are
> several intended use cases for malleable transactions (the sighash
> flags pretty directly are a way to signify what malleabilities are
> *wanted*), this is not about outlawing malleability.
>
> While we could right now make all these rules non-standard, and
> schedule a soft fork in a year or so to make them illegal, it would
> mean removing potential functionality that can only be re-enabled
> through a hard fork. This is significantly harder, so we should think
> about it very well in advance.
>
> About new transaction and block versions: this allows implementing and
> automatically scheduling a softfork without waiting for wallets to
> upgrade. The non-DER signature change was discussed for over two
> years, and implemented almost a year ago, and we still notice wallets
> that don't support it. We can't expect every wallet to be instantly
> modified (what about hardware wallets like the Trezor, for example?
> they may not just be able to be upgraded). Nor is it necessary: if
> your software only spends confirmed change, and tracks all debits
> correctly, there is no need.
>
> --
> Pieter
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4165 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Allen Piscitello]

[-- Attachment #1: Type: text/plain, Size: 4681 bytes --]

This is somewhat problematic in my use case since some parts need to be in
the chain earlier than others and have the same ID as expected.

https://bitcointalk.org/index.php?topic=260898.10

I haven't gone back to see if there are any ways around it, but the main
problem here is I need the Contract TX to be in the chain much earlier than
redeeming, but I need the refund transaction to be in the chain much
earlier.  Perhaps there are some tricks to pull off to get it to work, but
I haven't been working on this for a while so I'm a bit rusty in that area.

This might be helpful enough to help a lot of use cases, but shouldn't be
final.

-Allen

On Wed, Feb 19, 2014 at 6:22 PM, Natanael <natanael.l@gmail.com> wrote:

> Regarding chains of transactions intended to be published at once
> together, wouldn't it be easier to add a "only-mine-with-child flag"?
>
> That way the parent transactions aren't actually valid unless spent
> together with the transaction that depends on it, and only the original
> will have a child referencing it.
>
> Then malleability is not an issue at all for transaction chains if you
> only need to broadcast your full transaction chain once, and don't need to
> extend it in two or more occasions, *after* broadcasting subchains to the
> network, from the same set of pregenerated transactions.
>
> If you need to broadcast pregenerated subchains separately, then you need
> the last child in the chain to be non-malleable.
>
> This would require all miners to start to respect it at once in order to
> avoid forking the network.
>
> - Sent from my phone
> Den 19 feb 2014 22:13 skrev "Pieter Wuille" <pieter.wuille@gmail.com>:
>
> On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com>
>> wrote:
>> > I think that we could guarantee fewer incidents by making version 1
>> transactions unmalleable and then optionally introduce a version 3 that
>> supported the malleability feature. That way most existing problematic
>> implementations would be fixed and no doors were closed for people
>> experimenting with other stuff - tx v 3 would probably then be called
>> experimental transactions.
>>
>> Just to be clear: this change is not directly intended to avoid
>> "incidents". It will take way too long to deploy this. Software should
>> deal with malleability. This is a longer-term solution intended to
>> provide non-malleability guarantees for clients that a) are upgraded
>> to use them  b) willing to restrict their functionality. As there are
>> several intended use cases for malleable transactions (the sighash
>> flags pretty directly are a way to signify what malleabilities are
>> *wanted*), this is not about outlawing malleability.
>>
>> While we could right now make all these rules non-standard, and
>> schedule a soft fork in a year or so to make them illegal, it would
>> mean removing potential functionality that can only be re-enabled
>> through a hard fork. This is significantly harder, so we should think
>> about it very well in advance.
>>
>> About new transaction and block versions: this allows implementing and
>> automatically scheduling a softfork without waiting for wallets to
>> upgrade. The non-DER signature change was discussed for over two
>> years, and implemented almost a year ago, and we still notice wallets
>> that don't support it. We can't expect every wallet to be instantly
>> modified (what about hardware wallets like the Trezor, for example?
>> they may not just be able to be upgraded). Nor is it necessary: if
>> your software only spends confirmed change, and tracks all debits
>> correctly, there is no need.
>>
>> --
>> Pieter
>>
>>
>> ------------------------------------------------------------------------------
>> Managing the Performance of Cloud-Based Applications
>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> Read the Whitepaper.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 6555 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Natanael]

[-- Attachment #1: Type: text/plain, Size: 6046 bytes --]

You could pregenerate entire "trees" of alternative outcomes where you pick
one branch / chain to broadcast based on the real world events as they
happen.

But I see another problem regarding use of oracles, if you have a P2SH
address with 2-of-3 signatures or similar in the chain, amd some
transactions following it, then the oracle needs to pregenerate both
transactions for both outcomes in advance. But the oracle probably don't
want to actually share it in advance to any third party before the event
happened.

This can be solved if the oracle only shares the transaction hash in
advance and then hands out a Zero-knowledge proof of that transaction with
the given hash is following the agreed upon rules, so you can trust the
transaction chain anyway and still being able to pregenerate a full tree of
transactions.

And then the oracle will release one of the possible transactions after the
event in question has happened, so you can broadcast the chain of choice.

This unfortunately breaks down if the number of possible outcomes becomes
too many as you would need to both generate and store a tree of possible
outcomes that is massive.

- Sent from my phone
Den 20 feb 2014 02:29 skrev "Allen Piscitello" <allen.piscitello@gmail.com>:

> This is somewhat problematic in my use case since some parts need to be in
> the chain earlier than others and have the same ID as expected.
>
> https://bitcointalk.org/index.php?topic=260898.10
>
> I haven't gone back to see if there are any ways around it, but the main
> problem here is I need the Contract TX to be in the chain much earlier than
> redeeming, but I need the refund transaction to be in the chain much
> earlier.  Perhaps there are some tricks to pull off to get it to work, but
> I haven't been working on this for a while so I'm a bit rusty in that area.
>
> This might be helpful enough to help a lot of use cases, but shouldn't be
> final.
>
> -Allen
>
> On Wed, Feb 19, 2014 at 6:22 PM, Natanael <natanael.l@gmail.com> wrote:
>
>> Regarding chains of transactions intended to be published at once
>> together, wouldn't it be easier to add a "only-mine-with-child flag"?
>>
>> That way the parent transactions aren't actually valid unless spent
>> together with the transaction that depends on it, and only the original
>> will have a child referencing it.
>>
>> Then malleability is not an issue at all for transaction chains if you
>> only need to broadcast your full transaction chain once, and don't need to
>> extend it in two or more occasions, *after* broadcasting subchains to the
>> network, from the same set of pregenerated transactions.
>>
>> If you need to broadcast pregenerated subchains separately, then you need
>> the last child in the chain to be non-malleable.
>>
>> This would require all miners to start to respect it at once in order to
>> avoid forking the network.
>>
>> - Sent from my phone
>> Den 19 feb 2014 22:13 skrev "Pieter Wuille" <pieter.wuille@gmail.com>:
>>
>> On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com>
>>> wrote:
>>> > I think that we could guarantee fewer incidents by making version 1
>>> transactions unmalleable and then optionally introduce a version 3 that
>>> supported the malleability feature. That way most existing problematic
>>> implementations would be fixed and no doors were closed for people
>>> experimenting with other stuff - tx v 3 would probably then be called
>>> experimental transactions.
>>>
>>> Just to be clear: this change is not directly intended to avoid
>>> "incidents". It will take way too long to deploy this. Software should
>>> deal with malleability. This is a longer-term solution intended to
>>> provide non-malleability guarantees for clients that a) are upgraded
>>> to use them  b) willing to restrict their functionality. As there are
>>> several intended use cases for malleable transactions (the sighash
>>> flags pretty directly are a way to signify what malleabilities are
>>> *wanted*), this is not about outlawing malleability.
>>>
>>> While we could right now make all these rules non-standard, and
>>> schedule a soft fork in a year or so to make them illegal, it would
>>> mean removing potential functionality that can only be re-enabled
>>> through a hard fork. This is significantly harder, so we should think
>>> about it very well in advance.
>>>
>>> About new transaction and block versions: this allows implementing and
>>> automatically scheduling a softfork without waiting for wallets to
>>> upgrade. The non-DER signature change was discussed for over two
>>> years, and implemented almost a year ago, and we still notice wallets
>>> that don't support it. We can't expect every wallet to be instantly
>>> modified (what about hardware wallets like the Trezor, for example?
>>> they may not just be able to be upgraded). Nor is it necessary: if
>>> your software only spends confirmed change, and tracks all debits
>>> correctly, there is no need.
>>>
>>> --
>>> Pieter
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Managing the Performance of Cloud-Based Applications
>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>> Read the Whitepaper.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Managing the Performance of Cloud-Based Applications
>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> Read the Whitepaper.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 8204 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Michael Gronager]

[-- Attachment #1: Type: text/plain, Size: 4187 bytes --]

As I see the BIP it is basically stressing that ver 1 transactions are malleable.

It then addresses the need for unmalleable transactions for e.g. spending unconfirmed outputs in a deterministic way (i.e. no 3rd party can sabotage) - this transaction type is defined as ver 3.

A lot of clients today spend unconfirmed outputs (even bitcoin-qt) and as such make an implicit assumption that this is kind of safe, which it is not - it can be intervened and sabotaged through tx malleability.

What I suggested was to ensure that a subclass of version 1 transactions become unmalleable - namely those with sighash=all. Note that only the sender can modify the sighash as it is part of the hash signed. So instead of defining a version 3, we could constrain version 1 txes with sighash=all to have a unmalleable hash. If you e.g. would like to still have a sighash=all type of transaction with malleable features you can simply use that sighash=all today is checked for using sighash&0x1f=sighash_all, so just OR'ing with 0x20 or 0x40 will get you the 'old' feature.

I do however buy the argument of Peter and Gregory that there might exist unpublished transactions out there that does not even conform to the DER rules etc, and as such we cannot forbid them from being mined, nor can we timestamp them and include 'only the old ones'. Hence we cannot change the consensus rule for version 1 transactions - and only changing the relay rules will not provide a certain guarantee.

So, I think the two line argument for the BIP is as follows:
1. We cannot change the consensus rules for version 1 transactions as that might invalidate unpublished non-standard transactions (= voiding peoples money, which is a line we don't want to cross)
2. The prime usecase for unmalleable transactions is being able to spend unconfirmed outputs - this is done today by almost all clients, but it is really broken. Hence a need for a fix asap.

I am all in favor for the BIP, but I expect the realistic timeline for enforced version 3 transactions is roughly one year, which is better than two, but it would have been nice to get it faster...

/M


On Feb 19, 2014, at 10:11 PM, Pieter Wuille <pieter.wuille@gmail.com> wrote:

> On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com> wrote:
>> I think that we could guarantee fewer incidents by making version 1 transactions unmalleable and then optionally introduce a version 3 that supported the malleability feature. That way most existing problematic implementations would be fixed and no doors were closed for people experimenting with other stuff - tx v 3 would probably then be called experimental transactions.
> 
> Just to be clear: this change is not directly intended to avoid
> "incidents". It will take way too long to deploy this. Software should
> deal with malleability. This is a longer-term solution intended to
> provide non-malleability guarantees for clients that a) are upgraded
> to use them  b) willing to restrict their functionality. As there are
> several intended use cases for malleable transactions (the sighash
> flags pretty directly are a way to signify what malleabilities are
> *wanted*), this is not about outlawing malleability.
> 
> While we could right now make all these rules non-standard, and
> schedule a soft fork in a year or so to make them illegal, it would
> mean removing potential functionality that can only be re-enabled
> through a hard fork. This is significantly harder, so we should think
> about it very well in advance.
> 
> About new transaction and block versions: this allows implementing and
> automatically scheduling a softfork without waiting for wallets to
> upgrade. The non-DER signature change was discussed for over two
> years, and implemented almost a year ago, and we still notice wallets
> that don't support it. We can't expect every wallet to be instantly
> modified (what about hardware wallets like the Trezor, for example?
> they may not just be able to be upgraded). Nor is it necessary: if
> your software only spends confirmed change, and tracks all debits
> correctly, there is no need.
> 
> -- 
> Pieter


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 5009 bytes --]

We've done forking changes before much faster than a year and that was with
less experience. If we want, we can get this done within months.
On 20 Feb 2014 16:30, "Michael Gronager" <gronager@mac.com> wrote:

> As I see the BIP it is basically stressing that ver 1 transactions are
> malleable.
>
> It then addresses the need for unmalleable transactions for e.g. spending
> unconfirmed outputs in a deterministic way (i.e. no 3rd party can sabotage)
> - this transaction type is defined as ver 3.
>
> A lot of clients today spend unconfirmed outputs (even bitcoin-qt) and as
> such make an implicit assumption that this is kind of safe, which it is not
> - it can be intervened and sabotaged through tx malleability.
>
> What I suggested was to ensure that a subclass of version 1 transactions
> become unmalleable - namely those with sighash=all. Note that only the
> sender can modify the sighash as it is part of the hash signed. So instead
> of defining a version 3, we could constrain version 1 txes with sighash=all
> to have a unmalleable hash. If you e.g. would like to still have a
> sighash=all type of transaction with malleable features you can simply use
> that sighash=all today is checked for using sighash&0x1f=sighash_all, so
> just OR'ing with 0x20 or 0x40 will get you the 'old' feature.
>
> I do however buy the argument of Peter and Gregory that there might exist
> unpublished transactions out there that does not even conform to the DER
> rules etc, and as such we cannot forbid them from being mined, nor can we
> timestamp them and include 'only the old ones'. Hence we cannot change the
> consensus rule for version 1 transactions - and only changing the relay
> rules will not provide a certain guarantee.
>
> So, I think the two line argument for the BIP is as follows:
> 1. We cannot change the consensus rules for version 1 transactions as that
> might invalidate unpublished non-standard transactions (= voiding peoples
> money, which is a line we don't want to cross)
> 2. The prime usecase for unmalleable transactions is being able to spend
> unconfirmed outputs - this is done today by almost all clients, but it is
> really broken. Hence a need for a fix asap.
>
> I am all in favor for the BIP, but I expect the realistic timeline for
> enforced version 3 transactions is roughly one year, which is better than
> two, but it would have been nice to get it faster...
>
> /M
>
>
> On Feb 19, 2014, at 10:11 PM, Pieter Wuille <pieter.wuille@gmail.com>
> wrote:
>
> > On Wed, Feb 19, 2014 at 9:28 PM, Michael Gronager <gronager@mac.com>
> wrote:
> >> I think that we could guarantee fewer incidents by making version 1
> transactions unmalleable and then optionally introduce a version 3 that
> supported the malleability feature. That way most existing problematic
> implementations would be fixed and no doors were closed for people
> experimenting with other stuff - tx v 3 would probably then be called
> experimental transactions.
> >
> > Just to be clear: this change is not directly intended to avoid
> > "incidents". It will take way too long to deploy this. Software should
> > deal with malleability. This is a longer-term solution intended to
> > provide non-malleability guarantees for clients that a) are upgraded
> > to use them  b) willing to restrict their functionality. As there are
> > several intended use cases for malleable transactions (the sighash
> > flags pretty directly are a way to signify what malleabilities are
> > *wanted*), this is not about outlawing malleability.
> >
> > While we could right now make all these rules non-standard, and
> > schedule a soft fork in a year or so to make them illegal, it would
> > mean removing potential functionality that can only be re-enabled
> > through a hard fork. This is significantly harder, so we should think
> > about it very well in advance.
> >
> > About new transaction and block versions: this allows implementing and
> > automatically scheduling a softfork without waiting for wallets to
> > upgrade. The non-DER signature change was discussed for over two
> > years, and implemented almost a year ago, and we still notice wallets
> > that don't support it. We can't expect every wallet to be instantly
> > modified (what about hardware wallets like the Trezor, for example?
> > they may not just be able to be upgraded). Nor is it necessary: if
> > your software only spends confirmed change, and tracks all debits
> > correctly, there is no need.
> >
> > --
> > Pieter
>
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 5902 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Thu, Feb 20, 2014 at 6:08 AM, Mike Hearn <mike@plan99.net> wrote:
> We've done forking changes before much faster than a year and that was with
> less experience. If we want, we can get this done within months.

You mean P2SH... which your implementation has only picked up support
for in the last month or so?

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

I think we should get Pieter's proposal done and implemented quickly. I
agree with Mike, it doesn't have to take a long time for the core network
to fully support this.

Getting wallets to start generating transaction.version=3 might take years,
but that is OK.

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 397 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Thu, Feb 20, 2014 at 6:29 AM, Gavin Andresen <gavinandresen@gmail.com> wrote:
> I think we should get Pieter's proposal done and implemented quickly. I
> agree with Mike, it doesn't have to take a long time for the core network to
> fully support this.
>
> Getting wallets to start generating transaction.version=3 might take years,
> but that is OK.

Sure I'm all for doing what Pieter suggested— it's basically the plan
we've been executing for some time already but with the version check
to make it sane to complete.

My reserved sounding comments were relative to the proposals to do
things with nversion=1 transactions, frankly I think thats completely
insane. Though while we're on the subject of reservations, I am far
from confident that we've uncovered all the possible malleability
routes— that list gained a new, never before discussed entry, when
Pieter was writing it a couple weeks ago.  We also have no proof of
the absence of further algebraic malleability in DSA (though I think
its somewhat unlikely, a solid proof of it has been somewhat elusive).

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

Great, I'm hearing rough consensus to proceed with Pieter's plan.

RE: far from confident on malleability routes:  I'm reasonably confident
that we can squash malleability for IsStandard, SIGHASH_ALL transactions. A
proper proof of DSA signature un-malleability (or an lower bound for how
much work it would be to create a valid doppleganger signature) would be
great, but I don't think it is necessary to proceed.


On Thu, Feb 20, 2014 at 9:36 AM, Gregory Maxwell <gmaxwell@gmail.com> wrote:

> On Thu, Feb 20, 2014 at 6:29 AM, Gavin Andresen <gavinandresen@gmail.com>
> wrote:
> > I think we should get Pieter's proposal done and implemented quickly. I
> > agree with Mike, it doesn't have to take a long time for the core
> network to
> > fully support this.
> >
> > Getting wallets to start generating transaction.version=3 might take
> years,
> > but that is OK.
>
> Sure I'm all for doing what Pieter suggested-- it's basically the plan
> we've been executing for some time already but with the version check
> to make it sane to complete.
>
> My reserved sounding comments were relative to the proposals to do
> things with nversion=1 transactions, frankly I think thats completely
> insane. Though while we're on the subject of reservations, I am far
> from confident that we've uncovered all the possible malleability
> routes-- that list gained a new, never before discussed entry, when
> Pieter was writing it a couple weeks ago.  We also have no proof of
> the absence of further algebraic malleability in DSA (though I think
> its somewhat unlikely, a solid proof of it has been somewhat elusive).
>



-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 2239 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Pieter Wuille]

I hereby request a BIP number.


On Thu, Feb 20, 2014 at 3:58 PM, Gavin Andresen <gavinandresen@gmail.com> wrote:
> Great, I'm hearing rough consensus to proceed with Pieter's plan.
>
> RE: far from confident on malleability routes:  I'm reasonably confident
> that we can squash malleability for IsStandard, SIGHASH_ALL transactions. A
> proper proof of DSA signature un-malleability (or an lower bound for how
> much work it would be to create a valid doppleganger signature) would be
> great, but I don't think it is necessary to proceed.
>
>
> On Thu, Feb 20, 2014 at 9:36 AM, Gregory Maxwell <gmaxwell@gmail.com> wrote:
>>
>> On Thu, Feb 20, 2014 at 6:29 AM, Gavin Andresen <gavinandresen@gmail.com>
>> wrote:
>> > I think we should get Pieter's proposal done and implemented quickly. I
>> > agree with Mike, it doesn't have to take a long time for the core
>> > network to
>> > fully support this.
>> >
>> > Getting wallets to start generating transaction.version=3 might take
>> > years,
>> > but that is OK.
>>
>> Sure I'm all for doing what Pieter suggested-- it's basically the plan
>> we've been executing for some time already but with the version check
>> to make it sane to complete.
>>
>> My reserved sounding comments were relative to the proposals to do
>> things with nversion=1 transactions, frankly I think thats completely
>> insane. Though while we're on the subject of reservations, I am far
>> from confident that we've uncovered all the possible malleability
>> routes-- that list gained a new, never before discussed entry, when
>> Pieter was writing it a couple weeks ago.  We also have no proof of
>> the absence of further algebraic malleability in DSA (though I think
>> its somewhat unlikely, a solid proof of it has been somewhat elusive).
>
>
>
>
> --
> --
> Gavin Andresen
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Thu, Feb 20, 2014 at 7:11 AM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
> I hereby request a BIP number.

62 assigned.

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

No, I was thinking of the height in coinbase change. At any rate, p2sh was
supported by the consensus code in bitcoinj for a long time already, since
it was first written.

Support for sending to such addresses in the wallet appeared once an app
that wanted that support also appeared, which seems OK - the market for
wallets is very competitive so there will always be some skew in what
features are worked on in what order. V3 transactions are a consensus
change that wallets will pick up at different times like any other feature.
On 20 Feb 2014 19:45, "Gregory Maxwell" <gmaxwell@gmail.com> wrote:

> On Thu, Feb 20, 2014 at 6:08 AM, Mike Hearn <mike@plan99.net> wrote:
> > We've done forking changes before much faster than a year and that was
> with
> > less experience. If we want, we can get this done within months.
>
> You mean P2SH... which your implementation has only picked up support
> for in the last month or so?
>

[-- Attachment #2: Type: text/html, Size: 1264 bytes --]

Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability

[Gregory Maxwell]

On Thu, Feb 20, 2014 at 10:07 PM, Mike Hearn <mike@plan99.net> wrote:
> No, I was thinking of the height in coinbase change. At any rate, p2sh was
> supported by the consensus code in bitcoinj for a long time already, since
> it was first written.
>
> Support for sending to such addresses in the wallet appeared once an app
> that wanted that support also appeared, which seems OK - the market for
> wallets is very competitive so there will always be some skew in what
> features are worked on in what order. V3 transactions are a consensus change
> that wallets will pick up at different times like any other feature.

We're in agreement.  I had mistakenly believed you were supporting the
discussion about trying to force these constraints on current version
transactions, in which case "wallets will pick up at different times"
is an absolute deal breaker.  :)