[Bitcoin-development] An initial replace-by-fee implementation is now available

[Bitcoin-development] An initial replace-by-fee implementation is now available

[John Dillon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

After some consultation with affected sites by myself and Peter we have decided
to release an initial replace-by-fee implementation and setup a server using
those rules on testnet. This implementation does not include recursive fee
evaluation, and is therefore vulnerable to DoS attack, so hopefully that will
continue to allow adoption to proceed gradually. We can-not recommend mining on
mainnet with it. It does not include an "undo" RPC command or an adjust fees,
and Peter says he has not implemented one yet.  Patches are welcome.

Specifically there were requests from vulnerable parties, which interestingly
included a site that knew they had bugs related to replacement but not
financial vulnerabilities, to put up a server on testnet to check wallet code.
The vulnerable requested to remain undisclosed. An additional consideration was
the upcoming anti-dust rules which are yet another example of why zero-conf is
so much more dangerous to accept than single-conf. Two of the people contacting
us brought up that issue in fact.

The code is on github:

    https://github.com/petertodd/bitcoin/tree/replace-by-fee

and a replace-by-fee server operating on testnet is available at
testnet-replace-by-fee.bitcoin.petertodd.org To test you will need to use the
raw transaction API and manually create the replacement transaction. Do note
that your wallet will retain the existing one and no mechanism yet exists to
delete the old transaction from your wallet. Again, a certain amount of
"cludgyness" to this is intentional to discourage premature non-testing use.


Regarding the reward, I've decided Peter will collect the full amount even
though the work is not %100 complete (the mempool aspect) due to his concern
about staging an implementation properly, working with vulnerable sites, and
overall genuine interest in the actual issues at hand rather than the reward.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJRi3LeAAoJEEWCsU4mNhiPwscH/2CI0d3h/3jix3iyz2I9I8Sz
6nbP8eA01l9kzG37cH1rFAbt7C+fL/nardV4U1qmiwC0MN7NPpX6BFn5eQ2PUKbu
41+AnjgWicB2tnCC07ngboQ1JCeZ+RTfATepuMxEdWFBsc8ZQXs0apWS01FT+TDq
J/a7QkhNfTaAQzXyqmLp0TQO7/Z7ysmCftOhtGbfvfhF2o23BuphQiRVA9IOoUuj
Fgb5wrfQqJ8TjvXRXAUQ7SUlzfN9BlPxMkTc6NhbcgIpuq1Kb43mLoDl3s2irH4A
GtjRtobV5Cfozm1r+8KPtIYEoQoj0PqTjO5YMwD/vTaRfNzdS4Tse5LQLGT6Jug=
=M1mj
-----END PGP SIGNATURE-----

Re: [Bitcoin-development] An initial replace-by-fee implementation is now available

[Adam Back]

In this thread discussing this idea

https://bitcointalk.org/index.php?topic=179612.0 

it is suggested that the approach risks making 0-confirm double-spends
easier.

I dont see why this should be.  Cant part of the validation of accepting a
fee revision be that every aspct of the revision except the reward must be
unchanged, otherwise the revision is considered invalid and discarded?

(ie same payment amount, same input coins, same recipient and same change
address.)

Adam

On Thu, May 09, 2013 at 09:58:50AM +0000, John Dillon wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>After some consultation with affected sites by myself and Peter we have decided
>to release an initial replace-by-fee implementation and setup a server using
>those rules on testnet. This implementation does not include recursive fee
>evaluation, and is therefore vulnerable to DoS attack, so hopefully that will
>continue to allow adoption to proceed gradually. We can-not recommend mining on
>mainnet with it. It does not include an "undo" RPC command or an adjust fees,
>and Peter says he has not implemented one yet.  Patches are welcome.
>
>Specifically there were requests from vulnerable parties, which interestingly
>included a site that knew they had bugs related to replacement but not
>financial vulnerabilities, to put up a server on testnet to check wallet code.
>The vulnerable requested to remain undisclosed. An additional consideration was
>the upcoming anti-dust rules which are yet another example of why zero-conf is
>so much more dangerous to accept than single-conf. Two of the people contacting
>us brought up that issue in fact.
>
>The code is on github:
>
>    https://github.com/petertodd/bitcoin/tree/replace-by-fee
>
>and a replace-by-fee server operating on testnet is available at
>testnet-replace-by-fee.bitcoin.petertodd.org To test you will need to use the
>raw transaction API and manually create the replacement transaction. Do note
>that your wallet will retain the existing one and no mechanism yet exists to
>delete the old transaction from your wallet. Again, a certain amount of
>"cludgyness" to this is intentional to discourage premature non-testing use.
>
>
>Regarding the reward, I've decided Peter will collect the full amount even
>though the work is not %100 complete (the mempool aspect) due to his concern
>about staging an implementation properly, working with vulnerable sites, and
>overall genuine interest in the actual issues at hand rather than the reward.
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.11 (GNU/Linux)
>
>iQEcBAEBCAAGBQJRi3LeAAoJEEWCsU4mNhiPwscH/2CI0d3h/3jix3iyz2I9I8Sz
>6nbP8eA01l9kzG37cH1rFAbt7C+fL/nardV4U1qmiwC0MN7NPpX6BFn5eQ2PUKbu
>41+AnjgWicB2tnCC07ngboQ1JCeZ+RTfATepuMxEdWFBsc8ZQXs0apWS01FT+TDq
>J/a7QkhNfTaAQzXyqmLp0TQO7/Z7ysmCftOhtGbfvfhF2o23BuphQiRVA9IOoUuj
>Fgb5wrfQqJ8TjvXRXAUQ7SUlzfN9BlPxMkTc6NhbcgIpuq1Kb43mLoDl3s2irH4A
>GtjRtobV5Cfozm1r+8KPtIYEoQoj0PqTjO5YMwD/vTaRfNzdS4Tse5LQLGT6Jug=
>=M1mj
>-----END PGP SIGNATURE-----
>
>------------------------------------------------------------------------------
>Learn Graph Databases - Download FREE O'Reilly Book
>"Graph Databases" is the definitive new guide to graph databases and
>their applications. This 200-page book is written by three acclaimed
>leaders in the field. The early access version is available now.
>Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
>_______________________________________________
>Bitcoin-development mailing list
>Bitcoin-development@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] An initial replace-by-fee implementation is now available

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2043 bytes --]

On Thu, May 09, 2013 at 01:19:13PM +0200, Adam Back wrote:
> In this thread discussing this idea
> 
> https://bitcointalk.org/index.php?topic=179612.0 
> 
> it is suggested that the approach risks making 0-confirm double-spends
> easier.

The patch makes the concept of a 0-confirm double-spend obsolete
basically. The model is rather than having some vague, insecure, easily
broken, de-facto no-replacement rule it replaces it with something very
easy to reason about: you are bidding for blockchain space, and you can
adjust your bid after the fact.

The reality is zero-conf double-spends aren't that big of a problem
because the vast majority of payments have other mechanisms they can use
instead of relying on the defacto behavior of dozens of major miners and
nodes.

Long story short, we're better off if we don't give people a false sense
of security.

> I dont see why this should be.  Cant part of the validation of accepting a
> fee revision be that every aspct of the revision except the reward must be
> unchanged, otherwise the revision is considered invalid and discarded?

A node has no idea which transaction output is change and which one
isn't; if nodes could distinguish change from payment your privacy would
be badly violated.

By allowing simple replacement without further rules the fee adjustment
process can go on as long as required, without you running out of
additional transaction inputs and without causing the transaction to get
bigger and bigger each time.

It also allows more interesting use cases, like adding additional
outputs to a transaction after the fact as more payees become known, or
if two unrelated parties decide to combine their transactions to save on
blockchain space and preserve their privacy.

Eventually the P2P protocol can have delta compression support, so the
network bandwidth required to merge two transactions into one will be
minimal.

-- 
'peter'[:-1]@petertodd.org
000000000000014c26728a13e351b4dd7a32e99e28d43a960b5ac5f98696ae5b

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] An initial replace-by-fee implementation is now available

[Adam Back]

I have to say I do not think you want to have it be random as to who gets
paid (by having conflicting double spends floating around with different
payee & change addresses all from the same sending address.)  

About current defacto no replacement: it is the best bitcoin currently has,
and it has value, with it you need to do a net-split to attack eg
1-confirmation, and this proposal weakens it.  Net-splits are possible but
not trivial.  This proposal moves them into spec - ie free.

About privacy: I think you are going to inherently disclose which is the
change address, because you will decrease the change when you increase the
fee.  There is no coin management in the client, and as far as I saw so far,
no privacy management of which coins to reduce coin cross linking.  Who's to
say the client has 100s of times as many coins as the payment.

If people dont want to reveal which is change and which payment, they need
to put a big enough fee up front based on a margin over prevailing fee
statistics.

It would also be better to try to get the fee right first time than to create
more traffic revising it due to experience.  Though the ability to revise
the fee IFF the best effort fee doesnt work empirically after a couple of
blocks seems like a good feature.  (But not with revised recipient/change
addresses.

Also if the bids are too flexibly different how do you stop both bids being
processed (eg one in a block, the next in the next block).

Adam

On Thu, May 09, 2013 at 07:46:05AM -0400, Peter Todd wrote:
>On Thu, May 09, 2013 at 01:19:13PM +0200, Adam Back wrote:
>> In this thread discussing this idea
>>
>> https://bitcointalk.org/index.php?topic=179612.0
>>
>> it is suggested that the approach risks making 0-confirm double-spends
>> easier.
>
>The patch makes the concept of a 0-confirm double-spend obsolete
>basically. The model is rather than having some vague, insecure, easily
>broken, de-facto no-replacement rule it replaces it with something very
>easy to reason about: you are bidding for blockchain space, and you can
>adjust your bid after the fact.
>
>The reality is zero-conf double-spends aren't that big of a problem
>because the vast majority of payments have other mechanisms they can use
>instead of relying on the defacto behavior of dozens of major miners and
>nodes.
>
>Long story short, we're better off if we don't give people a false sense
>of security.
>
>> I dont see why this should be.  Cant part of the validation of accepting a
>> fee revision be that every aspct of the revision except the reward must be
>> unchanged, otherwise the revision is considered invalid and discarded?
>
>A node has no idea which transaction output is change and which one
>isn't; if nodes could distinguish change from payment your privacy would
>be badly violated.
>
>By allowing simple replacement without further rules the fee adjustment
>process can go on as long as required, without you running out of
>additional transaction inputs and without causing the transaction to get
>bigger and bigger each time.
>
>It also allows more interesting use cases, like adding additional
>outputs to a transaction after the fact as more payees become known, or
>if two unrelated parties decide to combine their transactions to save on
>blockchain space and preserve their privacy.
>
>Eventually the P2P protocol can have delta compression support, so the
>network bandwidth required to merge two transactions into one will be
>minimal.
>
>-- 
>'peter'[:-1]@petertodd.org
>000000000000014c26728a13e351b4dd7a32e99e28d43a960b5ac5f98696ae5b

Re: [Bitcoin-development] An initial replace-by-fee implementation is now available

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

On Thu, May 09, 2013 at 02:07:23PM +0200, Adam Back wrote:
> I have to say I do not think you want to have it be random as to who gets
> paid (by having conflicting double spends floating around with different
> payee & change addresses all from the same sending address.)

Indeed. That's the point of the blockchain, to take all those potential
inconsistencies and vote on a true transaction history to achieve
consensus.

> About current defacto no replacement: it is the best bitcoin currently has,
> and it has value, with it you need to do a net-split to attack eg
> 1-confirmation, and this proposal weakens it.  Net-splits are possible but
> not trivial.  This proposal moves them into spec - ie free.

Right now we don't have double-spend proof propagation, so the
"net-split" attack is actually totally trivial: just broadcast two
different, mutually incompatible, transactions at the same time. About
half the time the recipient will get the payment, the other half of the
time the payment they thought they were going to get is invalidated.

It's very, very rare for sites to have protection against that;
blockchain.info's shared-send mixer is one of the few exceptions. But
the have access to a whole network monitoring service with connections
to nodes all over the planet.

> About privacy: I think you are going to inherently disclose which is the
> change address, because you will decrease the change when you increase the
> fee.  There is no coin management in the client, and as far as I saw so far,
> no privacy management of which coins to reduce coin cross linking.  Who's to
> say the client has 100s of times as many coins as the payment.
>
> If people dont want to reveal which is change and which payment, they need
> to put a big enough fee up front based on a margin over prevailing fee
> statistics.

It's not ideal, but it still protects against after-the-fact blockchain
analysis.

Statistics is hard - you can't get it right all the time. Besides, what
happens when everyone adds a safety margin? Some people can afford to
wait, so for them starting at a low bid and raises it makes a lot of
sense.

> Also if the bids are too flexibly different how do you stop both bids being
> processed (eg one in a block, the next in the next block).

Think about that problem a bit harder. :)

-- 
'peter'[:-1]@petertodd.org
00000000000000220dc3b283e651068535f8e934096cfef35342bc688d8ee578

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]