[Bitcoin-development] Floating fees and SPV clients

[Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2767 bytes --]

Lately I was pondering how to make floating fees and SPV wallets work well
together.

I propose the following plan:

1) 0.9 ships with something dead simple, like a command to query what a
node estimates and then clients just take the average, or cross-check a
centralised estimate against the P2P network. It's fast to implement and
simple, but not very secure or decentralised. However it will allow the
feature to launch on some kind of reasonable timeframe.

2) We bump the protocol version and the tx message now gets an optional
protobuf buffer stuck on the end. The first thing put in this protobuf is a
list of the values of the inputs. Using this data, the fee paid by a
transaction can be calculated. In step 2 the data is unauthenticated.

3) Some SPV wallets already set themselves up so that they sync with the
network in the background, e.g. the Android wallet syncs at least every 24
hours. This should become more common, using scheduler capabilities built
into most operating systems. When the wallet syncs with the network, it
sets a deliberately very noisy Bloom filter on its peers and waits around
for 30-60 seconds or so. The wallet observes some of the broadcasts taking
place and records the hashes and associated fees that were paid to disk.
Next time it syncs, it includes the observed hashes into the Bloom filter
used to download the chain, and thus learns how quickly they confirmed. It
can calculate its own fee estimate from that.

4) Finally, when we next hard fork, we make v2 transactions include the
output value in the signature, same as the output script (this proposal has
been on the forums for a while now). That allows the fee data added in step
2 to be cross-checked against the signatures on the inputs, thus
authenticating it.


I think this is a small and easy set of steps that would make it quite hard
to attack - malicious nodes could make it appear that some transactions
never confirmed thus seeming to force the price up, but it's easy to simply
exclude transactions which never confirm at all from the calculations. Plus
of course you can cross-check nodes against each other to try and catch
nodes that are failing to match transactions properly.

One obvious concern is what to do if nodes don't converge on very similar
estimates. Wallets will always want to pay the lowest fee possible, so that
means they'll always be riding the very edge of what's acceptable, opening
up tx propagation to random flaky failures if fee estimates change whilst a
transaction is in progress, or if some nodes don't calculate the same
estimates as others.

If a wallet gets a reject message for a tx that has a fee that are by its
own estimates acceptable, what should it do? What if only some nodes report
that and others don't?

[-- Attachment #2: Type: text/html, Size: 3067 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Andreas Schildbach]

On 12/01/2013 12:51 PM, Mike Hearn wrote:

> I propose the following plan:

Thanks taking the initiative and writing this up!

> If a wallet gets a reject message for a tx that has a fee that are by
> its own estimates acceptable, what should it do? What if only some nodes
> report that and others don't?

As long as the tx is not confirmed (by a broadcast), apps can offer to
bump up the fee a little bit.

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2727 bytes --]

> As long as the tx is not confirmed (by a broadcast), apps can offer to
> bump up the fee a little bit.

Unfortunately there are risks to that approach. 

The most obvious one is that nodes could keep sending reject messages to get wallets to attach ridiculously high fees. If half a wallets peers do this and the other half don’t, then effectively the wallet will double spend against itself. The bad nodes can keep the fat transaction and send it directly to a corrupt miner, no broadcast. If some other miner includes the original normal transaction, no problem, just take it out of the current block. If the corrupt miner finds the current block, they get to claim huge fee premiums.

Quite apart from the problem of malicious nodes/miners, how would you represent this in the wallet GUI? Current wallets are designed on the assumption that 1 payment == 1 transaction == 1 paid fee. If a single payment could have several different fees, and there’s no way to know which you will actually pay until later, then complexity would explode. Even the notion of balance would become even more complicated than it already is.

So I really don’t like the idea of creating different transactions depending on error messages from remote nodes. The only time when it could make sense is if *all* nodes reject a transaction. Then (assuming no MITM) you can assume the first transaction can be thrown away and a new attempt made.

But if you think about what the UI flows for that would look like - it’s just a mess.

There are other risks to fee estimation. Let’s say wallet authors create transactions with exactly the estimated fee needed to get into the next block. But due to mempool skew, estimates vary, and so those transactions don’t propagate cleanly everywhere. Now we have two problems:

1) Unpredictable failure to enter the mempools can lead to double spending and slow confirmations

2) Wallet authors may be tempted to ensure that doesn’t happen by taking the estimate, adding 10% and using that. But then if a bunch of popular wallets all do the same thing, the estimation algorithm might get confused and decide that as everyone seems to be attaching a fee of X+10%, the correct estimate for what fee to attach is X+10%. Then wallets would immediate raise their attached fees again and you’d enter into a infinite upward spiral.

The more I think about this, the more complicated it gets.

It’s tempting to try and just push all the complexity onto the merchant side, but one of the best things about Bitcoin is there isn’t any strong notion of “merchant” - that’s inherent to being peer to peer. So just hand-waving and saying sellers will deal with complicated fee processes is just a punt.


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4127 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Andreas Schildbach]

(my post hasn't shown up for an hour, so I'm sending it again)


On 12/01/2013 02:41 PM, Mike Hearn wrote:

>> As long as the tx is not confirmed (by a broadcast), apps can offer to
>> bump up the fee a little bit.
>
> Unfortunately there are risks to that approach.

I assume you're right, since I do not have so much experience with game
theory.

About the UI:

Generally, for pending tx I'd like to measure time they're not being
broadcast-confirmed and count blocks that they missed being included.
Both can be combined into adapting the current generic messages ("This
payment should become spendable shortly" for incoming and "This payment
has not been transmitted yet" for outgoing transactions). Hint:
Statistics could be offered by bitcoinj.

For outgoing transactions, if it is very clear that they're never going
to be confirmed, I'd like to see a "Revoke" button. This would have
saved us some support hassles with the transmit bugs. It could also
offer a "Top up fee" button, which would replace the tx by a new one.
I'm aware about a possible double spend but who cares? It doesn't matter
which of the two transactions gets into the chain, as long as not both
will be included.

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]

> Both can be combined into adapting the current generic messages ("This
> payment should become spendable shortly" for incoming and "This payment
> has not been transmitted yet" for outgoing transactions). 

What would the new messages say?

We need to get away from the notion of senders attaching fees anyway. This is the wrong way around because it’s the recipient who cares about double spending risk, not the sender. That’s why merchants keep running into issues with people attaching zero fees. Of course they attach zero fees. They know they aren’t going to double spend. It’s the merchant who cares about getting the security against that.

The UI for sending money should end up dead simple - no mention of fees anywhere, IMO.

The UI for receiving money could be a bit more complicated but even then - I think if ordinary people using smartphone wallets are having to think about how quickly they want their transaction to confirm and adjust fees, etc on the receiving side then we’re getting dangerously close to the usability failure zone.

Unfortunately we lack the protocol pieces to get the right UI here :( Someone needs to sit down and figure out what the UI *should* look like, in the ideal world, and then work backwards to figure out what needs to be done to get us there.

> For outgoing transactions, if it is very clear that they're never going
> to be confirmed, I'd like to see a "Revoke" button.

Disagree. There should never be any cases in which a transaction doesn’t confirm. Period. I know there have been bugs with bitcoinj that could cause this in the past, but they were bugs and they got fixed/will get fixed.

Settlement failure is just unacceptable and building a UI around the possibility will just encourage people to think of it as normal, when it should not be so.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4127 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Andreas Schildbach]

On 12/01/2013 06:19 PM, Mike Hearn wrote:
>> Both can be combined into adapting the current generic messages ("This
>> payment should become spendable shortly" for incoming and "This payment
>> has not been transmitted yet" for outgoing transactions).
>
> What would the new messages say?

Well, for starters I'd suggest something like

"This payment did not become spendable since xxx minutes. Check with the
sender if s/he paid the Bitcoin network fee. Check if your internet
connection is working properly." (incoming)

"This payment still has not been transmitted. Check if your internet
connection is working properly." (outgoing)

> We need to get away from the notion of senders attaching fees anyway.
This is the wrong way around because it’s the recipient who cares about
double spending risk, not the sender. That’s why merchants keep running
into issues with people attaching zero fees. Of course they attach zero
fees. They know they aren’t going to double spend. It’s the merchant who
cares about getting the security against that.

Guess you're right. But as you said, we're not there yet.

> The UI for sending money should end up dead simple - no mention of
fees anywhere, IMO.

Agreed, if the sender does not need to pay a fee any more. On the
receiving side it of course needs to be mentioned. (Or the other way
round, as of today.)

> Unfortunately we lack the protocol pieces to get the right UI here :(
Someone needs to sit down and figure out what the UI *should* look like,
in the ideal world, and then work backwards to figure out what needs to
be done to get us there.

(The ideal world doesn't need a UI for money.)

>> For outgoing transactions, if it is very clear that they're never going
>> to be confirmed, I'd like to see a "Revoke" button.
>
> Disagree. There should never be any cases in which a transaction
doesn’t confirm. Period. I know there have been bugs with bitcoinj that
could cause this in the past, but they were bugs and they got fixed/will
get fixed.
>
> Settlement failure is just unacceptable and building a UI around the
possibility will just encourage people to think of it as normal, when it
should not be so.

I fully understand your point of view. However, its not the reality
currently. (Hopefully it is, after the fixes to bitcoinj.)

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]


> "This payment did not become spendable since xxx minutes. Check with the
> sender if s/he paid the Bitcoin network fee. Check if your internet
> connection is working properly." (incoming)

That seems reasonable.

The other message should be implementable today, I think? If numBroadcastPeers > 0 post 0.10.3 then you know the tx made it out to the internet.

Unfortunately if nodes start to diverge a lot in terms of what they will accept, then “transmitted” is no longer a clean binary yes/no thing. Guess we’ll have to jump that hurdle when we come to it.

> Guess you're right. But as you said, we're not there yet.

The payment protocol at least would need some notion of fee, or possibly (better?) the ability for a recipient to specify some inputs as well as some outputs.

Originally I think we were hoping for child-pays-for-parent. I guess that needs someone to sit down and focus on it for a while, assuming we still think that’s a good idea.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4127 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2798 bytes --]

On Sun, Dec 01, 2013 at 06:19:14PM +0100, Mike Hearn wrote:
> > Both can be combined into adapting the current generic messages ("This
> > payment should become spendable shortly" for incoming and "This payment
> > has not been transmitted yet" for outgoing transactions). 
> 
> What would the new messages say?
> 
> We need to get away from the notion of senders attaching fees anyway. This is the wrong way around because it’s the recipient who cares about double spending risk, not the sender. That’s why merchants keep running into issues with people attaching zero fees. Of course they attach zero fees. They know they aren’t going to double spend. It’s the merchant who cares about getting the security against that.
> 
> The UI for sending money should end up dead simple - no mention of fees anywhere, IMO.
> 
> The UI for receiving money could be a bit more complicated but even then - I think if ordinary people using smartphone wallets are having to think about how quickly they want their transaction to confirm and adjust fees, etc on the receiving side then we’re getting dangerously close to the usability failure zone.
> 
> Unfortunately we lack the protocol pieces to get the right UI here :( Someone needs to sit down and figure out what the UI *should* look like, in the ideal world, and then work backwards to figure out what needs to be done to get us there.
> 
> > For outgoing transactions, if it is very clear that they're never going
> > to be confirmed, I'd like to see a "Revoke" button.
> 
> Disagree. There should never be any cases in which a transaction doesn’t confirm. Period. I know there have been bugs with bitcoinj that could cause this in the past, but they were bugs and they got fixed/will get fixed.
> 
> Settlement failure is just unacceptable and building a UI around the possibility will just encourage people to think of it as normal, when it should not be so.

Bitcoin is and always will be limited in capacity - transactions may not
confirm in a reasonable about of time because of high-demand and/or DoS
attacks. Giving senders and/or receivers the ability to increase fees
after the fact is the only way you'll ever be able to deal with these
situations. Of course, in those situations revoke isn't going to be 100%
reliable until the txins get spent elsewhere, but that just indicates
the UI problem is around that kind of functionality is subtle.


re: merchants paying tx fees, child-pays-for-parent is inefficient, and
micropayments direct to miners isn't implemented. (though I did write up
a rough sketch of how to do that in a decentralized fashion on
#bitcoin-dev) Propose something concrete.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

> Bitcoin is and always will be limited in capacity - transactions may not
> confirm in a reasonable about of time because of high-demand and/or DoS
> attacks.

I agree in the general case, but I was talking about the mobile wallet case specifically (i.e. people who are sending money between themselves or making small purchases of physical things). I think Bitcoin should be able to scale to handle these sorts of ordinary every-day transactions. Where I’d expect to see transactions falling off the edge is in more specialised cases like very small single micropayments, or “optional” internal transactions like mixing/re/defragmentation of wallets that don’t correspond to an actual payment. Those sorts of transactions would I guess be the first to go when faced with a sudden capacity crunch, but they wouldn’t show up in a mobile wallet UI anyway.

> re: merchants paying tx fees, child-pays-for-parent is inefficient

I know the existing code is, but is that fundamentally the case or just how the code has been written? I haven’t looked at this issue much but I know you’ve worked on it, so I’m curious to learn about why it’s inefficient and whether there are any fixes possible.	

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4127 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1846 bytes --]

On Sun, Dec 01, 2013 at 07:18:07PM +0100, Mike Hearn wrote:
> > Bitcoin is and always will be limited in capacity - transactions may not
> > confirm in a reasonable about of time because of high-demand and/or DoS
> > attacks.
> 
> I agree in the general case, but I was talking about the mobile wallet case specifically (i.e. people who are sending money between themselves or making small purchases of physical things). I think Bitcoin should be able to scale to handle these sorts of ordinary every-day transactions. Where I’d expect to see transactions falling off the edge is in more specialised cases like very small single micropayments, or “optional” internal transactions like mixing/re/defragmentation of wallets that don’t correspond to an actual payment. Those sorts of transactions would I guess be the first to go when faced with a sudden capacity crunch, but they wouldn’t show up in a mobile wallet UI anyway.

Maybe, maybe not. We have no idea what fees will be because the system's
entire capacity is, and always will be, limited. That's just how
fundementally unscalable systems with huge global state work. What
demand will be for that limited capacity is unknown.


> > re: merchants paying tx fees, child-pays-for-parent is inefficient
> 
> I know the existing code is, but is that fundamentally the case or just how the code has been written? I haven’t looked at this issue much but I know you’ve worked on it, so I’m curious to learn about why it’s inefficient and whether there are any fixes possible.	

No, Luke's existing code uses good algorithms with O(n) scaling for n
transactions. The inefficiency is needing a second transaction, bloating
the blockchain and driving up fees.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Patrick Mead]

First time posting to this mailing list so feel free to ignore me if
this is a stupid idea.


On Mon, Dec 2, 2013 at 3:49 AM, Mike Hearn <mike@plan99.net> wrote:
>
> We need to get away from the notion of senders attaching fees anyway. This is the wrong
> way around because it’s the recipient who cares about double spending risk, not the sender.
>


It seems to me that a common problem currently revolves around
accepting transactions in
retail scenarios, such as paying for a sandwich from Subway. A
solution could be to give the
vendor responsibility for setting the fee, which means they can choose
the trade-off that works
best for them in terms of fee size vs. speed of processing.

Idea:
Add a "fee" parameter to the payment URI specification.
When processing the transaction, the customer's UI should show only
the total price, including
both the transfer amount and the fee. The vendor only accepts the
transaction if the customer
uses the right amount and fee. If the fee is too small (for example,
the user might be using an
older wallet and has selected a fee of zero), the vendor can issue a
refund transaction
immediately and tell the user to try again.

Pros:
- could easily be implemented immediately
- old wallets would still be supported by just manually entering the
fee as users do now
- no greater risk of double spending on either side
- maintains the distributed nature of the system
- relies on humans to judge the fee (who are much less likely to
spiral infinitely upwards)
- flexible enough to support varying sizes of transaction and varying
degrees of security

Cons
- requires the vendor to have sufficient understanding of Bitcoin to
make the trade-off
- doesn't solve the problem of selecting a fee for transactions
between individuals/laymen
- doesn't solve fee selection for automated transactions such as
mixing/de/refragmentation


Thoughts?

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3541 bytes --]

Right, as I said earlier:

"The payment protocol at least would need some notion of fee, or possibly
(better?) the ability for a recipient to specify some inputs as well as
some outputs."

Having thought about it a bit more, I think it's better to just have a fee
field that lets the receiver request the sender to attach the given fee.
The outputs would have less value associated with them, so effectively the
seller folds the fee into the price. If the seller is charging a round
price like 1 mBTC, the user sees "1 mBTC" as the price, even if behind the
scenes the created tx only sends 0.99999 BTC

Allowing specification of inputs seems to add too much complexity in other
cases, like when value isn't specified at all.


On Mon, Dec 2, 2013 at 2:54 PM, Patrick Mead <patrick@meadia.com.au> wrote:

> First time posting to this mailing list so feel free to ignore me if
> this is a stupid idea.
>
>
> On Mon, Dec 2, 2013 at 3:49 AM, Mike Hearn <mike@plan99.net> wrote:
> >
> > We need to get away from the notion of senders attaching fees anyway.
> This is the wrong
> > way around because it’s the recipient who cares about double spending
> risk, not the sender.
> >
>
>
> It seems to me that a common problem currently revolves around
> accepting transactions in
> retail scenarios, such as paying for a sandwich from Subway. A
> solution could be to give the
> vendor responsibility for setting the fee, which means they can choose
> the trade-off that works
> best for them in terms of fee size vs. speed of processing.
>
> Idea:
> Add a "fee" parameter to the payment URI specification.
> When processing the transaction, the customer's UI should show only
> the total price, including
> both the transfer amount and the fee. The vendor only accepts the
> transaction if the customer
> uses the right amount and fee. If the fee is too small (for example,
> the user might be using an
> older wallet and has selected a fee of zero), the vendor can issue a
> refund transaction
> immediately and tell the user to try again.
>
> Pros:
> - could easily be implemented immediately
> - old wallets would still be supported by just manually entering the
> fee as users do now
> - no greater risk of double spending on either side
> - maintains the distributed nature of the system
> - relies on humans to judge the fee (who are much less likely to
> spiral infinitely upwards)
> - flexible enough to support varying sizes of transaction and varying
> degrees of security
>
> Cons
> - requires the vendor to have sufficient understanding of Bitcoin to
> make the trade-off
> - doesn't solve the problem of selecting a fee for transactions
> between individuals/laymen
> - doesn't solve fee selection for automated transactions such as
> mixing/de/refragmentation
>
>
> Thoughts?
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4858 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Jeff Garzik]

On Mon, Dec 2, 2013 at 9:33 AM, Mike Hearn <mike@plan99.net> wrote:
> "The payment protocol at least would need some notion of fee, or possibly
> (better?) the ability for a recipient to specify some inputs as well as some
> outputs."

<vendor hat: on>

BitPay noticed this detail last week.  We were noticing that some
transactions were not even reaching our bitcoind border routers (edge
nodes), due to low/no fees.  That led to a long discussion of all
things fee-related.  SPV fees are a big issue.  Getting
child-pays-for-parent in some form out to miners is another.  Getting
a smart, dynamic fee market Gavin mentions is a big need.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]

PPv1 doesn't have any notion of fee unfortunately. I suppose it could be
added easily, but we also need to launch the existing feature set.

There's code pending review to implement PPv1 in bitcoinj, unfortunately
it's currently not passing unit tests and the author can't figure out why.
I didn't have time to debug it yet myself. I'm hopeful we can get it
working and merged by EOY.

It may be time to start talking about timelines for 0.9. I am wondering if
floating fees should be broken out of the 0.9 release and launched in a
quick 0.10 followup - if that were to be done then I think 0.9 could go to
beta relatively soon, like early next year. There have been a lot of
improvements already and it'd be a shame to block them all further.



On Mon, Dec 2, 2013 at 3:37 PM, Jeff Garzik <jgarzik@bitpay.com> wrote:

> On Mon, Dec 2, 2013 at 9:33 AM, Mike Hearn <mike@plan99.net> wrote:
> > "The payment protocol at least would need some notion of fee, or possibly
> > (better?) the ability for a recipient to specify some inputs as well as
> some
> > outputs."
>
> <vendor hat: on>
>
> BitPay noticed this detail last week.  We were noticing that some
> transactions were not even reaching our bitcoind border routers (edge
> nodes), due to low/no fees.  That led to a long discussion of all
> things fee-related.  SPV fees are a big issue.  Getting
> child-pays-for-parent in some form out to miners is another.  Getting
> a smart, dynamic fee market Gavin mentions is a big need.
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc.      https://bitpay.com/
>

[-- Attachment #2: Type: text/html, Size: 2254 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Jeff Garzik]

Current rough timeline proposed for 0.9 was end-of-January, IIRC.

On Mon, Dec 2, 2013 at 9:44 AM, Mike Hearn <mike@plan99.net> wrote:
> PPv1 doesn't have any notion of fee unfortunately. I suppose it could be
> added easily, but we also need to launch the existing feature set.
>
> There's code pending review to implement PPv1 in bitcoinj, unfortunately
> it's currently not passing unit tests and the author can't figure out why. I
> didn't have time to debug it yet myself. I'm hopeful we can get it working
> and merged by EOY.
>
> It may be time to start talking about timelines for 0.9. I am wondering if
> floating fees should be broken out of the 0.9 release and launched in a
> quick 0.10 followup - if that were to be done then I think 0.9 could go to
> beta relatively soon, like early next year. There have been a lot of
> improvements already and it'd be a shame to block them all further.
>
>
>
> On Mon, Dec 2, 2013 at 3:37 PM, Jeff Garzik <jgarzik@bitpay.com> wrote:
>>
>> On Mon, Dec 2, 2013 at 9:33 AM, Mike Hearn <mike@plan99.net> wrote:
>> > "The payment protocol at least would need some notion of fee, or
>> > possibly
>> > (better?) the ability for a recipient to specify some inputs as well as
>> > some
>> > outputs."
>>
>> <vendor hat: on>
>>
>> BitPay noticed this detail last week.  We were noticing that some
>> transactions were not even reaching our bitcoind border routers (edge
>> nodes), due to low/no fees.  That led to a long discussion of all
>> things fee-related.  SPV fees are a big issue.  Getting
>> child-pays-for-parent in some form out to miners is another.  Getting
>> a smart, dynamic fee market Gavin mentions is a big need.
>>
>> --
>> Jeff Garzik
>> Bitcoin core developer and open source evangelist
>> BitPay, Inc.      https://bitpay.com/
>
>



-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

Re: [Bitcoin-development] Floating fees and SPV clients

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 1146 bytes --]

On Tue, Dec 3, 2013 at 12:44 AM, Mike Hearn <mike@plan99.net> wrote:

> PPv1 doesn't have any notion of fee unfortunately. I suppose it could be
> added easily, but we also need to launch the existing feature set.
>

Lets bang out a merchant-pays-fee extension.

How about:

SPEC:

    optional uint64 allowfee    tag number=1000

Allow up to allowfee satoshis to be deducted from the amount paid to be
used to pay Bitcoin network transaction fees. A wallet implementation must
not reduce the amount paid for fees more than allowfee, and transaction
fees must be equal to or greater than the amount reduced.

:ENDSPEC

Rationale: we don't want wallet software giving users discounts-- sending
transactions that are amount-allowfee without paying any fee.  We also want
to allow users to pay MORE in fees, if they need to (fragmented wallet,
maybe, or big CoinJoin transaction) or decide to.


PS: I think there was also consensus that the BIP72  request=...   should
be shortened to just r=... (save 6 chars in QR codes).  Unless somebody
objects, I'll change the BIP and the reference implementation code to make
it so...

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 1732 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

On Tue, Dec 3, 2013 at 2:40 AM, Gavin Andresen <gavinandresen@gmail.com>wrote:

>     optional uint64 allowfee    tag number=1000
>

Let's just use a normal/low tag number. The extensions mechanism is great
for people who want to extend the protocol outside the core development
process. It'd be weird if nobody ever used the low numbers again though.

Tag numbers are varint encoded so using smaller ones does have a minor
efficiency benefit, it's not just aesthetics :)


> Allow up to allowfee satoshis to be deducted from the amount paid to be
> used to pay Bitcoin network transaction fees. A wallet implementation must
> not reduce the amount paid for fees more than allowfee, and transaction
> fees must be equal to or greater than the amount reduced.
>

Hmmm. Why "allow"? Should it not be called min_fee instead? Wallets would
have to attach at least that much in fees, right?

Also, why describe it as reducing the amount paid? Which output would be
reduced in value? Why not just have it be added to the total value
displayed to the user and the outputs are left alone/not reduced.


> We also want to allow users to pay MORE in fees, if they need to
> (fragmented wallet, maybe, or big CoinJoin transaction) or decide to.
>

I like the idea but it seems this gets us back to the original problem -
senders don't care about confirmations, ever, not even if they make an
annoying set of transactions. The protocol allows users to submit
transactions directly to receivers, I guess, if the receiver does not like
the transactions they get they could potentially reject the payment. But
I'd hope that's really rare.


> PS: I think there was also consensus that the BIP72  request=...   should
> be shortened to just r=... (save 6 chars in QR codes).  Unless somebody
> objects, I'll change the BIP and the reference implementation code to make
> it so...
>

Sweet, thanks!

[-- Attachment #2: Type: text/html, Size: 3194 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Drak]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

I dont like the idea of putting the min fee in the hands of the receiver.
Seems like that will work against the best interests of senders in the long
run.

Why not try a different path of calculating the min fee like difficulty
retarget. You can analyse the last 2016 blocks to find the average fee
accepted per kb (which would include transactions that were included
without fees) and then write that into the block as a soft recommendation
that wallets could use in the UI. This way the price can vary up and down
according to what people were willing to spend on fees and miners willing
to accept.

I absolutely do not trust vendors to set fees. I think it has to be based
on what senders are willing to pay and what miners are willing to accept.

Drak

[-- Attachment #2: Type: text/html, Size: 1061 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2138 bytes --]

On Tue, Dec 3, 2013 at 11:36 AM, Drak <drak@zikula.org> wrote:

> I dont like the idea of putting the min fee in the hands of the receiver.
> Seems like that will work against the best interests of senders in the long
> run.
>

Senders have no interest in ever attaching any kind of fee, which is one
reason we explored child-pays-for-parent for a while. It's not the sender
who cares about double spending risk. Left to their own devices, all
senders would always attach no fee at all (or rather: whatever the min was
to get the transaction relayed to the merchant).

However, receivers do want a fee attached, and ideally we would do this
without redundant transactions. Hence, receivers asking senders to attach a
fee and effectively folding it into the price that is paid. That is, if you
go into a restaurant and the menu says "Burger: 10mBTC" then when you come
to pay, what you see on your phone screen is 10mBTC. The fact that actually
the shop with receiver 9.9mBTC and the tx fee is 0.1mBTC is hidden in the
user interface - creating a situation like many others, where receivers eat
a transaction cost. For instance in Europe sales taxes are included in the
price, not attached separately later.

There's no need to trust the vendor. If a vendor asks for a ridiculously
high tx fee, it will just surface as uncompetitively priced goods/services.
Buyers will go elsewhere.


> Why not try a different path of calculating the min fee like difficulty
> retarget. You can analyse the last 2016 blocks to find the average fee
> accepted per kb (which would include transactions that were included
> without fees) and then write that into the block as a soft recommendation
> that wallets could use in the UI. This way the price can vary up and down
> according to what people were willing to spend on fees and miners willing
> to accept.
>

That's what fee estimation does, essentially, minus the encoding into
blocks. Once you start getting miners telling people what fees are directly
you run into cases where they might try to lie about their behaviour or
otherwise influence the average. Querying all nodes avoids that problem.

[-- Attachment #2: Type: text/html, Size: 2764 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Drak]

[-- Attachment #1: Type: text/plain, Size: 2731 bytes --]

On 3 December 2013 10:45, Mike Hearn <mike@plan99.net> wrote:

> On Tue, Dec 3, 2013 at 11:36 AM, Drak <drak@zikula.org> wrote:
>
>> I dont like the idea of putting the min fee in the hands of the receiver.
>> Seems like that will work against the best interests of senders in the long
>> run.
>>
>
> Senders have no interest in ever attaching any kind of fee, which is one
> reason we explored child-pays-for-parent for a while. It's not the sender
> who cares about double spending risk. Left to their own devices, all
> senders would always attach no fee at all (or rather: whatever the min was
> to get the transaction relayed to the merchant).
>

I respectfully disagree. Senders need their funds to be received. The
incentive is right there. Miners want mining fees. So if you want to pay
for something, you need to make sure payment arrives. Senders know that if
they exclude the fee it might not arrive at all. Miners increasingly ignore
no or low fees. So those two agents together ensure there is a fee more
than not. If what you said was true, we would hardly see fees being paid at
all, but on the contrary we see lots of fees, and much higher than the
minimum 0.0001/kb rate that is currently required.

Merchants will just include ridiculous fees - there are some exchanges that
do it already - MtGox being the famous example requires a 0.001 fee 10x
higher than the network rate - the CEO does it because he says "it's
better". That's not a fee going to MtGox, that is the miner fee and they
have no plans to reduce it.

Typically vendor software may not get updated and or lag behind with fat
fees never decreasing or decreasing way too slowly - it's just not fluid or
dynamic enough when it rests with the vendor.

However, receivers do want a fee attached,
>

No - receivers want to be paid. If they are not paid they wont dispatch the
goods or service. Neither party is happy in that circumstance. The
incentive that the payment confirms is there naturally.


> That's what fee estimation does, essentially, minus the encoding into
> blocks. Once you start getting miners telling people what fees are directly
> you run into cases where they might try to lie about their behaviour or
> otherwise influence the average. Querying all nodes avoids that problem.
>

Miners cant lie about fees accepted because that's part of the transaction.
When Alice sends funds to Bob it includes the fee information and that is
included in the block. There is no way to fake it. The average fee paid is
provable - so there is no need to query nodes at all, you simply look at
the blockchain. You dont even need to write it into the blockchain since it
can be calculated from the blockchain, it would just make it simpler.

Drak

[-- Attachment #2: Type: text/html, Size: 4029 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

Ok, revised spec:

SPEC:

message PaymentDeatils {
    ...
    optional uint64 minfee    tag number=8

Pay at least minfee satoshis in transaction fees. Wallet software should
add minfee to the amount the user authorizes and pays, and include at least
minfee in the transaction created to pay miner's transaction fees. Wallet
software may request that the user pays more, if it must create a complex
transaction or judges that minfee is not sufficient for the transaction to
be accepted by the network..

:ENDSPEC

Making it fee-per-kilobyte is a bad idea, in my opinion; users don't care
how many kilobytes their transactions are, and they will just be confused
if they're paying for a 10mBTC burger and are asked to pay 10.00011 or
9.9994 because the merchant has no idea how many kilobytes the paying
transaction will be.

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 1565 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Tue, Dec 3, 2013 at 12:07 PM, Gavin Andresen <gavinandresen@gmail.com>wrote:

> Making it fee-per-kilobyte is a bad idea, in my opinion; users don't care
> how many kilobytes their transactions are, and they will just be confused
> if they're paying for a 10mBTC burger and are asked to pay 10.00011 or
> 9.9994 because the merchant has no idea how many kilobytes the paying
> transaction will be.
>

Wouldn't the idea be that the user always sees 10mBTC no matter what, but
the receiver may receive less if the user decides to pay with a huge
transaction?

It may be acceptable that receivers don't always receive exactly what they
requested, at least for person-to-business transactions.  For
person-to-person transactions of course any fee at all is confusing because
you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the
other end. I wonder if we'll end up in a world where buying things from
shops involves paying fees, and (more occasional?) person-to-person
transactions tend to be free and people just understand that the money
isn't going to be spendable for a while. Or alternatively that wallets let
you override the safeguards on spending unconfirmed coins when the user is
sure that they trust the sender.

[-- Attachment #2: Type: text/html, Size: 1637 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]

On Tue, Dec 03, 2013 at 12:29:03PM +0100, Mike Hearn wrote:
> On Tue, Dec 3, 2013 at 12:07 PM, Gavin Andresen <gavinandresen@gmail.com>wrote:
> 
> > Making it fee-per-kilobyte is a bad idea, in my opinion; users don't care
> > how many kilobytes their transactions are, and they will just be confused
> > if they're paying for a 10mBTC burger and are asked to pay 10.00011 or
> > 9.9994 because the merchant has no idea how many kilobytes the paying
> > transaction will be.
> >
> 
> Wouldn't the idea be that the user always sees 10mBTC no matter what, but
> the receiver may receive less if the user decides to pay with a huge
> transaction?
> 
> It may be acceptable that receivers don't always receive exactly what they
> requested, at least for person-to-business transactions.  For
> person-to-person transactions of course any fee at all is confusing because
> you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the
> other end. I wonder if we'll end up in a world where buying things from
> shops involves paying fees, and (more occasional?) person-to-person
> transactions tend to be free and people just understand that the money
> isn't going to be spendable for a while. Or alternatively that wallets let
> you override the safeguards on spending unconfirmed coins when the user is
> sure that they trust the sender.

Person-to-person payments are an *excellent* argument for keeping fees
visible to end-users; people will pay other people commonly in Bitcoin
and they will be very confused if those transactions act weirdly
differently than payments to merchants.


NAK on unconfirmed overrides - if something goes wrong even by accident
it just makes fixing the problem much harder and less intuitive.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

>
> Wouldn't the idea be that the user always sees 10mBTC no matter what, but
> the receiver may receive less if the user decides to pay with a huge
> transaction?
>

If users want to pay with a huge transaction then it seems to me the user
should cover that cost. Allowing users to pay merchants with 100K
transactions full of dust and expecting them to eat the cost seems like a
great way to enable bleed-the-merchant-dry attacks.


RE: hiding or showing fees:  I pointed out to Peter that there doesn't have
to be One True Answer.  Let wallets experiment with either hiding or
exposing fees, and may the best user experience win.

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 1209 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1013 bytes --]

On Tue, Dec 3, 2013 at 12:41 PM, Gavin Andresen <gavinandresen@gmail.com>wrote:
>
> If users want to pay with a huge transaction then it seems to me the user
> should cover that cost. Allowing users to pay merchants with 100K
> transactions full of dust and expecting them to eat the cost seems like a
> great way to enable bleed-the-merchant-dry attacks.
>

A merchant can always refuse the payment and refund it if that's a
practical problem. I doubt it would be though. If a user is trying to buy
something from the merchant, they will want it to work, and it'll be up to
the developers of the wallet they're using to ensure it never does anything
obnoxious or unacceptable that would result in people hating to receive
money from that app.


> RE: hiding or showing fees:  I pointed out to Peter that there doesn't
> have to be One True Answer.  Let wallets experiment with either hiding or
> exposing fees, and may the best user experience win.
>

Sure. I think there will be experimentation in this regard.

[-- Attachment #2: Type: text/html, Size: 1653 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

>
> A merchant can always refuse the payment and refund it if that's a
> practical problem.
>

No, they can't, at least not in bitcoin-qt:  when the user pokes the SEND
button, the transaction is broadcast on the network, and then the merchant
is also told with the Payment/PaymentACK round-trip.

Allowing merchants to cancel (e.g. having a PaymentNACK) makes
implementation harder, and brings up nasty issues if we want to allow
CoinJoin or CoinJoin-like transactions as payments to merchants.
 Bitcoin-Qt ALREADY allows you to pay several PaymentRequests with one
transaction; handling the case where one merchant gives you a PaymentACK
and another gives you (or wants to give you) a PaymentNACK is a nightmare.

--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 1114 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Drak]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

On 3 December 2013 11:46, Mike Hearn <mike@plan99.net> wrote:

> On Tue, Dec 3, 2013 at 12:41 PM, Gavin Andresen <gavinandresen@gmail.com>wrote:
>>
>> If users want to pay with a huge transaction then it seems to me the user
>> should cover that cost. Allowing users to pay merchants with 100K
>> transactions full of dust and expecting them to eat the cost seems like a
>> great way to enable bleed-the-merchant-dry attacks.
>>
>
> A merchant can always refuse the payment and refund it if that's a
> practical problem. I doubt it would be though. If a user is trying to buy
> something from the merchant, they will want it to work, and it'll be up to
> the developers of the wallet they're using to ensure it never does anything
> obnoxious or unacceptable that would result in people hating to receive
> money from that app.
>

Refunds in this circumstance would be problematic because someone is going
to lose because they have to pay the fee. If the sender's money is refunded
minus the fee, they will be unhappy. And the merchant will be unhappy about
having had an unacceptable transaction they have to send back, and eat a
fee for the privilege.

This kind of situation needs to be avoided at all costs.

[-- Attachment #2: Type: text/html, Size: 2091 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Taylor Gerring]

[-- Attachment #1: Type: text/plain, Size: 1797 bytes --]


On Dec 3, 2013, at 12:29 PM, Mike Hearn <mike@plan99.net> wrote:

> It may be acceptable that receivers don't always receive exactly what they requested, at least for person-to-business transactions.  For person-to-person transactions of course any fee at all is confusing because you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the other end. I wonder if we'll end up in a world where buying things from shops involves paying fees, and (more occasional?) person-to-person transactions tend to be free and people just understand that the money isn't going to be spendable for a while.


> person-to-business transactions.  For person-to-person transactions
Why should there be two classes of transactions? Where does paying a local business at a farmer’s stand lie in that realm? Transactions should work the same regardless of who is on the receiving end.

> any fee at all is confusing because you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the other end
The paradigm of sending money has an explicit cost is not new... I think people are used to Western Union/PayPal and associated fees, no?  It’s okay to have a fee if it’s reasonable, so let’s inform the user what the estimated cost is to send a transaction in a reasonable amount of time.

>  wonder if we'll end up in a world where buying things from shops involves paying fees
I stayed in a hotel for the first night here here in Milan, and there was an (anticipated) surcharge for the use of credit over cash. Again, nothing new here.


Fees are only confusing because existing clients do a terrible job of presenting the information to the user. In Hive Wallet, I’m of the opinion that we should inform the user in an intuitive way to let them make an informed decision.

[-- Attachment #2: Type: text/html, Size: 2687 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

On Tue, Dec 03, 2013 at 12:57:23PM +0100, Taylor Gerring wrote:
> 
> On Dec 3, 2013, at 12:29 PM, Mike Hearn <mike@plan99.net> wrote:
> 
> > It may be acceptable that receivers don't always receive exactly what they requested, at least for person-to-business transactions.  For person-to-person transactions of course any fee at all is confusing because you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the other end. I wonder if we'll end up in a world where buying things from shops involves paying fees, and (more occasional?) person-to-person transactions tend to be free and people just understand that the money isn't going to be spendable for a while.
> 
> 
> > person-to-business transactions.  For person-to-person transactions
> Why should there be two classes of transactions? Where does paying a local business at a farmer’s stand lie in that realm? Transactions should work the same regardless of who is on the receiving end.
> 
> > any fee at all is confusing because you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the other end
> The paradigm of sending money has an explicit cost is not new... I think people are used to Western Union/PayPal and associated fees, no?  It’s okay to have a fee if it’s reasonable, so let’s inform the user what the estimated cost is to send a transaction in a reasonable amount of time.

Indeed.

Transparency on fees is going to be good from a marketing point of view
as well: fact is, Bitcoin transations have fees involved, and if we're
up-front and honest about those fees and what they are and why, we
demystify the system and give people the confidence to tell others about
the cost-advantages of Bitcoin, and at the same time, combat fud about
fees with accurate and honest information.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Jamie McNaught]

[-- Attachment #1: Type: text/plain, Size: 4002 bytes --]

Hi all, first post so go easy on me!

Background/Intro: I'm a C/C++ software engineer with a keen interest in
Bitcoin working for everyone. I've spent the last couple of months pitching
Bitcoin to merchants & end users (previously I mined),

While I agree as Peter said, transparency with fees is good and leaving it
to the wallet designer to decide to either show / hide these (as Gavin
suggested) there is no denying that users hate fees/charges. That's why in
the UK so many online retailers offer free delivery (again perceived as a
zero value add charge). Infact, if you ask the average consumer, they only
have a mild inkling that there are fees paid by the merchants for using
credit cards.

So I'd agree with Gavin's proposed spec in an optional uint64 minfee which
senders(wallets) should deduct from the total paid to the receiver. If
however the sender is doing a dust collection txn (surely hugely unusual
for legit users?) then the sender should pay the additional costs. Does
that make "minfee" actually "minfeeperkb"? Perhaps, but wallets should aim
to not display such implementation details.

As a newb though, I have to ask, how does the receiver/requester/merchant
enforce these fee requests are respected?




On 3 December 2013 12:07, Peter Todd <pete@petertodd.org> wrote:

> On Tue, Dec 03, 2013 at 12:57:23PM +0100, Taylor Gerring wrote:
> >
> > On Dec 3, 2013, at 12:29 PM, Mike Hearn <mike@plan99.net> wrote:
> >
> > > It may be acceptable that receivers don't always receive exactly what
> they requested, at least for person-to-business transactions.  For
> person-to-person transactions of course any fee at all is confusing because
> you intuitively expect that if you send 1 mBTC, then 1 mBTC will arrive the
> other end. I wonder if we'll end up in a world where buying things from
> shops involves paying fees, and (more occasional?) person-to-person
> transactions tend to be free and people just understand that the money
> isn't going to be spendable for a while.
> >
> >
> > > person-to-business transactions.  For person-to-person transactions
> > Why should there be two classes of transactions? Where does paying a
> local business at a farmer’s stand lie in that realm? Transactions should
> work the same regardless of who is on the receiving end.
> >
> > > any fee at all is confusing because you intuitively expect that if you
> send 1 mBTC, then 1 mBTC will arrive the other end
> > The paradigm of sending money has an explicit cost is not new... I think
> people are used to Western Union/PayPal and associated fees, no?  It’s okay
> to have a fee if it’s reasonable, so let’s inform the user what the
> estimated cost is to send a transaction in a reasonable amount of time.
>
> Indeed.
>
> Transparency on fees is going to be good from a marketing point of view
> as well: fact is, Bitcoin transations have fees involved, and if we're
> up-front and honest about those fees and what they are and why, we
> demystify the system and give people the confidence to tell others about
> the cost-advantages of Bitcoin, and at the same time, combat fud about
> fees with accurate and honest information.
>
> --
> 'peter'[:-1]@petertodd.org
> 000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 5183 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2493 bytes --]

On Tue, Dec 3, 2013 at 12:57 PM, Taylor Gerring <taylor.gerring@gmail.com>wrote:

> Why should there be two classes of transactions? Where does paying a local
> business at a farmer’s stand lie in that realm? Transactions should work
> the same regardless of who is on the receiving end.
>

Lots and lots of people are psychologically trained to expect that they pay
the sticker price for things. Yes in recent times some places have started
to show additional fees for using credit cards, but only as a way to try
and push people onto cheaper forms of payment, not because customers love
surcharges. It's for that reason that many merchants don't do this, even
when they could - I pay for things with Maestro Debit all the time and I
don't think I've ever seen a surcharge. That system obviously has costs,
but they're included.

This is just a basic cultural thing - when I buy something from a shop, the
social expectation is that the seller should be grateful for receiving my
money. "The customer is always right". When I send money to a friend, the
social expectation is different. If my friend said, hey Mike, could you
send me that 10 bucks you owe me from last weekend and what he receives is
less than 10 bucks, he would probably feel annoyed - if I owe him 10 bucks
then I owe him 10 bucks and it's my job the cover the fees. That's why
PayPal makes sender pay fees in that case.

Maybe we need new terminology for this. *Interior fees* for included in the
price/receiver pays and *exterior fees* for excluded from the price/sender
pays?

Fees are only confusing because existing clients do a terrible job of
> presenting the information to the user. In Hive Wallet, I’m of the opinion
> that we should inform the user in an intuitive way to let them make an
> informed decision.
>

Have you thought through the UI for that in detail? How exactly are you
going to explain the fee structure? Let the user pick the number of blocks
they need to wait for? What's a block? Why should I care? Why shouldn't I
just set the slider all the way to the other end and pay no fees at all? Is
the merchant going to refuse to take my payment? Gavin just said that's not
possible with Bitcoin-Qt. I'm thinking for bitcoinj I might go in a
slightly different direction and not broadcast payments submitted via the
payment protocol (and definitely not have one wire tx pay multiple payment
requests simultaneously, at least not for consumer wallets).

[-- Attachment #2: Type: text/html, Size: 3295 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Taylor Gerring]

[-- Attachment #1: Type: text/plain, Size: 3161 bytes --]


On Dec 3, 2013, at 2:20 PM, Mike Hearn <mike@plan99.net> wrote:

> On Tue, Dec 3, 2013 at 12:57 PM, Taylor Gerring <taylor.gerring@gmail.com> wrote:
> Why should there be two classes of transactions? Where does paying a local business at a farmer’s stand lie in that realm? Transactions should work the same regardless of who is on the receiving end.
> 
> Lots and lots of people are psychologically trained to expect that they pay the sticker price for things. Yes in recent times some places have started to show additional fees for using credit cards, but only as a way to try and push people onto cheaper forms of payment, not because customers love surcharges. It's for that reason that many merchants don't do this, even when they could - I pay for things with Maestro Debit all the time and I don't think I've ever seen a surcharge. That system obviously has costs, but they're included.
> 
> This is just a basic cultural thing - when I buy something from a shop, the social expectation is that the seller should be grateful for receiving my money. "The customer is always right". When I send money to a friend, the social expectation is different. If my friend said, hey Mike, could you send me that 10 bucks you owe me from last weekend and what he receives is less than 10 bucks, he would probably feel annoyed - if I owe him 10 bucks then I owe him 10 bucks and it's my job the cover the fees. That's why PayPal makes sender pay fees in that case.
> 
> Maybe we need new terminology for this. Interior fees for included in the price/receiver pays and exterior fees for excluded from the price/sender pays?
> 
> Fees are only confusing because existing clients do a terrible job of presenting the information to the user. In Hive Wallet, I’m of the opinion that we should inform the user in an intuitive way to let them make an informed decision.
> 
> Have you thought through the UI for that in detail? How exactly are you going to explain the fee structure? Let the user pick the number of blocks they need to wait for? What's a block? Why should I care? Why shouldn't I just set the slider all the way to the other end and pay no fees at all? Is the merchant going to refuse to take my payment? Gavin just said that's not possible with Bitcoin-Qt. I'm thinking for bitcoinj I might go in a slightly different direction and not broadcast payments submitted via the payment protocol (and definitely not have one wire tx pay multiple payment requests simultaneously, at least not for consumer wallets).
> 
> 

Most of what you mentioned is entirely culture-dependant. In the majority of North America, sales tax is calculated at the point of sale on top of the advertised price. When my local government increases sales taxes, we feel it BECAUSE we see it. Expose information in a digestible way. Just because you don’t instinctively know how to implement a UI for varying sender fees doesn’t mean that other wallets don’t.

Leave the fee structure alone. Instead, let’s concentrate on how to calculate an accurate assessment of what a reasonable fee is for reliable service and let the software shake out the rest.

Taylor


[-- Attachment #2: Type: text/html, Size: 4343 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

Heh. People feel rises in sales tax elsewhere too. When VAT rises merchants
all raise their prices, they don't normally swallow it (or if they do, they
make a big fuss over how awesome they are).

The US system is a complete pain in the ass. You never know how much money
you actually need to pay for anything unless you happen to know the local
rate and do the multiplication in your head. There's a reason this system
is not used in big chunks of the world economy.


> Expose information in a digestible way. Just because you don’t
> instinctively know how to implement a UI for varying sender fees doesn’t
> mean that other wallets don’t.
>

I would love to know how to do it. If you have an intuitive GUI in mind
please show us so other wallet authors can copy it :)

[-- Attachment #2: Type: text/html, Size: 1167 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Quinn Harris]

[-- Attachment #1: Type: text/plain, Size: 4426 bytes --]

The merchant wants to include a fee to ensure the transaction is 
confirmed which is dependent on the fee per kilobyte, but they don't 
want to pay unexpectedly high fees. So what about including a 
min_fee_per_kilobyte and a max_fee in PaymentDetails describing what 
fees the merchant will pay.  The sender would be expected to respect the 
min_fee_per_kilobyte but if the result exceeds max_fee the sender would 
be agreeing to pay the extra fee (exterior fees).  The sender might also 
agree to pay fees in excess of min_fee_per_kilobyte.

The sender would deduct the interior or merchant fees from the first output.

The UI could show the payment "price" which would match the sum of 
original outputs.  It would show the merchant fees (interior) and sender 
fees (exterior) if there are any.  The UI should always show fees so 
users learn to expect them for all transactions.

This should allow the merchant to pay fees in most cases while not 
having to pay excessive fees if the sender wants to use some large 
transaction.  If max_fee is 0 the sender would be expected to pay all fees.

On 12/03/2013 10:20 AM, Mike Hearn wrote:
> On Tue, Dec 3, 2013 at 12:57 PM, Taylor Gerring 
> <taylor.gerring@gmail.com <mailto:taylor.gerring@gmail.com>> wrote:
>
>     Why should there be two classes of transactions? Where does paying
>     a local business at a farmer's stand lie in that realm?
>     Transactions should work the same regardless of who is on the
>     receiving end.
>
>
> Lots and lots of people are psychologically trained to expect that 
> they pay the sticker price for things. Yes in recent times some places 
> have started to show additional fees for using credit cards, but only 
> as a way to try and push people onto cheaper forms of payment, not 
> because customers love surcharges. It's for that reason that many 
> merchants don't do this, even when they could - I pay for things with 
> Maestro Debit all the time and I don't think I've ever seen a 
> surcharge. That system obviously has costs, but they're included.
>
> This is just a basic cultural thing - when I buy something from a 
> shop, the social expectation is that the seller should be grateful for 
> receiving my money. "The customer is always right". When I send money 
> to a friend, the social expectation is different. If my friend said, 
> hey Mike, could you send me that 10 bucks you owe me from last weekend 
> and what he receives is less than 10 bucks, he would probably feel 
> annoyed - if I owe him 10 bucks then I owe him 10 bucks and it's my 
> job the cover the fees. That's why PayPal makes sender pay fees in 
> that case.
>
> Maybe we need new terminology for this. /Interior fees/ for included 
> in the price/receiver pays and /exterior fees/ for excluded from the 
> price/sender pays?
>
>     Fees are only confusing because existing clients do a terrible job
>     of presenting the information to the user. In Hive Wallet, I'm of
>     the opinion that we should inform the user in an intuitive way to
>     let them make an informed decision.
>
>
> Have you thought through the UI for that in detail? How exactly are 
> you going to explain the fee structure? Let the user pick the number 
> of blocks they need to wait for? What's a block? Why should I care? 
> Why shouldn't I just set the slider all the way to the other end and 
> pay no fees at all? Is the merchant going to refuse to take my 
> payment? Gavin just said that's not possible with Bitcoin-Qt. I'm 
> thinking for bitcoinj I might go in a slightly different direction and 
> not broadcast payments submitted via the payment protocol (and 
> definitely not have one wire tx pay multiple payment requests 
> simultaneously, at least not for consumer wallets).
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Type: text/html, Size: 7356 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Jeremy Spilman]

allowfee:

> Allow up to allowfee satoshis to be deducted from the amount paid to be  
> used to pay Bitcoin network transaction fees. A wallet >implementation  
> must not reduce the amount paid for fees more than allowfee, and  
> transaction fees must be equal to or greater than the >amount reduced.

minfee:

> Pay at least minfee satoshis in transaction fees. Wallet software should  
> add minfee to the amount the user authorizes and pays, and >include at  
> least minfee in the transaction created to pay miner's transaction fees.  
> Wallet software may request that the user pays more, if >it must create  
> a complex transaction or judges that minfee is not sufficient for the  
> transaction to be accepted by the network..
>

Thanks for the draft specs Gavin. Very clear and precise.

Personally I think 'allowfee' is more useful than 'minfee'. The 'allowfee'  
tells me something very useful and definitive about what the merchant will  
let me do when making payment, and if the merchant chooses 'allowfee'  
intelligently, they can provide real value to their customers without  
exposing them to undue risk.

A 'minfee' field on the other hand, is just a data point for the wallet  
software to consider, and likely to be noisy enough that wallets will tend  
to ignore it. (e.g. like Drak's example of Gox's 0.001 fee)

The sender's wallet software will always be free to choose the fee, and  
paying less than the 'allowfee' or 'minfee' can still get a TX included in  
the next block.

I think of the PaymentRequest is a part of the purchase contract. If a  
payer transmits a transaction before 'expires' but with less than  
'minfee', which gets included in the next block, have they  failed to meet  
the terms of payment?

If there is some time criticality, for example to reduce exchange rate  
risk, then a wallet might need to choose a higher fee to ensure the  
transaction clears in time. Instead of 'minfee' I'm thinking it would be  
more appropriate to communicate this using the existing 'expires' field --  
in other words, let the merchant express what their requirement is, not  
tell the wallet how to achieve it.

In the case of a transaction with too-low fee, either the payer can  
double-spend with a higher fee, or wait longer for the transaction to make  
it into a block. If it hits the blockchain before the 'expires' time, then  
the merchant should have no standing to refute it, regardless of the  
amount of fees paid.

A refund comes into play if a payer reduced the total amount in excess of  
an agreed upon 'allowfee', or if the transaction doesn't hit the  
blockchain until after 'expires'. It should be clear in these cases that  
payer would end up eating the fees in both directions. But then, what if a  
wallet pays the 'minfee' and broadcasts 1 block before 'expires' but the  
payment DOESN'T make the block? Is the merchant liable for too-slow  
transactions due to their own insufficient 'minfee' value?

So I see 'allowfee' as extremely useful, but 'minfee' as somewhat  
problematic.

Thanks,
Jeremy

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2237 bytes --]

I think this US/other cultural issue is complicating things more than we
appreciate.

I am trying to imagine in my head how all this will work and what it will
look like with allow_fee, and I just can't see it. Merchants want customers
to pay the sticker price, deviance from that social norm is extremely rare
even after the credit card company contracts that required it have been
invalidated. The only time it happens to me is when buying flight tickets
with credit cards: but it's only for that method, other payment methods are
still treated as "free" a.k.a interior fees.

If you walk into a physical shop and try to pay a large bill with bags of
pennies, the merchant won't enter into a complicated agreement where they
agree to split the cost of processing with you. They will just reject the
payment out of hand and tell you to get real. It has to be that way because
otherwise the shop would carry the cost of counting all the pennies and
hauling them around, not the buyer (who "knows" he put the right number of
pennies in the bags).

As a buyer, I do not care about whether my transaction will confirm. If I
try to pay with dust, there is no incentive for me to attach a higher fee
than allow_fee to make that confirm, especially if the merchant has no way
to reject the payment. What's more, as Jeremy points out, no clean fail
mechanism means large piles of manual work and lots of disputes due to
payments not clearing before the exchange rate shifts and other things like
that.

Trying to make the success of payment confirmation a two-person dance seems
to have so many edge cases it makes my head hurt. For most pay-to-merchant
cases, it has to be the receivers job to get a transaction confirmed, and
if the sender doesn't follow the instructions a payment should hard fail
and require trying again. If Bitcoin-Qt can't handle that today, that does
seem like a problem.

In the case of a transaction with too-low fee, either the payer can
> double-spend with a higher fee


You can't do that. When a tx doesn't have the right fee attached you're out
of luck today, except for the fact that some pools run with a custom child
pays for parent patch. So respending it would bump priority for some miners
and not others.

[-- Attachment #2: Type: text/html, Size: 2708 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256



Mike Hearn <mike@plan99.net> wrote:
>I think this US/other cultural issue is complicating things more than
>we
>appreciate.
>
>I am trying to imagine in my head how all this will work and what it
>will
>look like with allow_fee, and I just can't see it. Merchants want
>customers
>to pay the sticker price, deviance from that social norm is extremely
>rare
>even after the credit card company contracts that required it have been
>invalidated. The only time it happens to me is when buying flight
>tickets
>with credit cards: but it's only for that method, other payment methods
>are
>still treated as "free" a.k.a interior fees.
>
>If you walk into a physical shop and try to pay a large bill with bags
>of
>pennies, the merchant won't enter into a complicated agreement where
>they
>agree to split the cost of processing with you. They will just reject
>the
>payment out of hand and tell you to get real. It has to be that way
>because
>otherwise the shop would carry the cost of counting all the pennies and
>hauling them around, not the buyer (who "knows" he put the right number
>of
>pennies in the bags).
>
>As a buyer, I do not care about whether my transaction will confirm. If
>I
>try to pay with dust, there is no incentive for me to attach a higher
>fee
>than allow_fee to make that confirm, especially if the merchant has no
>way
>to reject the payment. What's more, as Jeremy points out, no clean fail
>mechanism means large piles of manual work and lots of disputes due to
>payments not clearing before the exchange rate shifts and other things
>like
>that.
>
>Trying to make the success of payment confirmation a two-person dance
>seems
>to have so many edge cases it makes my head hurt. For most
>pay-to-merchant
>cases, it has to be the receivers job to get a transaction confirmed,
>and
>if the sender doesn't follow the instructions a payment should hard
>fail
>and require trying again. If Bitcoin-Qt can't handle that today, that
>does
>seem like a problem.
>
>In the case of a transaction with too-low fee, either the payer can
>> double-spend with a higher fee
>
>
>You can't do that. When a tx doesn't have the right fee attached you're
>out
>of luck today, except for the fact that some pools run with a custom
>child
>pays for parent patch. So respending it would bump priority for some
>miners
>and not others.


Here at the dark wallet conf there seems yo be rough consensus that replacement for fee bumping is a good thing and should be supported; I was talking to Taylor from hive specifically yesterday. The code is trivial on the node side of things and doesn't need consent of anymore than a small minority, and coinjoin forces wallets to handle double spends well anyway. I haven't heard anyone caring about zeroconf safety.

I'll be proposing it for "formal" inclusion in our wallet best practices guidelines.


Also fwiw apparently libbitcoin already implements a memory limited mempool and Amir is open to the idea of it using the satoshi consensus critical code for block validity. (therefor fairly safe mining) I wouldn't be surprised if libbitcoin based nodes start getting usage, and with a limited mempool it is very DoS attack safe for them to relay replacements regardless of miner support.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.0.9

iQFQBAEBCAA6BQJSnwqsMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhR5yCAC3vaQQeoBrLdqn/rO5
Dzblqwl1B6AE1UjFj5+abQEZ2+uPy5P+7dZidpUn8Ms+tDDcCCge6HVOg+UeseaE
8pDP3+VIHZHH+9n6Y3+4facLNpQ3dP/+Zsg4pC+QSAjVV6408+yWPLtpbC6V0apK
T6K4qdq0Ct6V+54Ol0Thx+5cJlWLI+XbW2eXze3WjJzj3FgZUK0udBcVWa8JyWAV
WD1tv8DpPoUvDDzdmjEyf0EdjvcmamH9mcIvtxRdVwzyY/siZoizv9X8/gXNL+fg
JJ3Oxwrl1dOYSeENgp9VP8fU7GK7855bT1Wxd5zGNW7p/1gNxN4Lnx57XSMz2IHc
dHbg
=dcYz
-----END PGP SIGNATURE-----

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 193 bytes --]

Please don't try and drag this thread off topic. What I said is factually
correct. If you want to (again) try and convince people things should work
differently, start another thread for that.

[-- Attachment #2: Type: text/html, Size: 250 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

On Wed, Dec 04, 2013 at 12:09:42PM +0100, Mike Hearn wrote:
> Please don't try and drag this thread off topic. What I said is factually
> correct. If you want to (again) try and convince people things should work
> differently, start another thread for that.

replace-by-fee is no less speculative than your original proposals;
you're also trying to convince people that things should work
differently re: fees

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

On Wed, Dec 4, 2013 at 2:06 PM, Peter Todd <pete@petertodd.org> wrote:

> replace-by-fee is no less speculative than your original proposals;
> you're also trying to convince people that things should work
> differently re: fees


The original proposal I started this thread with hasn't even received
comments - presumably it's uncontroversial. The other discussions are about
how to handle fees in requests that use the payment protocol, which isn't
currently used anywhere so doing things differently isn't possible.

On the other hand you have been talking about a fundamental change to the
behaviour of how all Bitcoin nodes operate, which is off topic for this
thread.

If you have something specific to say about how floating fees should be
managed by SPV wallets or how fees should be negotiated when the payment
protocol is in use, this thread is appropriate. Otherwise please take it
elsewhere.

[-- Attachment #2: Type: text/html, Size: 1470 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]

On Wed, Dec 04, 2013 at 02:48:08PM +0100, Mike Hearn wrote:
> On Wed, Dec 4, 2013 at 2:06 PM, Peter Todd <pete@petertodd.org> wrote:
> 
> > replace-by-fee is no less speculative than your original proposals;
> > you're also trying to convince people that things should work
> > differently re: fees
> 
> 
> The original proposal I started this thread with hasn't even received
> comments - presumably it's uncontroversial. The other discussions are about
> how to handle fees in requests that use the payment protocol, which isn't
> currently used anywhere so doing things differently isn't possible.
> 
> On the other hand you have been talking about a fundamental change to the
> behaviour of how all Bitcoin nodes operate, which is off topic for this
> thread.
> 
> If you have something specific to say about how floating fees should be
> managed by SPV wallets or how fees should be negotiated when the payment
> protocol is in use, this thread is appropriate. Otherwise please take it
> elsewhere.

Other than you, replacement for fee changing isn't controversial; I know
this because no-one other than you comments on it... just like the
fundemental changes involving your proposed hardfork presumably. (which
I did comment on)


Besides, "Happily, there does not have to be One Correct Answer here.
Let wallets compete, and may the best user experience win..."

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]

On Tue, Dec 03, 2013 at 11:40:35AM +1000, Gavin Andresen wrote:
> On Tue, Dec 3, 2013 at 12:44 AM, Mike Hearn <mike@plan99.net> wrote:
> 
> > PPv1 doesn't have any notion of fee unfortunately. I suppose it could be
> > added easily, but we also need to launch the existing feature set.
> >
> 
> Lets bang out a merchant-pays-fee extension.
> 
> How about:
> 
> SPEC:
> 
>     optional uint64 allowfee    tag number=1000
> 
> Allow up to allowfee satoshis to be deducted from the amount paid to be
> used to pay Bitcoin network transaction fees. A wallet implementation must
> not reduce the amount paid for fees more than allowfee, and transaction
> fees must be equal to or greater than the amount reduced.

Fees are per byte of tx data; call it allowfeeperkb, and given that fees
are required - the merchant would really rather not waste up to about
twice as much on fees for a child-pays-for-parent - it should be called
requirefeeperkb.

Back to your point, the merchant wants to limit total fees that have
been deducted - 'allowfee' is still a good idea - but only in
conjunction with specifying fee-per-kb requirements.

UI once both are implemented is to not show anything in the default
case, and explain to the user why they have to pay extra in the unusual
case where they are spending a whole bunch of dust.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Drak]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

On 3 December 2013 11:03, Peter Todd <pete@petertodd.org> wrote:

> UI once both are implemented is to not show anything in the default
> case, and explain to the user why they have to pay extra in the unusual
> case where they are spending a whole bunch of dust.


Yes, that's the other problem with a merchant setting a fee - they have no
idea how large the transaction might be. If you spend a bunch of dust the
fee could be 2 or 3x the expected fee. Then you might get merchants
including higher fees by default to account for this. That means we end up
paying more per kb over time.

[-- Attachment #2: Type: text/html, Size: 976 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On Tue, Dec 03, 2013 at 11:09:51AM +0000, Drak wrote:
> On 3 December 2013 11:03, Peter Todd <pete@petertodd.org> wrote:
> 
> > UI once both are implemented is to not show anything in the default
> > case, and explain to the user why they have to pay extra in the unusual
> > case where they are spending a whole bunch of dust.
> 
> 
> Yes, that's the other problem with a merchant setting a fee - they have no
> idea how large the transaction might be. If you spend a bunch of dust the
> fee could be 2 or 3x the expected fee. Then you might get merchants
> including higher fees by default to account for this. That means we end up
> paying more per kb over time.

Right, which is solved by requiring a fee-per-kb, and only allowing up
to a certain amount to be deducted from the amount paid to pay that
total fee.


But really, we're better off leaving fees visible to the user in the
first place: they're how Bitcoin works and it's not going to change.
(was just talking to Taylor from Hive Wallet about that in person
actually)

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[kjj]

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]

After reading all 99 messages in this thread, I think allowfee is just 
about perfect.

It effectively lets merchants to give an allowance against the purchase 
price for network fees, if they choose.  It is still up to the sender 
(and/or the sender's software) to get the fees right.  Sometimes the 
sender will need to pay more fees than allowed, and sometimes the sender 
will need to pay less.

We can't solve the fee problem, in general.  I'm not sure that we can 
even define it properly.  But this is something that we can do, that 
will be useful at least occasionally, and that will cause no harm the 
rest of the time.

P.S.  Clever senders can use this to defrag their wallets.  Who wants to 
write the patch for that?

Gavin Andresen wrote:
> On Tue, Dec 3, 2013 at 12:44 AM, Mike Hearn <mike@plan99.net 
> <mailto:mike@plan99.net>> wrote:
>
>     PPv1 doesn't have any notion of fee unfortunately. I suppose it
>     could be added easily, but we also need to launch the existing
>     feature set.
>
>
> Lets bang out a merchant-pays-fee extension.
>
> How about:
>
> SPEC:
>
>     optional uint64 allowfee    tag number=1000
>
> Allow up to allowfee satoshis to be deducted from the amount paid to 
> be used to pay Bitcoin network transaction fees. A wallet 
> implementation must not reduce the amount paid for fees more than 
> allowfee, and transaction fees must be equal to or greater than the 
> amount reduced.
>
> :ENDSPEC
>
> Rationale: we don't want wallet software giving users discounts-- 
> sending transactions that are amount-allowfee without paying any fee. 
>  We also want to allow users to pay MORE in fees, if they need to 
> (fragmented wallet, maybe, or big CoinJoin transaction) or decide to.
>
>
> PS: I think there was also consensus that the BIP72  request=...   
> should be shortened to just r=... (save 6 chars in QR codes).  Unless 
> somebody objects, I'll change the BIP and the reference implementation 
> code to make it so...
>
> -- 
> --
> Gavin Andresen
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Type: text/html, Size: 4989 bytes --]

Re: [Bitcoin-development] Floating fees and SPV clients

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]

On Sun, Dec 01, 2013 at 12:51:46PM +0100, Mike Hearn wrote:
> 4) Finally, when we next hard fork, we make v2 transactions include the
> output value in the signature, same as the output script (this proposal has
> been on the forums for a while now). That allows the fee data added in step
> 2 to be cross-checked against the signatures on the inputs, thus
> authenticating it.

FTFY s/hard-fork/soft-fork/

The proposals on the forums with regard to a soft-fork are for a new
transaction format - to be done in parallel with the old one - that
commits to transaction inputs and outputs via a merkle tree extending
into the transaction. This data is then committed to via the merge-mine
standard.


In addition, all this discussion about trying to figure out transaction
fees based on transaction is a bit irrelevant if you suppose a
soft-fork. We already know that we need a merkle-sum-tree of fees and
transaction size to be able to audit miners and create compact fraud
proofs, and using that merkle-sum-tree you can easily calculate sum
fees/sum size for the whole block to determine fees.

We know that we can make a good assumption that transactions in
blocks will have been broadcast prior by the assumption you and Gavin
are making that miners would by far prefer to mine transactions that
have already been broadcast so that block propagation via txid lists
works. That advantage is on the order of 10x to 100x (or even higher)
lower cost per KB to the miner. (if this is not true, let us all know,
because then your scalability arguments are bunk with regard to
blocksize)

In addition faking fees via non-disclosed high-fee transactions has a
cost of about 1% due to the risk your block gets orphaned and the tx fee
gets mined by another miner.


Between those two the merkle-sum-trees for fees and size can be used for
various levels of average fee for a whole block, plus statistical
distributions.


Next it is also desirable for another, related, soft-fork to include a
sorted list of txids and/or H(scriptPubKey)'s in a block. (the latter is
explicitly part of my TXO commitments proposal) With the txids version
especially it's easy and low-bandwidth for SPV nodes to get proof that a
given transaction has been mined recently, by asking for proof of
inclusion or exclusion to accompany a bloom filter match.


Finally with various anti-sybil techniques that create long-term
identities it's easy to show that a peer lied about the data required
for fee estimates anyway.

> One obvious concern is what to do if nodes don't converge on very similar
> estimates. Wallets will always want to pay the lowest fee possible, so that
> means they'll always be riding the very edge of what's acceptable, opening
> up tx propagation to random flaky failures if fee estimates change whilst a
> transaction is in progress, or if some nodes don't calculate the same
> estimates as others.

Propagation failures are not a major problem if transactions can be
rebroadcast with new, higher, fees; propagation failures can be detected
easily and quickly with my proof-of-tx-mining proposal.

-- 
'peter'[:-1]@petertodd.org
000000000000000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]