[Bitcoin-development] BIP70 message delivery reliability

[Bitcoin-development] BIP70 message delivery reliability

[Chuck]

I spoke briefly with Peter (sipa). He recommend I forward this post to 
the mailing list for further discussion.

My apologies if this has been discussed before, but I was curious about 
some things re BIP70 message delivery.  In particular, I don't clearly 
see the value of the PaymentACK message.  Allow me to explain...

The current BIP70 workflow designates PaymentACK as the final message in 
a payment exchange. However, it doesn't appear that any mention is made 
of what happens if that delivery fails. I assume that re-delivery is 
left as a detail to the implementation, actually.

For sake of argument, let's assume that PaymentACK is never delivered 
either because of a network outage or a malicious merchant or 
incompatible software between wallets or a bug.  I ask myself: what 
would be necessary for sufficient proof of payment, say, to an arbiter?  
I presume the receipt R=(PaymentRequest,[transactions]) would suffice.  
Am I correct there?

But if the PaymentRequest and broadcasted transactions are enough to 
prove payment, what's the point of the Payment message? The merchant 
never has to verify the Payment message, possibly maliciously ignoring 
it.  In the well-behaved case, I presume the point is to help the 
merchant associate some arbitrary data with the purchase as well as 
provide a refunding address for the customer.  If that's the case, 
couldn't this protocol be slightly improved like so:

Required steps:
1. Customer clicks "pay now"
2. Merchant sends PaymentRequest/PaymentDetails, which should be signed
3. Customer builds a set of transactions and sends a new 
PaymentApprovalRequest message which includes a refund address and the 
unsigned transactions and their associated fully-signed transaction 
hash, the whole message signed with the private key of the refund address.
4. Merchant responds with PaymentApproved message, signing the 
PaymentApprovalRequest message with the same key from step 2.

Optional steps:
5. The customer can send a Payment message, which is only a set of 
signed transactions.
6. The merchant can respond with a PaymentACK message.

In Step 4, the merchant is acknowledging that if the transactions 
provided PaymentApprovalRequest are broadcast, then payment is complete 
and no other steps are required. Steps 5 and 6 aren't required but are 
considered considerate:)

After step 4, all the merchant needs is to do is watch for the 
transactions that were listed in PaymentApprovalRequest.  The 
(PaymentApproved,[signed transactions]) pair is the customer's proof of 
payment and this proof of payment includes a refund address that the 
merchant has agreed to prior to payment, instead of after.  Step 3 & 4 
also allow the merchant to verify transactions, providing an extra layer 
of redundancy.  The merchant will also be able to ack on fees, time lock 
(time sensitive purchases?), sequence numbers, etc.

In Step 3, it's critical the customer sign the message with the private 
key of the refund address, so that the merchant can be confident the 
refund address is correct.

In each step along the way until step 5, if a message delivery fails 
nobody is harmed because the purchase is incomplete.

Thoughts?

Chuck

Re: [Bitcoin-development] BIP70 message delivery reliability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

Hi Chuck,

Both Bitcoin Core and bitcoinj are about to ship with the protocol as-is,
so any changes from this point on have to be backwards compatible.

On Thu, Jan 30, 2014 at 6:47 AM, Chuck <chuck+bitcoindev@borboggle.com>wrote:

> I presume the receipt R=(PaymentRequest,[transactions]) would suffice.
>

That's all you need to prove payment, yes.


> In the well-behaved case, I presume the point is to help the
> merchant associate some arbitrary data with the purchase as well as
> provide a refunding address for the customer.


That's right (+memo). And to provide an additional hook for future
features, like recurring billing, ECDH key agreements etc.


> In Step 3, it's critical the customer sign the message with the private
> key of the refund address, so that the merchant can be confident the
> refund address is correct.
>

Refund addresses as specced currently are optional. For instance bitcoinj
currently doesn't use them and won't until HD wallets support is done.

Let's get some practical experience with what we've got so far. We can
evolve PaymentRequest/Payment/PaymentACK in the right direction with
backwards compatible upgrades, I am hoping.

[-- Attachment #2: Type: text/html, Size: 1903 bytes --]

Re: [Bitcoin-development] BIP70 message delivery reliability

[Chuck]

Hi Mike.  Thanks for replying.

On 1/30/2014 5:49 PM, Mike Hearn wrote:
> Both Bitcoin Core and bitcoinj are about to ship with the protocol 
> as-is, so any changes from this point on have to be backwards compatible.
Then I think it's critically important to talk about failure situations 
now, rather than trying to patch on solutions later; it's going to be 
very hard to wedge/"hack" in fixes for potential problems when they 
could be addressed now with minor changes.
> Let's get some practical experience with what we've got so far. We can 
> evolve PaymentRequest/Payment/PaymentACK in the right direction with 
> backwards compatible upgrades, I am hoping.
I think what I'm trying to discuss or find out here is whether the 
current PP description is defunct or incomplete in some manner, thus 
making any experience we gain from the current implementation moot.

It seems the largest hole in the implementation is delivery of the 
Payment message, but I'm happy to accept that maybe I'm just missing 
something.  A malicious merchant could claim he never received the 
Payment message, or a faulty network connection could cause the message 
to never be delivered. In arbitration the merchant could argue the 
transactions seen on the network were insufficient.

To me, this could be a problem.

Cheers,

Chuck

Re: [Bitcoin-development] BIP70 message delivery reliability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle.com>wrote:

> In arbitration the merchant could argue the transactions seen on the
> network were insufficient.
>

The arbitrator would presumably have some rules about what is or isn't an
acceptable form of payment.

HTTP has response codes for submission of the Payment message. We could add
signing to PaymentACK and other things in future, if that turns out to be
insufficient in practice.

[-- Attachment #2: Type: text/html, Size: 878 bytes --]

Re: [Bitcoin-development] BIP70 message delivery reliability

[Chuck]

On 1/30/2014 6:31 PM, Mike Hearn wrote:
> The arbitrator would presumably have some rules about what is or isn't 
> an acceptable form of payment.
Do you think this puts unnecessary trust into a third party?  If the 
merchant instead signed and agreed to the unsigned transactions before 
they were broadcast (as in my OP), these arbitration concerns disappear.

> HTTP has response codes for submission of the Payment message. We 
> could add signing to PaymentACK and other things in future, if that 
> turns out to be insufficient in practice.
HTTP isn't the only message delivery mechanism.  Merchants can also lie: 
reply with 200 OK and an empty body.  Or, reply with 404 not found and 
broadcast transactions anyway.

Cheers,

Chuck

Re: [Bitcoin-development] BIP70 message delivery reliability

[Pieter Wuille]

On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle.com> wrote:
> Hi Mike.  Thanks for replying.
>
> On 1/30/2014 5:49 PM, Mike Hearn wrote:
>> Both Bitcoin Core and bitcoinj are about to ship with the protocol
>> as-is, so any changes from this point on have to be backwards compatible.
> Then I think it's critically important to talk about failure situations
> now, rather than trying to patch on solutions later; it's going to be
> very hard to wedge/"hack" in fixes for potential problems when they
> could be addressed now with minor changes.
>> Let's get some practical experience with what we've got so far. We can
>> evolve PaymentRequest/Payment/PaymentACK in the right direction with
>> backwards compatible upgrades, I am hoping.
> I think what I'm trying to discuss or find out here is whether the
> current PP description is defunct or incomplete in some manner, thus
> making any experience we gain from the current implementation moot.
>
> It seems the largest hole in the implementation is delivery of the
> Payment message, but I'm happy to accept that maybe I'm just missing
> something.  A malicious merchant could claim he never received the
> Payment message, or a faulty network connection could cause the message
> to never be delivered. In arbitration the merchant could argue the
> transactions seen on the network were insufficient.

You don't even have to assume malicious intent. A payment message
could just fail to arrive because the server is unreachable. As the
specification currently doesn't even suggest retrying, there is no way
the merchant can rely at all on the memo and refund address being
delivered, which makes them in my opinion useless.

Your proposal makes the whole protocol more atomic, which may be a
step too far at this point (though I like the idea very much), but I
really think the specification should do everything possible to
prevent transactions confirming without the payment message ever being
delivered (i.e., store them in the sender's client, retry when
necessary, exponential backoff, ...).

-- 
Pieter

Re: [Bitcoin-development] BIP70 message delivery reliability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

With the way it works in bitcoinj, the tx is only committed to the wallet
if the server accepts the Payment message and ACKs it. So the tx would not
be retried if there's a failure submitting or some kind of internal server
error, and the UI would show that the payment failed. That seems
straightforward and how I'd expect things to work as a user.


On Thu, Jan 30, 2014 at 12:46 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:

> On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle.com>
> wrote:
> > Hi Mike.  Thanks for replying.
> >
> > On 1/30/2014 5:49 PM, Mike Hearn wrote:
> >> Both Bitcoin Core and bitcoinj are about to ship with the protocol
> >> as-is, so any changes from this point on have to be backwards
> compatible.
> > Then I think it's critically important to talk about failure situations
> > now, rather than trying to patch on solutions later; it's going to be
> > very hard to wedge/"hack" in fixes for potential problems when they
> > could be addressed now with minor changes.
> >> Let's get some practical experience with what we've got so far. We can
> >> evolve PaymentRequest/Payment/PaymentACK in the right direction with
> >> backwards compatible upgrades, I am hoping.
> > I think what I'm trying to discuss or find out here is whether the
> > current PP description is defunct or incomplete in some manner, thus
> > making any experience we gain from the current implementation moot.
> >
> > It seems the largest hole in the implementation is delivery of the
> > Payment message, but I'm happy to accept that maybe I'm just missing
> > something.  A malicious merchant could claim he never received the
> > Payment message, or a faulty network connection could cause the message
> > to never be delivered. In arbitration the merchant could argue the
> > transactions seen on the network were insufficient.
>
> You don't even have to assume malicious intent. A payment message
> could just fail to arrive because the server is unreachable. As the
> specification currently doesn't even suggest retrying, there is no way
> the merchant can rely at all on the memo and refund address being
> delivered, which makes them in my opinion useless.
>
> Your proposal makes the whole protocol more atomic, which may be a
> step too far at this point (though I like the idea very much), but I
> really think the specification should do everything possible to
> prevent transactions confirming without the payment message ever being
> delivered (i.e., store them in the sender's client, retry when
> necessary, exponential backoff, ...).
>
> --
> Pieter
>

[-- Attachment #2: Type: text/html, Size: 3332 bytes --]

Re: [Bitcoin-development] BIP70 message delivery reliability

[Pieter Wuille]

On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99.net> wrote:
> With the way it works in bitcoinj, the tx is only committed to the wallet if
> the server accepts the Payment message and ACKs it. So the tx would not be
> retried if there's a failure submitting or some kind of internal server
> error, and the UI would show that the payment failed. That seems
> straightforward and how I'd expect things to work as a user.

That's one right way to do it imho, but not what is suggested or
required by the specification, and not what bitcoin core master
currently implements.

-- 
Pieter

Re: [Bitcoin-development] BIP70 message delivery reliability

[Chuck]

On 1/30/2014 7:02 PM, Pieter Wuille wrote:
> On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99.net> wrote:
>> With the way it works in bitcoinj, the tx is only committed to the wallet if
>> the server accepts the Payment message and ACKs it. So the tx would not be
>> retried if there's a failure submitting or some kind of internal server
>> error, and the UI would show that the payment failed. That seems
>> straightforward and how I'd expect things to work as a user.
> That's one right way to do it imho, but not what is suggested or
> required by the specification, and not what bitcoin core master
> currently implements.
>
If you sent the Payment message and the server goes silent after 
receiving it, you retry to delivery.  However, the merchant can 
broadcast the transactions and force them into your wallet anyway. You 
could, quite likely, pay and never get an ACK.

Re: [Bitcoin-development] BIP70 message delivery reliability

[Roy Badami]

On Thu, Jan 30, 2014 at 07:03:57PM +0700, Chuck wrote:
> On 1/30/2014 7:02 PM, Pieter Wuille wrote:
> > On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99.net> wrote:
> >> With the way it works in bitcoinj, the tx is only committed to the wallet if
> >> the server accepts the Payment message and ACKs it. So the tx would not be
> >> retried if there's a failure submitting or some kind of internal server
> >> error, and the UI would show that the payment failed. That seems
> >> straightforward and how I'd expect things to work as a user.
> > That's one right way to do it imho, but not what is suggested or
> > required by the specification, and not what bitcoin core master
> > currently implements.
> >
> If you sent the Payment message and the server goes silent after 
> receiving it, you retry to delivery.  However, the merchant can 
> broadcast the transactions and force them into your wallet anyway. You 
> could, quite likely, pay and never get an ACK.

I think in that case, you absolultely have to invalidate all the
transactions in the Payment message by broadcasting a send-to-self
transaction as soon as possible; until that point your wallet balance
is indeterminate.  Otherwise what will happen if the merchant did in
fact receive the Payment message, and then processes it (and
broadcasts the transaction) after some delay?

Then what the user will see is: an apparently failed attempt to pay,
leaving their wallet balance unchanged.  Then, perhaps many hours
later, their wallet balance will appear to spontaneously decrement.

roy

Re: [Bitcoin-development] BIP70 message delivery reliability

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

>
> If you sent the Payment message and the server goes silent after receiving
> it, you retry to delivery.  However, the merchant can broadcast the
> transactions and force them into your wallet anyway. You could, quite
> likely, pay and never get an ACK.
>

No retries. If there's a timeout the wallet will consider the payment not
made, and if the merchant broadcasts anyway, the wallet will see the
transactions and sync with them correctly. The ACK is not particularly
important in the current design, so that's no big deal.

If we see this situation crop up routinely we can take measures to improve
things. I doubt we will.

[-- Attachment #2: Type: text/html, Size: 883 bytes --]