[bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[Alex Morcos]

[-- Attachment #1: Type: text/plain, Size: 4700 bytes --]

Hi,

I'm proposing the addition of a new optional p2p message to help reduce
unnecessary network traffic.  The draft BIP is available here and pasted
below:
https://gist.github.com/morcos/9aab223c443c9258c979

The goal of this message is to take advantage of the fact that when a node
has reached its mempool limit, there is a minimum fee below which no
transactions are accepted to the mempool.  Informing peers of this minimum
would save them inv'ing your node for those transaction id's and save your
node requesting them if they are not in your recentRejects filter.

This message is optional and may be ignored as a protocol rule.  There is
also an option to turn off sending the messages in the implementation.

Thanks to Suhas Daftuar, Greg Maxwell, and others for helping develop the
idea.

-Alex

Draft BIP text:

<pre>
  BIP: <unassigned>
  Title: feefilter message
  Author: Alex Morcos <morcos@chaincode.com>
  Status: Draft
  Type: Standards Track
  Created: 2016-02-13
</pre>

==Abstract==

Add a new message, "feefilter", which serves to instruct peers not to send
"inv"'s to the node for transactions with fees below the specified fee rate.

==Motivation==

The concept of a limited mempool was introduced in Bitcoin Core 0.12 to
provide protection against attacks or spam transactions of low fees that
are not being mined. A reject filter was also introduced to help prevent
repeated requests for the same transaction that might have been recently
rejected for insufficient fee. These methods help keep resource utilization
on a node from getting out of control.

However, there are limitations to the effectiveness of these approaches.
The reject filter is reset after every block which means transactions that
are inv'ed over a longer time period will be rerequested and there is no
method to prevent requesting the transaction the first time.  Furthermore,
inv data is sent at least once either to or from each peer for every
transaction accepted to the mempool and there is no mechanism by which to
know that an inv sent to a given peer would not result in a getdata request
because it represents a transaction with too little fee.

After receiving a feefilter message, a node can know before sending an inv
that a given transaction's fee rate is below the minimum currently required
by a given peer, and therefore the node can skip relaying an inv for that
transaction to that peer.

==Specification==

# The feefilter message is defined as a message containing an int64_t where
pchCommand == "feefilter"
# Upon receipt of a "feefilter" message, the node will be permitted, but
not required, to filter transaction invs for transactions that fall below
the feerate provided in the feefilter message interpreted as satoshis per
kilobyte.
# The fee filter is additive with a bloom filter for transactions so if an
SPV client were to load a bloom filter and send a feefilter message,
transactions would only be relayed if they passed both filters.
# Inv's generated from a mempool message are also subject to a fee filter
if it exists.
# Feature discovery is enabled by checking protocol version >= 70013

==Considerations==
The propagation efficiency of transactions across the network should not be
adversely affected by this change. In general, transactions which are not
accepted to your mempool are not relayed and the funcionality implemented
with this message is meant only to filter those transactions.  There could
be a small number of edge cases where a node's mempool min fee is actually
less than the filter value a peer is aware of and transactions with fee
rates between these values will now be newly inhibited.

Feefilter messages are not sent to whitelisted peers if the
"-whitelistforcerelay" option is set. In that case, transactions are
intended to be relayed even if they are not accepted to the mempool.

There are privacy concerns with deanonymizing a node by the fact that it is
broadcasting identifying information about its mempool min fee. To help
ameliorate this concern, the implementaion quantizes the filter value
broadcast with a small amount of randomness, in addition, the messages are
broadcast to different peers at individually randomly distributed times.

If a node is using prioritisetransaction to accept transactions whose
actual fee rates might fall below the node's mempool min fee, it may want
to consider setting "-nofeefilter" to make sure it is exposed to all
possible txid's.

==Backward compatibility==

Older clients remain fully compatible and interoperable after this change.
The sending of feefilter messages can be disabled by unsetting the
"-feefilter" option.

==Implementation==

https://github.com/bitcoin/bitcoin/pull/7542

[-- Attachment #2: Type: text/html, Size: 5753 bytes --]

Re: [bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[Luke Dashjr]

On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:
> # The feefilter message is defined as a message containing an int64_t where
> pchCommand == "feefilter"

What happened to extensibility? And why waste 64 bits for what is almost 
certainly a small number?

> # The fee filter is additive with a bloom filter for transactions so if an
> SPV client were to load a bloom filter and send a feefilter message,
> transactions would only be relayed if they passed both filters.

This seems to make feefilter entirely useless for wallets?

Luke

Re: [bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[Alex Morcos]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

On Tue, Feb 16, 2016 at 6:46 PM, Luke Dashjr <luke@dashjr.org> wrote:

> On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:
> > # The feefilter message is defined as a message containing an int64_t
> where
> > pchCommand == "feefilter"
>
> What happened to extensibility? And why waste 64 bits for what is almost
> certainly a small number?
>

I thought that extensibility was already sufficient with the command string
system.  If we come up with a better version of the feefilter later we can
just give it a different command name.  This seemed to encapsulate a fairly
complete idea for now.  As for the 8 bytes, it didn't seem necessary to me
to over optimize with a custom encoding for what amounts to well under 20%
of ping traffic.  (pings are sent every 2 mins per peer, feefilters on
average every 10 mins, but when the quantized value to be sent would be the
same it is skipped)


> > # The fee filter is additive with a bloom filter for transactions so if
> an
> > SPV client were to load a bloom filter and send a feefilter message,
> > transactions would only be relayed if they passed both filters.
>
> This seems to make feefilter entirely useless for wallets?
>
>
I don't follow this comment?  Transactions aren't synced with the wallet
unless they are accepted into the mempool.  Sending feefilter messages
should not reduce the number of transactions that are accepted to the
mempool.


> Luke
>

[-- Attachment #2: Type: text/html, Size: 2276 bytes --]

Re: [bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[Luke Dashjr]

On Wednesday, February 17, 2016 2:28:31 AM Alex Morcos wrote:
> On Tue, Feb 16, 2016 at 6:46 PM, Luke Dashjr <luke@dashjr.org> wrote:
> > On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev 
wrote:
> > > # The feefilter message is defined as a message containing an int64_t
> > 
> > where
> > 
> > > pchCommand == "feefilter"
> > 
> > What happened to extensibility? And why waste 64 bits for what is almost
> > certainly a small number?
> 
> I thought that extensibility was already sufficient with the command string
> system.  If we come up with a better version of the feefilter later we can
> just give it a different command name.

We shouldn't need a new protocol [extension] for every new policy. Obviously 
this can't be perfectly flexible, but supporting different feerate definition 
versions is trivial and obvious.

> > > # The fee filter is additive with a bloom filter for transactions so if
> > an
> > > SPV client were to load a bloom filter and send a feefilter message,
> > > transactions would only be relayed if they passed both filters.
> > 
> > This seems to make feefilter entirely useless for wallets?
> 
> I don't follow this comment?  Transactions aren't synced with the wallet
> unless they are accepted into the mempool.  Sending feefilter messages
> should not reduce the number of transactions that are accepted to the
> mempool.

In Core, they aren't (but Core never uses bloom filters anyway) - because 
otherwise it would leak privacy. But light clients (particularly overlapping 
with those that use bloom filters!) have no privacy in the first place, so 
they have no reason to use this rule.

Luke

Re: [bitcoin-dev] [BIP Proposal] New "feefilter" p2p message

[Gregory Maxwell]

On Wed, Feb 17, 2016 at 12:46 AM, Luke Dashjr via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:
>> # The feefilter message is defined as a message containing an int64_t where
>> pchCommand == "feefilter"
>
> What happened to extensibility?

I did think it might be interesting to do a priorityrate filter. but
since it seems no one is even working on adding an index for ancestor
priorityrate... or working on a space limited priority mempool... if
that extension would be needed it could just be a new "priorityfilt"
command.

> And why waste 64 bits for what is almost
> certainly a small number?

Technically fees per byte could be greater than 32 bits (e.g. a 9000
BTC fee is enough). Values are normally 64 bits already.

>> # The fee filter is additive with a bloom filter for transactions so if an
>> SPV client were to load a bloom filter and send a feefilter message,
>> transactions would only be relayed if they passed both filters.
>
> This seems to make feefilter entirely useless for wallets?

I think your reasoning is that you want to learn of your own
transactions even if they don't meet the filter?

I'm not sure this reasoning plays out though-- regardless of what your
own feefilter is, if a tx has too low a rate for your peers to relay
it, they won't and you won't learn of it.

I might wave my hands at a use case for OR "I will relay very high fee
txn ... or my own"; but considering that performance/privacy disaster
that bloom filters are-- and that to relay third party txn at all you
need to be able to validate them (or will get yourself banned).  Also,
if you really want the OR behavior you could make two connections...
the same cannot be said for and.

Maybe one argument I could add is that if we added a priorityrate
filter, that one would be an OR