* Re: [bitcoin-dev] Improving RBF Policy @ 2022-02-01 2:47 Prayank 2022-02-01 9:30 ` Bastien TEINTURIER 0 siblings, 1 reply; 25+ messages in thread From: Prayank @ 2022-02-01 2:47 UTC (permalink / raw) To: Bastien TEINTURIER; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 1133 bytes --] Hi Bastein, > This work will highly improve the security of any multi-party contract trying to build on top of bitcoin Do you think such multi party contracts are vulnerable by design considering they rely on policy that cannot be enforced? > For starters, let me quickly explain why the current rules are hard to work with in the context of lightning Using the term 'rules' can be confusing sometimes because it's just a policy and different from consensus rules. I wish we could change this in the BIP with something else. > I'm actually paying a high fee twice instead of once (and needlessly using on-chain space, our scarcest asset, because we could have avoided that additional transaction Not sure I understand this part because if a transaction is on-chain it can't be replaced. > The second biggest pain point is rule 3. It prevents me from efficiently using my capital while it's unconfirmed > I'm curious to hear other people's thoughts on that. If it makes sense, I would propose the following very simple rules Looks interesting however not sure about X and Y. -- Prayank A3B1 E430 2298 178F [-- Attachment #2: Type: text/html, Size: 1828 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-01 2:47 [bitcoin-dev] Improving RBF Policy Prayank @ 2022-02-01 9:30 ` Bastien TEINTURIER 2022-02-02 10:21 ` Anthony Towns 0 siblings, 1 reply; 25+ messages in thread From: Bastien TEINTURIER @ 2022-02-01 9:30 UTC (permalink / raw) To: Prayank, Anthony Towns; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 4381 bytes --] Hi AJ, Prayank, > I think that's backwards: we're trying to discourage people from wasting > the network's bandwidth, which they would do by publishing transactions > that will never get confirmed -- if they were to eventually get confirmed > it wouldn't be a waste of bandwith, after all. But if the original > descendent txs were that sort of spam, then they may well not be > submitted again if the ancestor tx reaches a fee rate that's actually > likely to confirm. But do you agree that descendants only matter for DoS resistance then, not for miner incentives? I'm asking this because I think a new set of policies should separate policies that address the miner incentives from policies that address the DoS issues. The two policies I proposed address miner incentives. I think they're insufficient to address DoS issues. But adding a 3rd policy to address DoS issues may be a good solution? I think that rate-limiting p2p as you suggest (and Gloria also mentioned it) is likely a better way of fixing the DoS concerns than a descendant rule like BIP 125 rule 5 (which as I mentioned earlier, is problematic because the descendent set varies from one mempool to another). I would like to add a small update to my policy suggestions. The X and Y percentage increase should be met for both the ancestor scores AND the transaction in isolation. Otherwise I could replace txA with txA' that uses a new ancestor txB that has a high fee and high feerate, while txA' has a low fee and low feerate. It's then possible for txB to confirm without txA', and what would remain then in the mempool would be worse than before the replacement. > All you need is for there to be *a* path that follows the new relay rules > and gets from your node/wallet to perhaps 10% of hashpower, which seems > like something wallet providers could construct relatively quickly? That's true, maybe we can be more optimistic about the timeline for using an updated set of policies ;) > Do you think such multi party contracts are vulnerable by design > considering they rely on policy that cannot be enforced? It's a good question. Even though these policies cannot be enforced, if they are rational to apply by nodes, I think it's ok to rely on them. Others may disagree with that, but I guess it's worth a separate thread. > Not sure I understand this part because if a transaction is on-chain > it can't be replaced. Sorry, that was a bit unclear. Suppose I have txA that I want to RBF, but I only have unconfirmed utxos and I can't simply lower its existing outputs to reach my desired feerate. I must make one of my unconfirmed utxos confirm asap just to be able to use it to RBF txA. That means I'll need to pay fees a first time just to convert one of my unconfirmed utxos to a confirmed one. Then I'll pay the fees to bump txA. I had to overpay fees compared to just using my unconfirmed utxo in the first place (and manage more complexity to track the confirmation of my unconfirmed utxo). Thanks for your feedback! Bastien Le mar. 1 févr. 2022 à 03:47, Prayank <prayank@tutanota.de> a écrit : > Hi Bastein, > > > This work will highly improve the security of any multi-party contract > trying to build on top of bitcoin > > Do you think such multi party contracts are vulnerable by design > considering they rely on policy that cannot be enforced? > > > For starters, let me quickly explain why the current rules are hard to > work with in the context of lightning > > Using the term 'rules' can be confusing sometimes because it's just a > policy and different from consensus rules. I wish we could change this in > the BIP with something else. > > > I'm actually paying a high fee twice instead of once (and needlessly > using on-chain space, our scarcest asset, because we could have avoided > that additional transaction > > Not sure I understand this part because if a transaction is on-chain it > can't be replaced. > > > The second biggest pain point is rule 3. It prevents me from efficiently > using my capital while it's unconfirmed > > > I'm curious to hear other people's thoughts on that. If it makes sense, > I would propose the following very simple rules > > Looks interesting however not sure about X and Y. > > > -- > Prayank > > A3B1 E430 2298 178F > [-- Attachment #2: Type: text/html, Size: 5455 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-01 9:30 ` Bastien TEINTURIER @ 2022-02-02 10:21 ` Anthony Towns 0 siblings, 0 replies; 25+ messages in thread From: Anthony Towns @ 2022-02-02 10:21 UTC (permalink / raw) To: Bastien TEINTURIER, Bitcoin Protocol Discussion On Tue, Feb 01, 2022 at 10:30:12AM +0100, Bastien TEINTURIER via bitcoin-dev wrote: > But do you agree that descendants only matter for DoS resistance then, > not for miner incentives? There's an edge case where you're replacing tx A with tx X, and X's fee rate is higher than A's, but you'd be obsoleting descendent txs (B, C, D...) and thus replacing them with unrelated txs (L, M, N...), and the total feerate/fees of A+B+C+D... is nevertheless higher than X+L+M+N... But I think that's probably unusual (transactions D and L are adjacent in the mempool, that's why L is chosen for the block; but somehow there's a big drop off in value somewhere between B/C/D and L/M/N), and at least today, I don't think miners consider it essential to eke out every possible sat in fee income. (If, as per your example, you're actually replacing {A,B,C,D} with {X,Y,Z,W} where X pays higher fees than A and the package in total pays either the same or higher fees, that's certainly incentive compatible. The tricky question is what happens when X arrives on its own and it might be that no one ever sends a replacement for B,C,D) > The two policies I proposed address miner incentives. I think they're > insufficient to address DoS issues. But adding a 3rd policy to address > DoS issues may be a good solution? >>> 1. The transaction's ancestor absolute fees must be X% higher than the >>> previous transaction's ancestor fees >>> 2. The transaction's ancestor feerate must be Y% higher than the >>> previous transaction's ancestor feerate Absolute fees only matter if your backlog's feerate drops off. If you've got 100MB of txs offering 5sat/vb, then exchanging 50kB at 5sat/vb for 1kB at 6sat/vb is still a win: your block gains 1000 sats in fees even though your mempool loses 245,000 sats in fees. But if your backlog's feerate does drop off, *and* that matters, then I don't think you can ignore the impact of the descendent transactions that you might not get a replacement for. I think "Y% higher" rather than just "higher" is only useful for rate-limiting, not incentive compatibility. (Though maybe it helps stabilise a greedy algorithm in some cases?) Cheers, aj ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy @ 2022-02-09 17:57 lisa neigut 0 siblings, 0 replies; 25+ messages in thread From: lisa neigut @ 2022-02-09 17:57 UTC (permalink / raw) To: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 7417 bytes --] Changing the way that RBF works is an excellent idea. Bitcoin is overdue for revisiting these rules. Thanks to @glowzo et al for kicking off the discussion. I've been thinking about RBF for a very long time[1], and it's been fun to see other people's thoughts on the topics. Here's my current thinking about this, and proposal for how we should update the rules. ### Changing the Rules? How to Change Them Regarding how to change them, Bram and aj are right -- we should move to a model where transaction relay candidates are evaluated on their net increase in fees per byte paid, and remove the requirement that the gross fee of the preceding transaction is met or exceeded. Our current ruleset is over complicated because it attempts to solve two problems at once: the miner's best interests (highest possible fee take) and relay policy. I believe this is a mistake and the mempool should change its goals. Instead, the mempool relay design for RBFs should be built around 1) increasing the the per-byte fees paid of a transaction and 2) providing a simple policy for applications building on top of bitcoin, such that knowledge of the mempool is not required for successfully issuing relay-able RBF transactions. (A simple "must increase the per-byte feerate" heuristic for RBF relay candidates has the nice benefit of being easy to program to on the application side, and only requires knowledge of the previous candidate transaction, not the entire mempool or any previous tx's relative position within it.) Finally, when blockspace is competitive , this simple policy ensures that the per-byte value of every subsequent relayed transaction increases the per-byte value of pending bytes for the next block. This provides a measure of DoS protection and ensures that every relayed byte is more valuable (to the miner/network) than the last. *The only time that RBF is critical for relay is during full block periods -- if there's not enough transactions to fill a block, using RBF to ensure that a transaction is mined in a timely manner is moot. As such, RBF rules should not be concerned with low-block environments. ### Mempools and Relay The mempool currently serves two masters: the profit motive of the miner and the relay motive of a utxo holder. It is in the interest of a user to send the miner a high per-byte tx, such that it might end up in the next block. It is in the miner's best interest to include the highest per-byte set of transactions in their block template. There is some conflation here in the current RBF policies between what is in the mempool and what is considered a candidate for the block template. If a miner has already included a more profitable package of txs into their block template than a more valuable per-byte tx that the network has relayed to them, it should be the responsibility of the block template constructor to reject the new proposed tx, not the nodes relaying the transaction to said miner. This is a policy that the miner can (and should) implement at the level of the template construction, however. Is it the responsibility of the mempool to provide the best "historical" block opportunity for a miner (e.g. the highest paying block given all txs it's ever seen)? I would say no, that the ability of a utxo owner to re-state the spend condition of a pending transaction is more important, from a use-case perspective, and that the mempool should concern itself solely with relaying increasingly more profitable bytes to miners. Let the miners concern themselves with deciding what the best policy for their own block construction is, and the mempool with relaying the highest value bytes for the network. Net-net, this will benefit everyone as it becomes easier for users to re-submit txs with increasingly greater feerates, creating more active competition for available blockspace as more applications are able to include it as a feature (and it works, reliable, as a relay mechanism). ### Packages and RBF Packages make the increasing per-byte rule less of a guarantee that increasing the per-byte value of a single transaction will net a given miner more fees than including the entire package. Let's decompose this a bit. It's helpful to think of tx packages as 'composable txs'. Basically when you consider a 'package' it is actually a large tx with sub-components, the individual txs. As a 'composed tx', you can calculate the per-byte feerate of the entire set. This is the number that you, as someone issuing an RBF, would need to beat in order to move your tx up in the pending block queue. RBF, however, is a transaction level policy: it allows you to replace any *one* component of a package, or tree, with the side effect of possibly invalidating other candidate txs. If the 'composed tx' (aka package) had a net per-byte value that was higher than the new replacement transaction because of a leaf tx that had an outsized per-byte feerate, then it would be more profitable for the miner to have mined the entire package rather than the replacement. This edge case complicates the picture for the miner. Considered from the viewpoint of the user issuing the RBF, however, it is far simpler. In the ideal case, a person is issuing an RBF because the previous tx tree, even with its high fee sub-component, was not a candidate for the next block. And, in some cases, increasing the sub-component's per-byte feerate will not achieve the goal of moving the tx any closer to being mined. It's only by increasing the feerate above the present feerate of the candidate plus desendents (tx package) that the transaction will advance in the queue. While not uncomplicated, this is a simple enough metric for a wallet to track, and does not require any knowledge of the current mempool to effectively surpass. It's the wallet's responsibility to track this though; failure to take descendants into account when deciding on the next per-byte feerate for an RBF *will* mean that your RBF will be ineffective at achieving the goal of getting your UTXO spent. Any wallet is incentivized to always provide a higher per-byte feerate than the 'composed tx' (tx and its descendants), so as to ensure an actual improvement in the unconfirmed transaction's position in the block queue, so to speak. Note that the v2 protocol for channel opens in lightning uses an RBF negotiation that adheres basically to these rules (ea transaction must have a strictly greater per-byte feerate). We enforce a rate of 65/64th as the required increase in feerate for each subsequent channel open transaction. https://github.com/lightning/bolts/pull/851/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR1154 ### RBF and DoS Admittedly, changing these rules will increase the number of times that any UTXO is eligible to be retransmitted (relayed) via the bitcoin node network. Strictly increasing the per-byte feerate however ensures that this re-relay is increasingly more expensive to the UTXO owner, however. ### in exitus These are the things I've been thinking about with regards to RBF. I hope they can help to highlight the challenges in the RBF design space a bit more clearly, as well as spelling out the case for using a simple heuristic such as "solely increasing per-byte feerate" as a good candidate for the revised RBF policy. ~niftynei [1] https://basicbitch.software/posts/2018-12-27-Explaining-Replace-By-Fee.html [-- Attachment #2: Type: text/html, Size: 15920 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
[parent not found: <mailman.19693.1643292568.8511.bitcoin-dev@lists.linuxfoundation.org>]
* [bitcoin-dev] Improving RBF policy [not found] <mailman.19693.1643292568.8511.bitcoin-dev@lists.linuxfoundation.org> @ 2022-01-31 22:54 ` Bram Cohen 2022-02-01 0:08 ` Eric Voskuil 2022-02-01 0:42 ` Antoine Riard 0 siblings, 2 replies; 25+ messages in thread From: Bram Cohen @ 2022-01-31 22:54 UTC (permalink / raw) To: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 1857 bytes --] Gloria Zhao wrote: > > This post discusses limitations of current Bitcoin Core RBF policy and > attempts to start a conversation about how we can improve it, > summarizing some ideas that have been discussed. Please reply if you > have any new input on issues to be solved and ideas for improvement! > Is it still verboten to acknowledge that RBF is normal behavior and disallowing it is the feature, and that feature is mostly there to appease some people's delusions that zeroconf is a thing? It seems a bit overdue to disrespect the RBF flag in the direction of always assuming it's on. > - **Incentive Compatibility**: Ensure that our RBF policy would not > accept replacement transactions which would decrease fee profits > of a miner. In general, if our mempool policy deviates from what is > economically rational, it's likely that the transactions in our > mempool will not match the ones in miners' mempools, making our > fee estimation, compact block relay, and other mempool-dependent > functions unreliable. Incentive-incompatible policy may also > encourage transaction submission through routes other than the p2p > network, harming censorship-resistance and privacy of Bitcoin payments. > There are two different common regimes which result in different incentivized behavior. One of them is that there's more than a block's backlog in the mempool in which case between two conflicting transactions the one with the higher fee rate should win. In the other case where there isn't a whole block's worth of transactions the one with higher total value should win. It would be nice to have consolidated logic which handles both, it seems the issue has to do with the slope of the supply/demand curve which in the first case is gentle enough to keep the one transaction from hitting the rate but in the second one is basically infinite. [-- Attachment #2: Type: text/html, Size: 2326 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF policy 2022-01-31 22:54 ` [bitcoin-dev] Improving RBF policy Bram Cohen @ 2022-02-01 0:08 ` Eric Voskuil 2022-02-01 8:32 ` Bram Cohen 2022-02-01 0:42 ` Antoine Riard 1 sibling, 1 reply; 25+ messages in thread From: Eric Voskuil @ 2022-02-01 0:08 UTC (permalink / raw) To: Bram Cohen, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 2159 bytes --] > On Jan 31, 2022, at 15:15, Bram Cohen via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: … > Is it still verboten to acknowledge that RBF is normal behavior and disallowing it is the feature, and that feature is mostly there to appease some people's delusions that zeroconf is a thing? It seems a bit overdue to disrespect the RBF flag in the direction of always assuming it's on. What flag? >> - **Incentive Compatibility**: Ensure that our RBF policy would not >> accept replacement transactions which would decrease fee profits >> of a miner. In general, if our mempool policy deviates from what is >> economically rational, it's likely that the transactions in our >> mempool will not match the ones in miners' mempools, making our >> fee estimation, compact block relay, and other mempool-dependent >> functions unreliable. Incentive-incompatible policy may also >> encourage transaction submission through routes other than the p2p >> network, harming censorship-resistance and privacy of Bitcoin payments. > > There are two different common regimes which result in different incentivized behavior. One of them is that there's more than a block's backlog in the mempool in which case between two conflicting transactions the one with the higher fee rate should win. In the other case where there isn't a whole block's worth of transactions the one with higher total value should win. These are not distinct scenarios. The rational choice is the highest fee block-valid subgraph of the set of unconfirmed transactions, in both cases (within the limits of what is computationally feasible of course). When collecting pooled txs the only issue is DoS protection, which is simply a question of what any given miner is willing to pay, in terms of disk space, to archive conflicts for the opportunity to optimize block reward. > It would be nice to have consolidated logic which handles both, it seems the issue has to do with the slope of the supply/demand curve which in the first case is gentle enough to keep the one transaction from hitting the rate but in the second one is basically infinite. [-- Attachment #2: Type: text/html, Size: 3237 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF policy 2022-02-01 0:08 ` Eric Voskuil @ 2022-02-01 8:32 ` Bram Cohen 2022-02-01 19:44 ` Eric Voskuil 0 siblings, 1 reply; 25+ messages in thread From: Bram Cohen @ 2022-02-01 8:32 UTC (permalink / raw) To: Eric Voskuil; +Cc: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 1437 bytes --] On Mon, Jan 31, 2022 at 4:08 PM Eric Voskuil <eric@voskuil.org> wrote: > > > On Jan 31, 2022, at 15:15, Bram Cohen via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > > Is it still verboten to acknowledge that RBF is normal behavior and > disallowing it is the feature, and that feature is mostly there to appease > some people's delusions that zeroconf is a thing? It seems a bit overdue to > disrespect the RBF flag in the direction of always assuming it's on. > > What flag? > The opt-in RBF flag in transactions. > There are two different common regimes which result in different > incentivized behavior. One of them is that there's more than a block's > backlog in the mempool in which case between two conflicting transactions > the one with the higher fee rate should win. In the other case where there > isn't a whole block's worth of transactions the one with higher total value > should win. > > These are not distinct scenarios. The rational choice is the highest fee > block-valid subgraph of the set of unconfirmed transactions, in both cases > (within the limits of what is computationally feasible of course). > It's weird because which of two or more conflicting transactions should win can oscillate back and forth depending on other stuff going on in the mempool. There's already a bit of that with child pays but this is stranger and has more oddball edge cases about which transactions to route. [-- Attachment #2: Type: text/html, Size: 2473 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF policy 2022-02-01 8:32 ` Bram Cohen @ 2022-02-01 19:44 ` Eric Voskuil 0 siblings, 0 replies; 25+ messages in thread From: Eric Voskuil @ 2022-02-01 19:44 UTC (permalink / raw) To: Bram Cohen; +Cc: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 3651 bytes --] > On Feb 1, 2022, at 00:32, Bram Cohen <bram@chia.net> wrote: > >> On Mon, Jan 31, 2022 at 4:08 PM Eric Voskuil <eric@voskuil.org> wrote: >> >> >>>> On Jan 31, 2022, at 15:15, Bram Cohen via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: >>> Is it still verboten to acknowledge that RBF is normal behavior and disallowing it is the feature, and that feature is mostly there to appease some people's delusions that zeroconf is a thing? It seems a bit overdue to disrespect the RBF flag in the direction of always assuming it's on. >> What flag? > > The opt-in RBF flag in transactions. Was being facetious. The “disrespect” referred to above assumes respect that some implementations have never given. >>> There are two different common regimes which result in different incentivized behavior. One of them is that there's more than a block's backlog in the mempool in which case between two conflicting transactions the one with the higher fee rate should win. In the other case where there isn't a whole block's worth of transactions the one with higher total value should win. >> These are not distinct scenarios. The rational choice is the highest fee block-valid subgraph of the set of unconfirmed transactions, in both cases (within the limits of what is computationally feasible of course). > > It's weird because which of two or more conflicting transactions should win can oscillate back and forth depending on other stuff going on in the mempool. The assumption of RAM storage is an error and unrelated to network protocol. There is nothing “going on” in a set of unconfirmed valid transactions. They are logically unchanging. > There's already a bit of that with child pays but this is stranger and has more oddball edge cases about which transactions to route. There’s really no such thing. The p2p network is necessarily permissionless. A person can route whatever he wants. Presumably people will not generally waste their own bandwidth by routing what they believe to be unconfirmable. And whatever they would retain themselves is their presumption of confirmable. This decision of what to retain one’s self is just a graph traversal to determine the most valuable subset - an optimizing CSP (though generally suboptimal due to the time constraint). Short of DoS, the most profitable model is to retain *all* valid transactions. [Note that a spend conflict is not an invalidity. Two valid transactions can be confirmed in sibling branch blocks - both valid in some context.] So the only consideration is low cost storage fill. The fee is a proof of spend, which like proof of work (for headers/blocks), is the basis of DoS protection (for unconfirmed transactions). The issue with two conflicting subgraphs is that one or the other is ultimately unspendable. As such the fee on each is non-cumulative and therefore only one (the highest) is providing DoS protection. Any subsequent conflicting subgraph must pay not only for itself, but for all preceding conflicting subgraphs. This pays for the storage, which is a trade accepted by the owner of the node in order to have a preview of confirmable transactions. This supports both mining generation of candidate blocks and rapid validation/confirmation of blocks. It’s a rather straightforward system when considered in terms of how it actually works (ie from a consensus standpoint). The only p2p issue is the need to package transactions for consideration as a set, as otherwise parents may be discarded before children can pay for them. Any set up to a full block is entirely reasonable for consideration. e [-- Attachment #2: Type: text/html, Size: 5317 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF policy 2022-01-31 22:54 ` [bitcoin-dev] Improving RBF policy Bram Cohen 2022-02-01 0:08 ` Eric Voskuil @ 2022-02-01 0:42 ` Antoine Riard 1 sibling, 0 replies; 25+ messages in thread From: Antoine Riard @ 2022-02-01 0:42 UTC (permalink / raw) To: Bram Cohen, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 3090 bytes --] > Is it still verboten to acknowledge that RBF is normal behavior and disallowing it is the feature, and that feature is mostly there to appease some people's delusions that zeroconf is a thing? It seems a bit overdue to disrespect the RBF flag in the direction of always assuming it's on. If you're thinking about the opt-in flag, not the RBF rules, please see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019074.html The latest state of the discussion is here : https://gnusha.org/bitcoin-core-dev/2021-10-21.log A gradual, multi-year deprecation sounds to be preferred to ease adaptation of the affected Bitcoin applications. Ultimately, I think it might not be the last time we have to change high-impact tx-relay/mempool rules and a more formalized Core policy deprecation process would be good. Le lun. 31 janv. 2022 à 18:15, Bram Cohen via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> a écrit : > Gloria Zhao wrote: > >> >> This post discusses limitations of current Bitcoin Core RBF policy and >> attempts to start a conversation about how we can improve it, >> summarizing some ideas that have been discussed. Please reply if you >> have any new input on issues to be solved and ideas for improvement! >> > > Is it still verboten to acknowledge that RBF is normal behavior and > disallowing it is the feature, and that feature is mostly there to appease > some people's delusions that zeroconf is a thing? It seems a bit overdue to > disrespect the RBF flag in the direction of always assuming it's on. > > >> - **Incentive Compatibility**: Ensure that our RBF policy would not >> accept replacement transactions which would decrease fee profits >> of a miner. In general, if our mempool policy deviates from what is >> economically rational, it's likely that the transactions in our >> mempool will not match the ones in miners' mempools, making our >> fee estimation, compact block relay, and other mempool-dependent >> functions unreliable. Incentive-incompatible policy may also >> encourage transaction submission through routes other than the p2p >> network, harming censorship-resistance and privacy of Bitcoin payments. >> > > There are two different common regimes which result in different > incentivized behavior. One of them is that there's more than a block's > backlog in the mempool in which case between two conflicting transactions > the one with the higher fee rate should win. In the other case where there > isn't a whole block's worth of transactions the one with higher total value > should win. It would be nice to have consolidated logic which handles both, > it seems the issue has to do with the slope of the supply/demand curve > which in the first case is gentle enough to keep the one transaction from > hitting the rate but in the second one is basically infinite. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 4240 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* [bitcoin-dev] Improving RBF Policy @ 2022-01-27 13:42 Gloria Zhao 2022-01-28 1:35 ` Jeremy 2022-01-30 22:53 ` Antoine Riard 0 siblings, 2 replies; 25+ messages in thread From: Gloria Zhao @ 2022-01-27 13:42 UTC (permalink / raw) To: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 21067 bytes --] Hi everyone, This post discusses limitations of current Bitcoin Core RBF policy and attempts to start a conversation about how we can improve it, summarizing some ideas that have been discussed. Please reply if you have any new input on issues to be solved and ideas for improvement! Just in case I've screwed up the text wrapping again, another copy can be found here: https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff ## Background Please feel free to skip this section if you are already familiar with RBF. Nodes may receive *conflicting* unconfirmed transactions, aka "double spends" of the same inputs. Instead of always keeping the first transaction, since v0.12, Bitcoin Core mempool policy has included a set of Replace-by-Fee (RBF) criteria that allows the second transaction to replace the first one and any descendants it may have. Bitcoin Core RBF policy was previously documented as BIP 125. The current RBF policy is documented [here][1]. In summary: 1. The directly conflicting transactions all signal replaceability explicitly. 2. The replacement transaction only includes an unconfirmed input if that input was included in one of the directly conflicting transactions. 3. The replacement transaction pays an absolute fee of at least the sum paid by the original transactions. 4. The additional fees pays for the replacement transaction's bandwidth at or above the rate set by the node's *incremental relay feerate*. 5. The sum of all directly conflicting transactions' descendant counts (number of transactions inclusive of itself and its descendants) does not exceed 100. We can split these rules into 3 categories/goals: - **Allow Opting Out**: Some applications/businesses are unable to handle transactions that are replaceable (e.g. merchants that use zero-confirmation transactions). We (try to) help these businesses by honoring BIP125 signaling; we won't replace transactions that have not opted in. - **Incentive Compatibility**: Ensure that our RBF policy would not accept replacement transactions which would decrease fee profits of a miner. In general, if our mempool policy deviates from what is economically rational, it's likely that the transactions in our mempool will not match the ones in miners' mempools, making our fee estimation, compact block relay, and other mempool-dependent functions unreliable. Incentive-incompatible policy may also encourage transaction submission through routes other than the p2p network, harming censorship-resistance and privacy of Bitcoin payments. - **DoS Protection**: Limit two types of DoS attacks on the node's mempool: (1) the number of times a transaction can be replaced and (2) the volume of transactions that can be evicted during a replacement. Even more abstract: our goal is to make a replacement policy that results in a useful interface for users and safe policy for node operators. ## Motivation There are a number of known problems with the current RBF policy. Many of these shortcomings exist due to mempool limitations at the time RBF was implemented or result from new types of Bitcoin usage; they are not criticisms of the original design. ### Pinning Attacks The most pressing concern is that attackers may take advantage of limitations in RBF policy to prevent other users' transactions from being mined or getting accepted as a replacement. #### SIGHASH_ANYONECANPAY Pinning BIP125#2 can be bypassed by creating intermediary transactions to be replaced together. Anyone can simply split a 1-input 1-output transaction off from the replacement transaction, then broadcast the transaction as is. This can always be done, and quite cheaply. More details in [this comment][2]. In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, anybody can just attach a low feerate parent to this transaction and lower its ancestor feerate. Even if you require SIGHASH\_ALL which prevents an attacker from changing any outputs, the input can be a very low amount (e.g. just above the dust limit) from a low-fee ancestor and still bring down the ancestor feerate of the transaction. TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and signals replaceability, regardless of the feerate you broadcast at, an attacker can lower its mining priority by adding an ancestor. #### Absolute Fee The restriction of requiring replacement transactions to increase the absolute fee of the mempool has been described as "bonkers." If the original transaction has a very large descendant that pays a large amount of fees, even if it has a low feerate, the replacement transaction must now pay those fees in order to meet Rule #3. #### Package RBF There are a number of reasons why, in order to enable Package RBF, we cannot use the same criteria. For starters, the absolute fee pinning attack is especially problematic if we apply the same rules (i.e. Rule #3 and #4) in Package RBF. Imagine that Alice (honest) and Bob (adversary) share a LN channel. The mempool is rather full, so their pre-negotiated commitment transactions' feerates would not be considered high priority by miners. Bob broadcasts his commitment transaction and attaches a very large child (100KvB with 100,000sat in fees) to his anchor output. Alice broadcasts her commitment transaction with a fee-bumping child (200vB with 50,000sat fees which is a generous 250sat/vB), but this does not meet the absolute fee requirement. She would need to add another 50,000sat to replace Bob's commitment transaction. Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be broken for packages containing transactions already in the mempool, explained [here][7]. Note: I originally [proposed][6] Package RBF using the same Rule #3 and #4 before I realized how significant this pinning attack is. I'm retracting that proposal, and a new set of Package RBF rules would follow from whatever the new individual RBF rules end up being. #### Same Txid Different Witness Two transactions with the same non-witness data but different witnesses have the same txid but different wtxid, and the same fee but not necessarily the same feerate. Currently, if we see a transaction that has the same txid as one in the mempool, we reject it as a duplicate, even if the feerate is much higher. It's unclear to me if we have a very strong reason to change this, but noting it as a limitation of our current replacement policy. See [#24007][12]. ### User Interface #### Using Unconfirmed UTXOs to Fund Replacements The restriction of only allowing confirmed UTXOs for funding a fee-bump (Rule #2) can hurt users trying to fee-bump their transactions and complicate wallet implementations. If the original transaction's output value isn't sufficient to fund a fee-bump and/or all of the user's other UTXOs are unconfirmed, they might not be able to fund a replacement transaction. Wallet developers also need to treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which adds complexity to wallet logic. For example, see BDK issues [#144][4] and [#414][5]. #### Interface Not Suitable for Coin Selection Currently, a user cannot simply create a replacement transaction targeting a specific feerate or meeting a minimum fee amount and expect to meet the RBF criteria. The fee amount depends on the size of the replacement transaction, and feerate is almost irrelevant. Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the replacement. It [estimates][13] a feerate which is "wallet incremental relay fee" (a conservative overestimation of the node's incremental relay fee) higher than the original transaction, selects coins for that feerate, and hopes that it meets the RBF rules. It never fails Rule #3 and #4 because it uses all original inputs and refuses to bump a transaction with mempool descendants. This is suboptimal, but is designed to work with the coin selection engine: select a feerate first, and then add fees to cover it. Following the exact RBF rules would require working the other way around: based on how much fees we've added to the transaction and its current size, calculate the feerate to see if we meet Rule #4. While this isn't completely broken, and the user interface is secondary to the safety of the mempool policy, we can do much better. A much more user-friendly interface would depend *only* on the fee and size of the original transactions. ### Updates to Mempool and Mining Since RBF was first implemented, a number of improvements have been made to mempool and mining logic. For example, we now use ancestor feerates in mining (allowing CPFP), and keep track of ancestor packages in the mempool. ## Ideas for Improvements ### Goals To summarize, these seem to be desired changes, in order of priority: 1. Remove Rule #3. The replacement should not be *required* to pay higher absolute fees. 2. Make it impossible for a replacement transaction to have a lower mining score than the original transaction(s). This would eliminate the `SIGHASH\_ANYONECANPAY` pinning attack. 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. 4. Create a more helpful interface that helps wallet fund replacement transactions that aim for a feerate and fee. ### A Different Model for Fees For incentive compatibility, I believe there are different formulations we should consider. Most importantly, if we want to get rid of the absolute fee rule, we can no longer think of it as "the transaction needs to pay for its own bandwidth," since we won't always be getting additional fees. That means we need a new method of rate-limiting replacements that doesn't require additional fees every time. While it makes sense to think about monetary costs when launching a specific type of attack, given that the fees are paid to the miner and not to the mempool operators, maybe it doesn't make much sense to think about "paying for bandwidth". Maybe we should implement transaction validation rate-limiting differently, e.g. building it into the P2P layer instead of the mempool policy layer. Recently, Suhas gave a [formulation][8] for incentive compatibility that made sense to me: "are the fees expected to be paid in the next (N?) blocks higher or lower if we process this transaction?" I started by thinking about this where N=1 or `1 + p`. Here, a rational miner is looking at what fees they would collect in the next block, and then some proportion `p` of the rest of the blocks based on their hashrate. We're assuming `p` isn't *so high* that they would be okay with lower absolute fees in the next 1 block. We're also assuming `p` isn't *so low* that the miner doesn't care about what's left of the mempool after this block. A tweak to this formulation is "if we process this transaction, would the fees in the next 1 block higher or lower, and is the feerate density of the rest of the mempool higher or lower?" This is pretty similar, where N=1, but we consider the rest of the mempool by feerate rather than fees. ### Mining Score of a Mempool Transaction We are often interested in finding out what the "mining score" of a transaction in the mempool is. That is, when the transaction is considered in block template building, what is the feerate it is considered at? Obviously, it's not the transaction's individual feerate. Bitcoin Core [mining code sorts][14] transactions by their ancestor feerate and includes them packages at a time, keeping track of how this affects the package feerates of remaining transactions in the mempool. *ancestor feerate*: Ancestor feerate is easily accessible information, but it's not accurate either, because it doesn't take into account the fact that subsets of a transaction's ancestor set can be included without it. For example, ancestors may have high feerates on their own or we may have [high feerate siblings][8]. TLDR: *Looking at the current ancestor feerate of a transaction is insufficient to tell us what feerate it will be considered at when building a block template in the future.* *min(individual feerate, ancestor feerate)*: Another heuristic that is simple to calculate based on current mempool tooling is to use the [minimum of a transaction's individual score and its ancestor score][10] as a conservative measure. But this can overestimate as well (see the example below). *min ancestor feerate(tx + possible ancestor subsets)* We can also take the minimum of every possible ancestor subset, but this can be computationally expensive since there can be lots and lots of ancestor subsets. *max ancestor feerate(tx + possible descendant subsets)*: Another idea is to use the [maximum ancestor score of the transaction + each of its descendants][9]. This doesn't work either; it has the same blindspot of ancestor subsets being mined on their own. #### Mining Score Example Here's an example illustrating why mining score is tricky to efficiently calculate for mempool transactions: Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), C(9sat/vB), D(5sat/vB). The layout is: grandparent A, parent B, and two children C and D. ``` A ^ B ^ ^ C D ``` A miner using ancestor packages to build block templates will first include A with a mining score of 21. Next, the miner will include B and C with a mining score of 6. This leaves D, with a mining score of 5. Note: in this case, mining by ancestor feerate results in the most rational decisions, but [a candidate set-based approach][10] which makes ancestor feerate much less relevant could be more advantageous in other situations. Here is a chart showing the "true" mining score alongside the values calculating using imperfect heuristics described above. All of them can overestimate or underestimate. ``` A B C D mining score | 21 | 6 | 6 | 5 | ancestor feerate | 21 | 11 | 10.3 | 9 | min(individual, ancestor) | 21 | 1 | 9 | 5 | min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | max(tx + descendants subsets) | 21 | 9 | 9 | 5 | ``` Possibly the best solution for finding the "mining score" of a transaction is to build a block template, see what feerate each package is included at. Perhaps at some cutoff, remaining mempool transactions can be estimated using some heuristic that leans {overestimating, underestimating} depending on the situation. Mining score seems to be relevant in multiple places: Murch and I recently [found][3] that it would be very important in "ancestor-aware" funding of transactions (the wallet doesn't incorporate ancestor fees when using unconfirmed transactions in coin selection, which is a bug we want to fix). In general, it would be nice to know the exact mining priority of one's unconfirmed transaction is. I can think of a few block/mempool explorers who might want to display this information for users. ### RBF Improvement Proposals After speaking to quite a few people, here are some suggestions for improvements that I have heard: * The ancestor score of the replacement must be {5, 10, N}% higher than that of every original transaction. * The ancestor score of the replacement must be 1sat/vB higher than that of every original transaction. * If the original transaction is in the top {0.75MvB, 1MvB} of the mempool, apply the current rules (absolute fees must increase and pay for the replacement transaction's new bandwidth). Otherwise, use a feerate-only rule. * If fees don't increase, the size of the replacement transaction must decrease by at least N%. * Rate-limit how many replacements we allow per prevout. * Rate-limit transaction validation in general, per peer. Perhaps some others on the mailing list can chime in to throw other ideas into the ring and/or combine some of these rules into a sensible policy. #### Replace by Feerate Only I don't think there's going to be a single-line feerate-based rule that can incorporate everything we need. On one hand, a feerate-only approach helps eliminate the issues associated with Rule #3. On the other hand, I believe the main concern with a feerate-only approach is how to rate limit replacements. We don't want to enable an attack such as: 1. Attacker broadcasts large, low-feerate transaction, and attaches a chain of descendants. 2. The attacker replaces the transaction with a smaller but higher feerate transaction, attaching a new chain of descendants. 3. Repeat 1000 times. #### Fees in Next Block and Feerate for the Rest of the Mempool Perhaps we can look at replacements like this: 1. Calculate the directly conflicting transactions and, with their descendants, the original transactions. Check signaling. Limit the total volume (e.g. can't be more than 100 total or 1MvB or something). 2. Find which original transactions would be in the next ~1 block. The replacement must pay at least this amount + X% in absolute fees. This guarantees that the fees of the next block doesn't decrease. 3. Find which transactions would be left in the mempool after that ~1 block. The replacement's feerate must be Y% higher than the maximum mining score of these transactions. This guarantees that you now have only *better* candidates in your after-this-block mempool than you did before, even if the size and fees the transactions decrease. 4. Now you have two numbers: a minimum absolute fee amount and a minimum feerate. Check to see if the replacement(s) meet these minimums. Also, a wallet would be able to ask the node "What fee and feerate would I need to put on a transaction replacing this?" and use this information to fund a replacement transaction, without needing to guess or overshoot. Obviously, there are some magic numbers missing here. X and Y are TBD constants to ensure we have some kind of rate limiting for the number of replacements allowed using some set of fees. What should they be? We can do some arithmetic to see what happens if you start with the biggest/lowest feerate transaction and do a bunch of replacements. Maybe we end up with values that are high enough to prevent abuse and make sense for applications/users that do RBF. ### Mempool Changes Need for Implementation As described in the mining score section above, we may want additional tooling to more accurately assess the economic gain of replacing transactions in our mempool. A few options have been discussed: * Calculate block templates on the fly when we need to consider a replacement. However, since replacements are [quite common][11] and the information might be useful for other things as well, it may be worth it to cache a block template. * Keep a persistent block template so that we know what transactions we would put in the next block. We need to remember the feerate at which each transaction was included in the template, because an ancestor package may be included in the same block template in multiple subsets. Transactions included earlier alter the ancestor feerate of the remaining transactions in the package. We also need to keep track of the new feerates of transactions left over. * Divide the mempool into two layers, "high feerate" and "low feerate." The high feerate layer contains ~1 block of packages with the highest ancestor feerates, and the low feerate layer contains everything else. At the edge of a block, we have a Knapsacky problem where the next highest ancestor feerate package might not fit, so we would probably want the high feerate layer ~2MvB or something to avoid underestimating the fees. ## Acknowledgements Thank you to everyone whose RBF-related suggestions, grievances, criticisms and ideas were incorporated in this document: Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, Antoine Poinsot, Antoine Riard, Larry Ruane, S3RK and Bastien Teinturier. Thanks for reading! Best, Gloria [1]: https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md [2]: https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 [3]: https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea [4]: https://github.com/bitcoindevkit/bdk/issues/144 [5]: https://github.com/bitcoindevkit/bdk/issues/414 [6]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html [7]: https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 [8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 [9]: https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 [10]: https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md [11]: https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 [12]: https://github.com/bitcoin/bitcoin/pull/24007 [13]: https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 [14]: https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 [-- Attachment #2: Type: text/html, Size: 24377 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-01-27 13:42 [bitcoin-dev] Improving RBF Policy Gloria Zhao @ 2022-01-28 1:35 ` Jeremy 2022-01-30 22:53 ` Antoine Riard 1 sibling, 0 replies; 25+ messages in thread From: Jeremy @ 2022-01-28 1:35 UTC (permalink / raw) To: Gloria Zhao, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 3717 bytes --] Gloria, This is a brilliant post! Great job systematizing many of the issues. Quite a lot to chew on & I hope other readers of this list digest the post fully. Three things come to mind as partial responses: under: - **DoS Protection**: Limit two types of DoS attacks on the node's > mempool: (1) the number of times a transaction can be replaced and > (2) the volume of transactions that can be evicted during a > replacement. I'd more simply put it: Limiting the amount of work that must be done to consider the replacement We don't particularly care about goal (1) or goal (2), we care about how much it costs to do (1) or (2). And there are scenarios where the (1) or (2) might not be particularly high, but the total work still might be. I can give you some examples to consider if needed. There are also scenarios where (1) and (2) might be high, but the cost is low overall. Therefore it makes sense to be a little more general with what the anti-DoS goal is. An issue I'd like to toss into the mix is that of iterative / additive batching. E.g., https://bitcoinops.org/en/cardcoins-rbf-batching/ This is where an business puts a txn in the mempool that pays to N users, and as they see additional requests for payouts the update it to N+2, N+2... N+M payouts. This iterative batching can be highly efficient because the number of transactions per business per 10 minutes is 1 (with variable number of outputs). One issue with this approach today is that because of the feerate rule, if you go from N to N+1 you need to pay 1 sat/byte over the whole txn. Applied M times, and you have to increase fees quadratically for this approach. Therefore the less efficient long-chain of batches model ends up being 'rational' with respect to mempool policy and irrational with respect to "optimally packing blocks with transactions". If the absolute fee rule is dropped, but feerate remains, one thing you might see is businesses doing iterative batches with N+2M outputs whereby they drop 2 outputs for every input they add, allowing the iterative batch to always increase the fee-rate but possibly not triggering the quadratic feerate issue since the transaction gets smaller over time. Another possible solution to this would be to allow relaying "txdiffs" which only require re-relay of signatures + new/modified outputs, and not the entire tx. I think this iterative batching is pretty desirable to support, and so I'd like to see a RBF model which doesn't make it "unfairly" expensive. (I'll spare everyone the details on how CTV batching also solves this, but feel free to ask elsewhere.) A counterargument to additive batching is that if you instead do non iterative batches every minute, and you have 100 txns that arrive uniformly, you'd end up with 10 batches of size 10 on average. The bulk of the benefit under this model is in the non-batched to batched transition, and the iterative part only saves on space/fees marginally after that point. A final point is that a verifiable delay function could be used over, e.g., each of the N COutpoints individually to rate-limit transaction replacement. The VDF period can be made shorter / eliminated depending on the feerate increase. E.g., always consider a much higher feerate txn whenever available, for things of equal feerate only consider 1 per minute. A VDF is like proof-of-work that doesn't parallelize, in case you are unfamiliar, so no matter how many computers you have it would take about the same amount of time (you could parallelize across N outputs, of course, but you're still bound minimally to the time it takes to replace 1 output, doing all outputs individually just is the most flexible option). Cheers, Jeremy [-- Attachment #2: Type: text/html, Size: 9089 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-01-27 13:42 [bitcoin-dev] Improving RBF Policy Gloria Zhao 2022-01-28 1:35 ` Jeremy @ 2022-01-30 22:53 ` Antoine Riard 2022-01-31 15:57 ` Bastien TEINTURIER 1 sibling, 1 reply; 25+ messages in thread From: Antoine Riard @ 2022-01-30 22:53 UTC (permalink / raw) To: Gloria Zhao, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 28589 bytes --] Hi Gloria, Thanks for this RBF sum up. Few thoughts and more context comments if it can help other readers. > For starters, the absolute fee pinning attack is especially > problematic if we apply the same rules (i.e. Rule #3 and #4) in > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a > LN channel. The mempool is rather full, so their pre-negotiated > commitment transactions' feerates would not be considered high > priority by miners. Bob broadcasts his commitment transaction and > attaches a very large child (100KvB with 100,000sat in fees) to his > anchor output. Alice broadcasts her commitment transaction with a > fee-bumping child (200vB with 50,000sat fees which is a generous > 250sat/vB), but this does not meet the absolute fee requirement. She > would need to add another 50,000sat to replace Bob's commitment > transaction. Solving LN pinning attacks, what we're aiming for is enabling a fair feerate bid between the counterparties, thus either forcing the adversary to overbid or to disengage from the confirmation competition. If the replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob to pick up the first option. Though if he does, that's a winning outcome for Alice, as one of the commitment transactions confirms and her time-sensitive second-stage HTLC can be subsequently confirmed. > It's unclear to me if > we have a very strong reason to change this, but noting it as a > limitation of our current replacement policy. See [#24007][12]. Deployment of Taproot opens interesting possibilities in the vaults/payment channels design space, where the tapscripts can commit to different set of timelocks/quorum of keys. Even if the pre-signed states stay symmetric, whoever is the publisher, the feerate cost to spend can fluctuate. > While this isn't completely broken, and the user interface is > secondary to the safety of the mempool policy I think with L2s transaction broadcast backend, the stability and clarity of the RBF user interface is primary. What we could be worried about is a too-much complex interface easing the way for an attacker to trigger your L2 node to issue policy-invalid chain of transactions. Especially, when we consider that an attacker might have leverage on chain of transactions composition ("force broadcast of commitment A then commitment B, knowing they will share a CPFP") or even transactions size ("overload commitment A with HTLCs"). > * If the original transaction is in the top {0.75MvB, 1MvB} of the > mempool, apply the current rules (absolute fees must increase and > pay for the replacement transaction's new bandwidth). Otherwise, use a > feerate-only rule. How this new replacement rule would behave if you have a parent in the "replace-by-feerate" half but the child is in the "replace-by-fee" one ? If we allow the replacement of the parent based on the feerate, we might decrease the top block absolute fees. If we block the replacement of the parent based on the feerate because the replacement absolute fees aren't above the replaced package, we still preclude a pinning vector. The child might be low-feerate junk and even attached to a low ancestor-score branch. If I'm correct on this limitation, maybe we could turn off the "replace-by-fee" behavior as soon as the mempool is fulfilled with a few blocks ? > * Rate-limit how many replacements we allow per prevout. Depending on how it is implemented, though I would be concerned it introduces a new pinning vector in the context of shared-utxo. If it's a hardcoded constant, it could be exhausted by an adversary starting at the lowest acceptable feerate then slowly increasing while still not reaching the top of the mempool. Same if it's time-based or block-based, no guarantee the replacement slot is honestly used by your counterparty. Further, an above-the-average replacement frequency might just be the reflection of your confirmation strategy reacting to block schedule or mempools historical data. As long as the feerate penalty is paid, I lean to allow replacement. (One solution could be to associate per-user "tag" to the LN transactions, where each "tag" would have its own replacement slots, but privacy?) > * Rate-limit transaction validation in general, per peer. I think we could improve on the Core's new transaction requester logic. Maybe we could bind the peer announced flow based on the feerate score (modulo validation time) of the previously validated transactions from that peer ? That said, while related to RBF, it sounds to me that enhancing Core's rate-limiting transaction strategy is a whole discussion in itself [0]. Especially ensuring it's tolerant to the specific requirements of LN & consorts. > What should they be? We can do some arithmetic to see what happens if > you start with the biggest/lowest feerate transaction and do a bunch > of replacements. Maybe we end up with values that are high enough to > prevent abuse and make sense for applications/users that do RBF. That's a good question. One observation is that the attacker can always renew the set of DoSy utxos to pursue the attack. So maybe we could pick up constants scaled on the block size ? That way an attacker would have to burn fees, thus deterring them from launching an attack. Even if the attackers are miners, they have to renounce their income to acquire new DoSy utxos. If a low-fee period, we could scale up the constants ? Overall, I think there is the deployment issue to warn of. Moving to a new set of RBF rules implies for a lot of Bitcoin applications to rewrite their RBF logics. We might have a more-or-less long transition period during which we support both... Cheers, Antoine [0] https://github.com/bitcoin/bitcoin/pull/21224 Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> a écrit : > Hi everyone, > > This post discusses limitations of current Bitcoin Core RBF policy and > attempts to start a conversation about how we can improve it, > summarizing some ideas that have been discussed. Please reply if you > have any new input on issues to be solved and ideas for improvement! > > Just in case I've screwed up the text wrapping again, another copy can be > found here: > https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff > > ## Background > > Please feel free to skip this section if you are already familiar > with RBF. > > Nodes may receive *conflicting* unconfirmed transactions, aka > "double spends" of the same inputs. Instead of always keeping the > first transaction, since v0.12, Bitcoin Core mempool policy has > included a set of Replace-by-Fee (RBF) criteria that allows the second > transaction to replace the first one and any descendants it may have. > > Bitcoin Core RBF policy was previously documented as BIP 125. > The current RBF policy is documented [here][1]. In summary: > > 1. The directly conflicting transactions all signal replaceability > explicitly. > > 2. The replacement transaction only includes an unconfirmed input if > that input was included in one of the directly conflicting > transactions. > > 3. The replacement transaction pays an absolute fee of at least the > sum paid by the original transactions. > > 4. The additional fees pays for the replacement transaction's > bandwidth at or above the rate set by the node's *incremental relay > feerate*. > > 5. The sum of all directly conflicting transactions' descendant counts > (number of transactions inclusive of itself and its descendants) > does not exceed 100. > > We can split these rules into 3 categories/goals: > > - **Allow Opting Out**: Some applications/businesses are unable to > handle transactions that are replaceable (e.g. merchants that use > zero-confirmation transactions). We (try to) help these businesses by > honoring BIP125 signaling; we won't replace transactions that have not > opted in. > > - **Incentive Compatibility**: Ensure that our RBF policy would not > accept replacement transactions which would decrease fee profits > of a miner. In general, if our mempool policy deviates from what is > economically rational, it's likely that the transactions in our > mempool will not match the ones in miners' mempools, making our > fee estimation, compact block relay, and other mempool-dependent > functions unreliable. Incentive-incompatible policy may also > encourage transaction submission through routes other than the p2p > network, harming censorship-resistance and privacy of Bitcoin payments. > > - **DoS Protection**: Limit two types of DoS attacks on the node's > mempool: (1) the number of times a transaction can be replaced and > (2) the volume of transactions that can be evicted during a > replacement. > > Even more abstract: our goal is to make a replacement policy that > results in a useful interface for users and safe policy for > node operators. > > ## Motivation > > There are a number of known problems with the current RBF policy. > Many of these shortcomings exist due to mempool limitations at the > time RBF was implemented or result from new types of Bitcoin usage; > they are not criticisms of the original design. > > ### Pinning Attacks > > The most pressing concern is that attackers may take advantage of > limitations in RBF policy to prevent other users' transactions from > being mined or getting accepted as a replacement. > > #### SIGHASH_ANYONECANPAY Pinning > > BIP125#2 can be bypassed by creating intermediary transactions to be > replaced together. Anyone can simply split a 1-input 1-output > transaction off from the replacement transaction, then broadcast the > transaction as is. This can always be done, and quite cheaply. More > details in [this comment][2]. > > In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, > anybody can just attach a low feerate parent to this transaction and > lower its ancestor feerate. Even if you require SIGHASH\_ALL which > prevents an attacker from changing any outputs, the input can be a > very low amount (e.g. just above the dust limit) from a low-fee > ancestor and still bring down the ancestor feerate of the transaction. > > TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and > signals replaceability, regardless of the feerate you broadcast at, an > attacker can lower its mining priority by adding an ancestor. > > #### Absolute Fee > > The restriction of requiring replacement transactions to increase the > absolute fee of the mempool has been described as "bonkers." If the > original transaction has a very large descendant that pays a large > amount of fees, even if it has a low feerate, the replacement > transaction must now pay those fees in order to meet Rule #3. > > #### Package RBF > > There are a number of reasons why, in order to enable Package RBF, we > cannot use the same criteria. > > For starters, the absolute fee pinning attack is especially > problematic if we apply the same rules (i.e. Rule #3 and #4) in > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a > LN channel. The mempool is rather full, so their pre-negotiated > commitment transactions' feerates would not be considered high > priority by miners. Bob broadcasts his commitment transaction and > attaches a very large child (100KvB with 100,000sat in fees) to his > anchor output. Alice broadcasts her commitment transaction with a > fee-bumping child (200vB with 50,000sat fees which is a generous > 250sat/vB), but this does not meet the absolute fee requirement. She > would need to add another 50,000sat to replace Bob's commitment > transaction. > > Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be > broken for packages containing transactions already in the mempool, > explained [here][7]. > > Note: I originally [proposed][6] Package RBF using the same Rule #3 > and #4 before I realized how significant this pinning attack is. I'm > retracting that proposal, and a new set of Package RBF rules would > follow from whatever the new individual RBF rules end up being. > > #### Same Txid Different Witness > > Two transactions with the same non-witness data but different > witnesses have the same txid but different wtxid, and the same fee but > not necessarily the same feerate. Currently, if we see a transaction > that has the same txid as one in the mempool, we reject it as a > duplicate, even if the feerate is much higher. It's unclear to me if > we have a very strong reason to change this, but noting it as a > limitation of our current replacement policy. See [#24007][12]. > > ### User Interface > > #### Using Unconfirmed UTXOs to Fund Replacements > > The restriction of only allowing confirmed UTXOs for funding a > fee-bump (Rule #2) can hurt users trying to fee-bump their > transactions and complicate wallet implementations. If the original > transaction's output value isn't sufficient to fund a fee-bump and/or > all of the user's other UTXOs are unconfirmed, they might not be able > to fund a replacement transaction. Wallet developers also need to > treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which > adds complexity to wallet logic. For example, see BDK issues [#144][4] > and [#414][5]. > > #### Interface Not Suitable for Coin Selection > > Currently, a user cannot simply create a replacement transaction > targeting a specific feerate or meeting a minimum fee amount and > expect to meet the RBF criteria. The fee amount depends on the size of > the replacement transaction, and feerate is almost irrelevant. > > Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the > replacement. It [estimates][13] a feerate which is "wallet incremental > relay fee" (a conservative overestimation of the node's incremental > relay fee) higher than the original transaction, selects coins for > that feerate, and hopes that it meets the RBF rules. It never fails > Rule #3 and #4 because it uses all original inputs and refuses to > bump a transaction with mempool descendants. > > This is suboptimal, but is designed to work with the coin selection > engine: select a feerate first, and then add fees to cover it. > Following the exact RBF rules would require working the other way > around: based on how much fees we've added to the transaction and its > current size, calculate the feerate to see if we meet Rule #4. > > While this isn't completely broken, and the user interface is > secondary to the safety of the mempool policy, we can do much better. > A much more user-friendly interface would depend *only* on the > fee and size of the original transactions. > > ### Updates to Mempool and Mining > > Since RBF was first implemented, a number of improvements have been > made to mempool and mining logic. For example, we now use ancestor > feerates in mining (allowing CPFP), and keep track of ancestor > packages in the mempool. > > ## Ideas for Improvements > > ### Goals > > To summarize, these seem to be desired changes, in order of priority: > > 1. Remove Rule #3. The replacement should not be *required* to pay > higher absolute fees. > > 2. Make it impossible for a replacement transaction to have a lower > mining score than the original transaction(s). This would eliminate > the `SIGHASH\_ANYONECANPAY` pinning attack. > > 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. > > 4. Create a more helpful interface that helps wallet fund replacement > transactions that aim for a feerate and fee. > > ### A Different Model for Fees > > For incentive compatibility, I believe there are different > formulations we should consider. Most importantly, if we want to get > rid of the absolute fee rule, we can no longer think of it as "the > transaction needs to pay for its own bandwidth," since we won't always > be getting additional fees. That means we need a new method of > rate-limiting replacements that doesn't require additional fees every > time. > > While it makes sense to think about monetary costs when launching a > specific type of attack, given that the fees are paid to the miner and > not to the mempool operators, maybe it doesn't make much sense to > think about "paying for bandwidth". Maybe we should implement > transaction validation rate-limiting differently, e.g. building it > into the P2P layer instead of the mempool policy layer. > > Recently, Suhas gave a [formulation][8] for incentive compatibility > that made sense to me: "are the fees expected to be paid in the next > (N?) blocks higher or lower if we process this transaction?" > > I started by thinking about this where N=1 or `1 + p`. > Here, a rational miner is looking at what fees they would > collect in the next block, and then some proportion `p` of the rest of > the blocks based on their hashrate. We're assuming `p` isn't *so high* > that they would be okay with lower absolute fees in the next 1 block. > We're also assuming `p` isn't *so low* that the miner doesn't care > about what's left of the mempool after this block. > > A tweak to this formulation is "if we process this transaction, would > the fees in the next 1 block higher or lower, and is the feerate > density of the rest of the mempool higher or lower?" This is pretty > similar, where N=1, but we consider the rest of the mempool by feerate > rather than fees. > > ### Mining Score of a Mempool Transaction > > We are often interested in finding out what > the "mining score" of a transaction in the mempool is. That is, when > the transaction is considered in block template building, what is the > feerate it is considered at? > > Obviously, it's not the transaction's individual feerate. Bitcoin Core > [mining code sorts][14] transactions by their ancestor feerate and > includes them packages at a time, keeping track of how this affects the > package feerates of remaining transactions in the mempool. > > *ancestor feerate*: Ancestor feerate is easily accessible information, > but it's not accurate either, because it doesn't take into account the > fact that subsets of a transaction's ancestor set can be included > without it. For example, ancestors may have high feerates on their own > or we may have [high feerate siblings][8]. > > TLDR: *Looking at the current ancestor feerate of a transaction is > insufficient to tell us what feerate it will be considered at when > building a block template in the future.* > > *min(individual feerate, ancestor feerate)*: Another > heuristic that is simple to calculate based on current mempool tooling > is to use the [minimum of a transaction's individual score and its > ancestor score][10] as a conservative measure. But this can > overestimate as well (see the example below). > > *min ancestor feerate(tx + possible ancestor subsets)* We can also > take the minimum of every possible ancestor subset, but this can be > computationally expensive since there can be lots and lots of ancestor > subsets. > > *max ancestor feerate(tx + possible descendant subsets)*: Another idea > is to use the [maximum ancestor score of the transaction + each of its > descendants][9]. This doesn't work either; it has the same blindspot > of ancestor subsets being mined on their own. > > #### Mining Score Example > > Here's an example illustrating why mining score is tricky to > efficiently calculate for mempool transactions: > > Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), > C(9sat/vB), D(5sat/vB). > The layout is: grandparent A, parent B, and two children C and D. > > ``` > A > ^ > B > ^ ^ > C D > ``` > > A miner using ancestor packages to build block templates will first > include A with a mining score of 21. Next, the miner will include B and > C with a mining score of 6. This leaves D, with a mining score of 5. > > Note: in this case, mining by ancestor feerate results in the most > rational decisions, but [a candidate set-based approach][10] which > makes ancestor feerate much less relevant could > be more advantageous in other situations. > > Here is a chart showing the "true" mining score alongside the values > calculating using imperfect heuristics described above. All of them > can overestimate or underestimate. > > ``` > A B C D > mining score | 21 | 6 | 6 | 5 | > ancestor feerate | 21 | 11 | 10.3 | 9 | > min(individual, ancestor) | 21 | 1 | 9 | 5 | > min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | > max(tx + descendants subsets) | 21 | 9 | 9 | 5 | > > ``` > > Possibly the best solution for finding the "mining score" of a > transaction is to build a block template, see what feerate each > package is included at. Perhaps at some cutoff, remaining mempool > transactions can be estimated using some heuristic that leans > {overestimating, underestimating} depending on the situation. > > Mining score seems to be relevant in multiple places: Murch and I > recently [found][3] that it would be very important in > "ancestor-aware" funding of transactions (the wallet doesn't > incorporate ancestor fees when using unconfirmed transactions in coin > selection, which is a bug we want to fix). > > In general, it would be nice to know the exact mining priority of > one's unconfirmed transaction is. I can think of a few block/mempool > explorers who might want to display this information for users. > > ### RBF Improvement Proposals > > After speaking to quite a few people, here are some suggestions > for improvements that I have heard: > > * The ancestor score of the replacement must be {5, 10, N}% higher > than that of every original transaction. > > * The ancestor score of the replacement must be 1sat/vB higher than > that of every original transaction. > > * If the original transaction is in the top {0.75MvB, 1MvB} of the > mempool, apply the current rules (absolute fees must increase and > pay for the replacement transaction's new bandwidth). Otherwise, use a > feerate-only rule. > > * If fees don't increase, the size of the replacement transaction must > decrease by at least N%. > > * Rate-limit how many replacements we allow per prevout. > > * Rate-limit transaction validation in general, per peer. > > Perhaps some others on the mailing list can chime in to throw other > ideas into the ring and/or combine some of these rules into a sensible > policy. > > #### Replace by Feerate Only > > I don't think there's going to be a single-line feerate-based > rule that can incorporate everything we need. > On one hand, a feerate-only approach helps eliminate the issues > associated with Rule #3. On the other hand, I believe the main concern > with a feerate-only approach is how to rate limit replacements. We > don't want to enable an attack such as: > > 1. Attacker broadcasts large, low-feerate transaction, and attaches a > chain of descendants. > > 2. The attacker replaces the transaction with a smaller but higher > feerate transaction, attaching a new chain of descendants. > > 3. Repeat 1000 times. > > #### Fees in Next Block and Feerate for the Rest of the Mempool > > Perhaps we can look at replacements like this: > > 1. Calculate the directly conflicting transactions and, with their > descendants, the original transactions. Check signaling. Limit the > total volume (e.g. can't be more than 100 total or 1MvB or something). > > 2. Find which original transactions would be in the next ~1 block. The > replacement must pay at least this amount + X% in absolute fees. This > guarantees that the fees of the next block doesn't decrease. > > 3. Find which transactions would be left in the mempool after that ~1 > block. The replacement's feerate must be Y% higher than the maximum > mining score of these transactions. This guarantees that you now have > only *better* candidates in your after-this-block mempool than you did > before, even if the size and fees the transactions decrease. > > 4. Now you have two numbers: a minimum absolute fee amount and a > minimum feerate. Check to see if the replacement(s) meet these > minimums. Also, a wallet would be able to ask the node "What fee and > feerate would I need to put on a transaction replacing this?" and use > this information to fund a replacement transaction, without needing to > guess or overshoot. > > Obviously, there are some magic numbers missing here. X and Y are > TBD constants to ensure we have some kind of rate limiting for the > number of replacements allowed using some set of fees. > > What should they be? We can do some arithmetic to see what happens if > you start with the biggest/lowest feerate transaction and do a bunch > of replacements. Maybe we end up with values that are high enough to > prevent abuse and make sense for applications/users that do RBF. > > ### Mempool Changes Need for Implementation > > As described in the mining score section above, > we may want additional tooling to more accurately assess > the economic gain of replacing transactions in our mempool. > > A few options have been discussed: > > * Calculate block templates on the fly when we need to consider a > replacement. However, since replacements are [quite common][11] > and the information might be useful for other things as well, > it may be worth it to cache a block template. > > * Keep a persistent block template so that we know what transactions > we would put in the next block. We need to remember the feerate > at which each transaction was included in the template, because an > ancestor package may be included in the same block template in > multiple subsets. Transactions included earlier alter the ancestor > feerate of the remaining transactions in the package. We also need > to keep track of the new feerates of transactions left over. > > * Divide the mempool into two layers, "high feerate" and "low > feerate." The high feerate layer contains ~1 block of packages with > the highest ancestor feerates, and the low feerate layer contains > everything else. At the edge of a block, we have a Knapsacky problem > where the next highest ancestor feerate package might not fit, so we > would probably want the high feerate layer ~2MvB or something to avoid > underestimating the fees. > > ## Acknowledgements > > Thank you to everyone whose RBF-related suggestions, grievances, > criticisms and ideas were incorporated in this document: > Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, > Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, > Antoine Poinsot, Antoine Riard, Larry Ruane, > S3RK and Bastien Teinturier. > > Thanks for reading! > > Best, > Gloria > > [1]: > https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md > [2]: https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 > [3]: > https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea > [4]: https://github.com/bitcoindevkit/bdk/issues/144 > [5]: https://github.com/bitcoindevkit/bdk/issues/414 > [6]: > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html > [7]: > https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 > [8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 > [9]: https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 > [10]: > https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md > [11]: https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 > [12]: https://github.com/bitcoin/bitcoin/pull/24007 > [13]: > https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 > [14]: > https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 31690 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-01-30 22:53 ` Antoine Riard @ 2022-01-31 15:57 ` Bastien TEINTURIER 2022-02-01 1:56 ` Anthony Towns 2022-02-05 13:21 ` Michael Folkson 0 siblings, 2 replies; 25+ messages in thread From: Bastien TEINTURIER @ 2022-01-31 15:57 UTC (permalink / raw) To: Antoine Riard, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 36506 bytes --] Hi Gloria, Many thanks for raising awareness on these issues and constantly pushing towards finding a better model. This work will highly improve the security of any multi-party contract trying to build on top of bitcoin (because most multi-party contracts will need to have timeout conditions and participants will need to make some transactions confirm before a timeout happens - otherwise they may lose funds). For starters, let me quickly explain why the current rules are hard to work with in the context of lightning (but I believe most L2 protocols will have the same issues). Feel free to skip this part if you are already convinced. ## Motivation The biggest pain point is BIP 125 rule 2. If I need to increase the fees of a time-sensitive transaction because the feerate has been rising since I broadcast it, I may need to also pay high fees just to produce a confirmed utxo that I can use. I'm actually paying a high fee twice instead of once (and needlessly using on-chain space, our scarcest asset, because we could have avoided that additional transaction!). It also has some annoying "non-determinism". Imagine that my transaction has been evicted from my mempool because its feerate was too low. I could think "Great, that means I don't have to apply BIP 125 restrictions, I can just fund this transaction as if it were a new one!". But actually I do, because my transaction could still be in miner's mempools and I have no way of knowing it...this means that whenever I have broadcast a transaction, I must assume that I will always need to abide by whatever replacement rules the network applies. Fortunately, as far as I understand it, this rule only exists because of a previous implementation detail of bitcoin core, so there's simply no good reason to keep it. The second biggest pain point is rule 3. It prevents me from efficiently using my capital while it's unconfirmed. Whenever I'm using a big utxo to fund a transaction, I will get a big change output, and it would really be a waste to be unable to use that change output to fund other transactions. In order to be capital-efficient, I will end up creating descendant trees for my time-sensitive transactions. But as Gloria explained, replacing all my children will cost me an absurdly large amount of fees. So what I'm actually planning to do instead is to RBF one of the descendants high enough to get the whole tree confirmed. But if those descendants' timeouts were far in the future, that's a waste, I paid a lot more fees for them than I should have. I'd like to just replace my transaction and republish the invalidated children independently. Rule 4 doesn't hurt as much as the two previous ones, I don't have too much to say about it. To be fair to the BIP 125 authors, all of these scenarios were very hard to forecast at the time this BIP was created. We needed years to build on those rules to get a better understanding of their limitations and if the rationale behind them made sense in the long term. ## Proposals I believe that now is a good time to re-think those, and I really like Gloria's categorization of the design constraints. I'd like to propose a different way of looking at descendants that makes it easier to design the new rules. The way I understand it, limiting the impact on descendant transactions is only important for DoS protection, not for incentive compatibility. I would argue that after evictions, descendant transactions will be submitted again (because they represent transactions that people actually want to make), so evicting them does not have a negative impact on mining incentives (in a world where blocks are full most of the time). I'm curious to hear other people's thoughts on that. If it makes sense, I would propose the following very simple rules: 1. The transaction's ancestor absolute fees must be X% higher than the previous transaction's ancestor fees 2. The transaction's ancestor feerate must be Y% higher than the previous transaction's ancestor feerate I believe it's completely ok to require increasing both the fees and feerate if we don't take descendants into account, because you control your ancestor set - whereas the descendant set may be completely out of your control. This is very easy to use by wallets, because the ancestor set is easy to obtain. And an important point is that the ancestor set is the same in every mempool, whereas the descendant set is not (your mempool may have rejected the last descendants, while other people's mempools may still contain them). Because of that reason, I'd like to avoid having a rule that relies on some size of the replaced descendant set: it may be valid in your mempool but invalid in someone else's, which makes it exploitable for pinning attacks. I believe these rules are incentive compatible (again, if you accept the fact that the descendants will be re-submitted and mined as well, so their fees aren't lost). Can we choose X and Y so that these two rules are also DoS-resistant? Unfortunately I'm not sure, so maybe we'll need to add a third rule to address that. But before we do, can someone detail what it costs for a node to evict a descendant tree? Given that bitcoin core doesn't allow chains of more than 25 transactions, the maximum number of transactions being replaced will be bounded by 25 * N (where N is the number of outputs of the transaction being replaced). If it's just O(n) pruning of a graph, maybe that's ok? Or maybe we make X or Y depend on the number of outputs of the transaction being replaced (this would need very careful thoughts)? If you made it this far, thanks for reading! A couple of comments on the previous messages: > Currently, if we see a transaction > that has the same txid as one in the mempool, we reject it as a > duplicate, even if the feerate is much higher. It's unclear to me if > we have a very strong reason to change this, but noting it as a > limitation of our current replacement policy. I don't see a strong reason from an L2 protocol's point of view yet, but there are many unkown unknowns. But from a miner incentive's point of view, we should keep the transaction with the higher feerate, shouldn't we? In that case it's also a more efficient use of on-chain space, which is a win, right? > We might have a more-or-less long transition period during which we support both... Yes, this is a long term thing. Even if bitcoin core releases a new version with updated RBF rules, as a wallet you'll need to keep using the old rules for a long time if you want to be safe. But it's all the more reason to try to ship this as soon as possible, this way maybe our grand-children will be able to benefit from it ;) (just kidding on the timespan obviously). Cheers, Bastien Le lun. 31 janv. 2022 à 00:11, Antoine Riard via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> a écrit : > Hi Gloria, > > Thanks for this RBF sum up. Few thoughts and more context comments if it > can help other readers. > > > For starters, the absolute fee pinning attack is especially > > problematic if we apply the same rules (i.e. Rule #3 and #4) in > > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a > > LN channel. The mempool is rather full, so their pre-negotiated > > commitment transactions' feerates would not be considered high > > priority by miners. Bob broadcasts his commitment transaction and > > attaches a very large child (100KvB with 100,000sat in fees) to his > > anchor output. Alice broadcasts her commitment transaction with a > > fee-bumping child (200vB with 50,000sat fees which is a generous > > 250sat/vB), but this does not meet the absolute fee requirement. She > > would need to add another 50,000sat to replace Bob's commitment > > transaction. > > Solving LN pinning attacks, what we're aiming for is enabling a fair > feerate bid between the counterparties, thus either forcing the adversary > to overbid or to disengage from the confirmation competition. If the > replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob > to > pick up the first option. Though if he does, that's a winning outcome for > Alice, as one of the commitment transactions confirms and her > time-sensitive second-stage HTLC can be subsequently confirmed. > > > It's unclear to me if > > we have a very strong reason to change this, but noting it as a > > limitation of our current replacement policy. See [#24007][12]. > > Deployment of Taproot opens interesting possibilities in the > vaults/payment channels design space, where the tapscripts can commit to > different set of timelocks/quorum of keys. Even if the pre-signed states > stay symmetric, whoever is the publisher, the feerate cost to spend can > fluctuate. > > > While this isn't completely broken, and the user interface is > > secondary to the safety of the mempool policy > > I think with L2s transaction broadcast backend, the stability and clarity > of the RBF user interface is primary. What we could be worried about is a > too-much complex interface easing the way for an attacker to trigger your > L2 node to issue policy-invalid chain of transactions. Especially, when we > consider that an attacker might have leverage on chain of transactions > composition ("force broadcast of commitment A then commitment B, knowing > they will share a CPFP") or even transactions size ("overload commitment A > with HTLCs"). > > > * If the original transaction is in the top {0.75MvB, 1MvB} of the > > mempool, apply the current rules (absolute fees must increase and > > pay for the replacement transaction's new bandwidth). Otherwise, use a > > feerate-only rule. > > How this new replacement rule would behave if you have a parent in the > "replace-by-feerate" half but the child is in the "replace-by-fee" one ? > > If we allow the replacement of the parent based on the feerate, we might > decrease the top block absolute fees. > > If we block the replacement of the parent based on the feerate because the > replacement absolute fees aren't above the replaced package, we still > preclude a pinning vector. The child might be low-feerate junk and even > attached to a low ancestor-score branch. > > If I'm correct on this limitation, maybe we could turn off the > "replace-by-fee" behavior as soon as the mempool is fulfilled with a few > blocks ? > > > * Rate-limit how many replacements we allow per prevout. > > Depending on how it is implemented, though I would be concerned it > introduces a new pinning vector in the context of shared-utxo. If it's a > hardcoded constant, it could be exhausted by an adversary starting at the > lowest acceptable feerate then slowly increasing while still not reaching > the top of the mempool. Same if it's time-based or block-based, no > guarantee the replacement slot is honestly used by your counterparty. > > Further, an above-the-average replacement frequency might just be the > reflection of your confirmation strategy reacting to block schedule or > mempools historical data. As long as the feerate penalty is paid, I lean to > allow replacement. > > (One solution could be to associate per-user "tag" to the LN transactions, > where each "tag" would have its own replacement slots, but privacy?) > > > * Rate-limit transaction validation in general, per peer. > > I think we could improve on the Core's new transaction requester logic. > Maybe we could bind the peer announced flow based on the feerate score > (modulo validation time) of the previously validated transactions from that > peer ? That said, while related to RBF, it sounds to me that enhancing > Core's rate-limiting transaction strategy is a whole discussion in itself > [0]. Especially ensuring it's tolerant to the specific requirements of LN & > consorts. > > > What should they be? We can do some arithmetic to see what happens if > > you start with the biggest/lowest feerate transaction and do a bunch > > of replacements. Maybe we end up with values that are high enough to > > prevent abuse and make sense for applications/users that do RBF. > > That's a good question. > > One observation is that the attacker can always renew the set of DoSy > utxos to pursue the attack. So maybe we could pick up constants scaled on > the block size ? That way an attacker would have to burn fees, thus > deterring them from launching an attack. Even if the attackers are miners, > they have to renounce their income to acquire new DoSy utxos. If a low-fee > period, we could scale up the constants ? > > > Overall, I think there is the deployment issue to warn of. Moving to a new > set of RBF rules implies for a lot of Bitcoin applications to rewrite their > RBF logics. We might have a more-or-less long transition period during > which we support both... > > Cheers, > Antoine > > [0] https://github.com/bitcoin/bitcoin/pull/21224 > > Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> a écrit : > >> Hi everyone, >> >> This post discusses limitations of current Bitcoin Core RBF policy and >> attempts to start a conversation about how we can improve it, >> summarizing some ideas that have been discussed. Please reply if you >> have any new input on issues to be solved and ideas for improvement! >> >> Just in case I've screwed up the text wrapping again, another copy can be >> found here: >> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff >> >> ## Background >> >> Please feel free to skip this section if you are already familiar >> with RBF. >> >> Nodes may receive *conflicting* unconfirmed transactions, aka >> "double spends" of the same inputs. Instead of always keeping the >> first transaction, since v0.12, Bitcoin Core mempool policy has >> included a set of Replace-by-Fee (RBF) criteria that allows the second >> transaction to replace the first one and any descendants it may have. >> >> Bitcoin Core RBF policy was previously documented as BIP 125. >> The current RBF policy is documented [here][1]. In summary: >> >> 1. The directly conflicting transactions all signal replaceability >> explicitly. >> >> 2. The replacement transaction only includes an unconfirmed input if >> that input was included in one of the directly conflicting >> transactions. >> >> 3. The replacement transaction pays an absolute fee of at least the >> sum paid by the original transactions. >> >> 4. The additional fees pays for the replacement transaction's >> bandwidth at or above the rate set by the node's *incremental relay >> feerate*. >> >> 5. The sum of all directly conflicting transactions' descendant counts >> (number of transactions inclusive of itself and its descendants) >> does not exceed 100. >> >> We can split these rules into 3 categories/goals: >> >> - **Allow Opting Out**: Some applications/businesses are unable to >> handle transactions that are replaceable (e.g. merchants that use >> zero-confirmation transactions). We (try to) help these businesses by >> honoring BIP125 signaling; we won't replace transactions that have not >> opted in. >> >> - **Incentive Compatibility**: Ensure that our RBF policy would not >> accept replacement transactions which would decrease fee profits >> of a miner. In general, if our mempool policy deviates from what is >> economically rational, it's likely that the transactions in our >> mempool will not match the ones in miners' mempools, making our >> fee estimation, compact block relay, and other mempool-dependent >> functions unreliable. Incentive-incompatible policy may also >> encourage transaction submission through routes other than the p2p >> network, harming censorship-resistance and privacy of Bitcoin payments. >> >> - **DoS Protection**: Limit two types of DoS attacks on the node's >> mempool: (1) the number of times a transaction can be replaced and >> (2) the volume of transactions that can be evicted during a >> replacement. >> >> Even more abstract: our goal is to make a replacement policy that >> results in a useful interface for users and safe policy for >> node operators. >> >> ## Motivation >> >> There are a number of known problems with the current RBF policy. >> Many of these shortcomings exist due to mempool limitations at the >> time RBF was implemented or result from new types of Bitcoin usage; >> they are not criticisms of the original design. >> >> ### Pinning Attacks >> >> The most pressing concern is that attackers may take advantage of >> limitations in RBF policy to prevent other users' transactions from >> being mined or getting accepted as a replacement. >> >> #### SIGHASH_ANYONECANPAY Pinning >> >> BIP125#2 can be bypassed by creating intermediary transactions to be >> replaced together. Anyone can simply split a 1-input 1-output >> transaction off from the replacement transaction, then broadcast the >> transaction as is. This can always be done, and quite cheaply. More >> details in [this comment][2]. >> >> In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, >> anybody can just attach a low feerate parent to this transaction and >> lower its ancestor feerate. Even if you require SIGHASH\_ALL which >> prevents an attacker from changing any outputs, the input can be a >> very low amount (e.g. just above the dust limit) from a low-fee >> ancestor and still bring down the ancestor feerate of the transaction. >> >> TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and >> signals replaceability, regardless of the feerate you broadcast at, an >> attacker can lower its mining priority by adding an ancestor. >> >> #### Absolute Fee >> >> The restriction of requiring replacement transactions to increase the >> absolute fee of the mempool has been described as "bonkers." If the >> original transaction has a very large descendant that pays a large >> amount of fees, even if it has a low feerate, the replacement >> transaction must now pay those fees in order to meet Rule #3. >> >> #### Package RBF >> >> There are a number of reasons why, in order to enable Package RBF, we >> cannot use the same criteria. >> >> For starters, the absolute fee pinning attack is especially >> problematic if we apply the same rules (i.e. Rule #3 and #4) in >> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >> LN channel. The mempool is rather full, so their pre-negotiated >> commitment transactions' feerates would not be considered high >> priority by miners. Bob broadcasts his commitment transaction and >> attaches a very large child (100KvB with 100,000sat in fees) to his >> anchor output. Alice broadcasts her commitment transaction with a >> fee-bumping child (200vB with 50,000sat fees which is a generous >> 250sat/vB), but this does not meet the absolute fee requirement. She >> would need to add another 50,000sat to replace Bob's commitment >> transaction. >> >> Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be >> broken for packages containing transactions already in the mempool, >> explained [here][7]. >> >> Note: I originally [proposed][6] Package RBF using the same Rule #3 >> and #4 before I realized how significant this pinning attack is. I'm >> retracting that proposal, and a new set of Package RBF rules would >> follow from whatever the new individual RBF rules end up being. >> >> #### Same Txid Different Witness >> >> Two transactions with the same non-witness data but different >> witnesses have the same txid but different wtxid, and the same fee but >> not necessarily the same feerate. Currently, if we see a transaction >> that has the same txid as one in the mempool, we reject it as a >> duplicate, even if the feerate is much higher. It's unclear to me if >> we have a very strong reason to change this, but noting it as a >> limitation of our current replacement policy. See [#24007][12]. >> >> ### User Interface >> >> #### Using Unconfirmed UTXOs to Fund Replacements >> >> The restriction of only allowing confirmed UTXOs for funding a >> fee-bump (Rule #2) can hurt users trying to fee-bump their >> transactions and complicate wallet implementations. If the original >> transaction's output value isn't sufficient to fund a fee-bump and/or >> all of the user's other UTXOs are unconfirmed, they might not be able >> to fund a replacement transaction. Wallet developers also need to >> treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which >> adds complexity to wallet logic. For example, see BDK issues [#144][4] >> and [#414][5]. >> >> #### Interface Not Suitable for Coin Selection >> >> Currently, a user cannot simply create a replacement transaction >> targeting a specific feerate or meeting a minimum fee amount and >> expect to meet the RBF criteria. The fee amount depends on the size of >> the replacement transaction, and feerate is almost irrelevant. >> >> Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the >> replacement. It [estimates][13] a feerate which is "wallet incremental >> relay fee" (a conservative overestimation of the node's incremental >> relay fee) higher than the original transaction, selects coins for >> that feerate, and hopes that it meets the RBF rules. It never fails >> Rule #3 and #4 because it uses all original inputs and refuses to >> bump a transaction with mempool descendants. >> >> This is suboptimal, but is designed to work with the coin selection >> engine: select a feerate first, and then add fees to cover it. >> Following the exact RBF rules would require working the other way >> around: based on how much fees we've added to the transaction and its >> current size, calculate the feerate to see if we meet Rule #4. >> >> While this isn't completely broken, and the user interface is >> secondary to the safety of the mempool policy, we can do much better. >> A much more user-friendly interface would depend *only* on the >> fee and size of the original transactions. >> >> ### Updates to Mempool and Mining >> >> Since RBF was first implemented, a number of improvements have been >> made to mempool and mining logic. For example, we now use ancestor >> feerates in mining (allowing CPFP), and keep track of ancestor >> packages in the mempool. >> >> ## Ideas for Improvements >> >> ### Goals >> >> To summarize, these seem to be desired changes, in order of priority: >> >> 1. Remove Rule #3. The replacement should not be *required* to pay >> higher absolute fees. >> >> 2. Make it impossible for a replacement transaction to have a lower >> mining score than the original transaction(s). This would eliminate >> the `SIGHASH\_ANYONECANPAY` pinning attack. >> >> 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. >> >> 4. Create a more helpful interface that helps wallet fund replacement >> transactions that aim for a feerate and fee. >> >> ### A Different Model for Fees >> >> For incentive compatibility, I believe there are different >> formulations we should consider. Most importantly, if we want to get >> rid of the absolute fee rule, we can no longer think of it as "the >> transaction needs to pay for its own bandwidth," since we won't always >> be getting additional fees. That means we need a new method of >> rate-limiting replacements that doesn't require additional fees every >> time. >> >> While it makes sense to think about monetary costs when launching a >> specific type of attack, given that the fees are paid to the miner and >> not to the mempool operators, maybe it doesn't make much sense to >> think about "paying for bandwidth". Maybe we should implement >> transaction validation rate-limiting differently, e.g. building it >> into the P2P layer instead of the mempool policy layer. >> >> Recently, Suhas gave a [formulation][8] for incentive compatibility >> that made sense to me: "are the fees expected to be paid in the next >> (N?) blocks higher or lower if we process this transaction?" >> >> I started by thinking about this where N=1 or `1 + p`. >> Here, a rational miner is looking at what fees they would >> collect in the next block, and then some proportion `p` of the rest of >> the blocks based on their hashrate. We're assuming `p` isn't *so high* >> that they would be okay with lower absolute fees in the next 1 block. >> We're also assuming `p` isn't *so low* that the miner doesn't care >> about what's left of the mempool after this block. >> >> A tweak to this formulation is "if we process this transaction, would >> the fees in the next 1 block higher or lower, and is the feerate >> density of the rest of the mempool higher or lower?" This is pretty >> similar, where N=1, but we consider the rest of the mempool by feerate >> rather than fees. >> >> ### Mining Score of a Mempool Transaction >> >> We are often interested in finding out what >> the "mining score" of a transaction in the mempool is. That is, when >> the transaction is considered in block template building, what is the >> feerate it is considered at? >> >> Obviously, it's not the transaction's individual feerate. Bitcoin Core >> [mining code sorts][14] transactions by their ancestor feerate and >> includes them packages at a time, keeping track of how this affects the >> package feerates of remaining transactions in the mempool. >> >> *ancestor feerate*: Ancestor feerate is easily accessible information, >> but it's not accurate either, because it doesn't take into account the >> fact that subsets of a transaction's ancestor set can be included >> without it. For example, ancestors may have high feerates on their own >> or we may have [high feerate siblings][8]. >> >> TLDR: *Looking at the current ancestor feerate of a transaction is >> insufficient to tell us what feerate it will be considered at when >> building a block template in the future.* >> >> *min(individual feerate, ancestor feerate)*: Another >> heuristic that is simple to calculate based on current mempool tooling >> is to use the [minimum of a transaction's individual score and its >> ancestor score][10] as a conservative measure. But this can >> overestimate as well (see the example below). >> >> *min ancestor feerate(tx + possible ancestor subsets)* We can also >> take the minimum of every possible ancestor subset, but this can be >> computationally expensive since there can be lots and lots of ancestor >> subsets. >> >> *max ancestor feerate(tx + possible descendant subsets)*: Another idea >> is to use the [maximum ancestor score of the transaction + each of its >> descendants][9]. This doesn't work either; it has the same blindspot >> of ancestor subsets being mined on their own. >> >> #### Mining Score Example >> >> Here's an example illustrating why mining score is tricky to >> efficiently calculate for mempool transactions: >> >> Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), >> C(9sat/vB), D(5sat/vB). >> The layout is: grandparent A, parent B, and two children C and D. >> >> ``` >> A >> ^ >> B >> ^ ^ >> C D >> ``` >> >> A miner using ancestor packages to build block templates will first >> include A with a mining score of 21. Next, the miner will include B and >> C with a mining score of 6. This leaves D, with a mining score of 5. >> >> Note: in this case, mining by ancestor feerate results in the most >> rational decisions, but [a candidate set-based approach][10] which >> makes ancestor feerate much less relevant could >> be more advantageous in other situations. >> >> Here is a chart showing the "true" mining score alongside the values >> calculating using imperfect heuristics described above. All of them >> can overestimate or underestimate. >> >> ``` >> A B C D >> mining score | 21 | 6 | 6 | 5 | >> ancestor feerate | 21 | 11 | 10.3 | 9 | >> min(individual, ancestor) | 21 | 1 | 9 | 5 | >> min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | >> max(tx + descendants subsets) | 21 | 9 | 9 | 5 | >> >> ``` >> >> Possibly the best solution for finding the "mining score" of a >> transaction is to build a block template, see what feerate each >> package is included at. Perhaps at some cutoff, remaining mempool >> transactions can be estimated using some heuristic that leans >> {overestimating, underestimating} depending on the situation. >> >> Mining score seems to be relevant in multiple places: Murch and I >> recently [found][3] that it would be very important in >> "ancestor-aware" funding of transactions (the wallet doesn't >> incorporate ancestor fees when using unconfirmed transactions in coin >> selection, which is a bug we want to fix). >> >> In general, it would be nice to know the exact mining priority of >> one's unconfirmed transaction is. I can think of a few block/mempool >> explorers who might want to display this information for users. >> >> ### RBF Improvement Proposals >> >> After speaking to quite a few people, here are some suggestions >> for improvements that I have heard: >> >> * The ancestor score of the replacement must be {5, 10, N}% higher >> than that of every original transaction. >> >> * The ancestor score of the replacement must be 1sat/vB higher than >> that of every original transaction. >> >> * If the original transaction is in the top {0.75MvB, 1MvB} of the >> mempool, apply the current rules (absolute fees must increase and >> pay for the replacement transaction's new bandwidth). Otherwise, use a >> feerate-only rule. >> >> * If fees don't increase, the size of the replacement transaction must >> decrease by at least N%. >> >> * Rate-limit how many replacements we allow per prevout. >> >> * Rate-limit transaction validation in general, per peer. >> >> Perhaps some others on the mailing list can chime in to throw other >> ideas into the ring and/or combine some of these rules into a sensible >> policy. >> >> #### Replace by Feerate Only >> >> I don't think there's going to be a single-line feerate-based >> rule that can incorporate everything we need. >> On one hand, a feerate-only approach helps eliminate the issues >> associated with Rule #3. On the other hand, I believe the main concern >> with a feerate-only approach is how to rate limit replacements. We >> don't want to enable an attack such as: >> >> 1. Attacker broadcasts large, low-feerate transaction, and attaches a >> chain of descendants. >> >> 2. The attacker replaces the transaction with a smaller but higher >> feerate transaction, attaching a new chain of descendants. >> >> 3. Repeat 1000 times. >> >> #### Fees in Next Block and Feerate for the Rest of the Mempool >> >> Perhaps we can look at replacements like this: >> >> 1. Calculate the directly conflicting transactions and, with their >> descendants, the original transactions. Check signaling. Limit the >> total volume (e.g. can't be more than 100 total or 1MvB or something). >> >> 2. Find which original transactions would be in the next ~1 block. The >> replacement must pay at least this amount + X% in absolute fees. This >> guarantees that the fees of the next block doesn't decrease. >> >> 3. Find which transactions would be left in the mempool after that ~1 >> block. The replacement's feerate must be Y% higher than the maximum >> mining score of these transactions. This guarantees that you now have >> only *better* candidates in your after-this-block mempool than you did >> before, even if the size and fees the transactions decrease. >> >> 4. Now you have two numbers: a minimum absolute fee amount and a >> minimum feerate. Check to see if the replacement(s) meet these >> minimums. Also, a wallet would be able to ask the node "What fee and >> feerate would I need to put on a transaction replacing this?" and use >> this information to fund a replacement transaction, without needing to >> guess or overshoot. >> >> Obviously, there are some magic numbers missing here. X and Y are >> TBD constants to ensure we have some kind of rate limiting for the >> number of replacements allowed using some set of fees. >> >> What should they be? We can do some arithmetic to see what happens if >> you start with the biggest/lowest feerate transaction and do a bunch >> of replacements. Maybe we end up with values that are high enough to >> prevent abuse and make sense for applications/users that do RBF. >> >> ### Mempool Changes Need for Implementation >> >> As described in the mining score section above, >> we may want additional tooling to more accurately assess >> the economic gain of replacing transactions in our mempool. >> >> A few options have been discussed: >> >> * Calculate block templates on the fly when we need to consider a >> replacement. However, since replacements are [quite common][11] >> and the information might be useful for other things as well, >> it may be worth it to cache a block template. >> >> * Keep a persistent block template so that we know what transactions >> we would put in the next block. We need to remember the feerate >> at which each transaction was included in the template, because an >> ancestor package may be included in the same block template in >> multiple subsets. Transactions included earlier alter the ancestor >> feerate of the remaining transactions in the package. We also need >> to keep track of the new feerates of transactions left over. >> >> * Divide the mempool into two layers, "high feerate" and "low >> feerate." The high feerate layer contains ~1 block of packages with >> the highest ancestor feerates, and the low feerate layer contains >> everything else. At the edge of a block, we have a Knapsacky problem >> where the next highest ancestor feerate package might not fit, so we >> would probably want the high feerate layer ~2MvB or something to avoid >> underestimating the fees. >> >> ## Acknowledgements >> >> Thank you to everyone whose RBF-related suggestions, grievances, >> criticisms and ideas were incorporated in this document: >> Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, >> Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, >> Antoine Poinsot, Antoine Riard, Larry Ruane, >> S3RK and Bastien Teinturier. >> >> Thanks for reading! >> >> Best, >> Gloria >> >> [1]: >> https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md >> [2]: https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 >> [3]: >> https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea >> [4]: https://github.com/bitcoindevkit/bdk/issues/144 >> [5]: https://github.com/bitcoindevkit/bdk/issues/414 >> [6]: >> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html >> [7]: >> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 >> [8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 >> [9]: https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 >> [10]: >> https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md >> [11]: >> https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 >> [12]: https://github.com/bitcoin/bitcoin/pull/24007 >> [13]: >> https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 >> [14]: >> https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 39894 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-01-31 15:57 ` Bastien TEINTURIER @ 2022-02-01 1:56 ` Anthony Towns 2022-02-05 13:21 ` Michael Folkson 1 sibling, 0 replies; 25+ messages in thread From: Anthony Towns @ 2022-02-01 1:56 UTC (permalink / raw) To: Bastien TEINTURIER, Bitcoin Protocol Discussion On Mon, Jan 31, 2022 at 04:57:52PM +0100, Bastien TEINTURIER via bitcoin-dev wrote: > I'd like to propose a different way of looking at descendants that makes > it easier to design the new rules. The way I understand it, limiting the > impact on descendant transactions is only important for DoS protection, > not for incentive compatibility. I would argue that after evictions, > descendant transactions will be submitted again (because they represent > transactions that people actually want to make), I think that's backwards: we're trying to discourage people from wasting the network's bandwidth, which they would do by publishing transactions that will never get confirmed -- if they were to eventually get confirmed it wouldn't be a waste of bandwith, after all. But if the original descendent txs were that sort of spam, then they may well not be submitted again if the ancestor tx reaches a fee rate that's actually likely to confirm. I wonder sometimes if it could be sufficient to just have a relay rate limit and prioritise by ancestor feerate though. Maybe something like: - instead of adding txs to each peers setInventoryTxToSend immediately, set a mempool flag "relayed=false" - on a time delay, add the top N (by fee rate) "relayed=false" txs to each peer's setInventoryTxToSend and mark them as "relayed=true"; calculate how much kB those txs were, and do this again after SIZE/RATELIMIT seconds - don't include "relayed=false" txs when building blocks? - keep high-feerate evicted txs around for a while in case they get mined by someone else to improve compact block relay, a la the orphan pool? That way if the network is busy, any attempt to do low fee rate tx spam will just cause those txs to sit as relayed=false until they're replaced or the network becomes less busy and they're worth relaying. And your actual mempool accept policy can just be "is this tx a higher fee rate than the txs it replaces"... > Even if bitcoin core releases a new version with updated RBF rules, as a > wallet you'll need to keep using the old rules for a long time if you > want to be safe. All you need is for there to be *a* path that follows the new relay rules and gets from your node/wallet to perhaps 10% of hashpower, which seems like something wallet providers could construct relatively quickly? Cheers, aj ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-01-31 15:57 ` Bastien TEINTURIER 2022-02-01 1:56 ` Anthony Towns @ 2022-02-05 13:21 ` Michael Folkson 2022-02-07 10:22 ` Bastien TEINTURIER 1 sibling, 1 reply; 25+ messages in thread From: Michael Folkson @ 2022-02-05 13:21 UTC (permalink / raw) To: Bastien TEINTURIER, Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 38188 bytes --] Thanks for this Bastien (and Gloria for initially posting about this). I sympathetically skimmed the eclair PR (https://github.com/ACINQ/eclair/pull/2113) dealing with replaceable transactions fee bumping. There will continue to be a (hopefully) friendly tug of war on this probably for the rest of Bitcoin's existence. I am sure people like Luke, Prayank etc will (rightfully) continue to raise that Lightning and other second layer protocols shouldn't demand that policy rules be changed if there is a reason (e.g. DoS vector) for those rules on the base network. But if there are rules that have no upside, introduce unnecessary complexity for no reason and make Lightning implementers like Bastien's life miserable attempting to deal with them I really hope we can make progress on removing or simplifying them. This is why I think it is important to understand the rationales for introducing the rules in the first place (and why it is safe to remove them if indeed it is) and being as rigorous as possible on the rationales for introducing additional rules. It sounds like from Gloria's initial post we are still at a brainstorming phase (which is fine) but knowing what we know today I really hope we can learn from the mistakes of the original BIP 125, namely the Core implementation not matching the BIP and the sparse rationales for the rules. As Bastien says this is not criticizing the original BIP 125 authors, 7 years is a long time especially in Bitcoin world and they probably weren't thinking about Bastien sitting down to write an eclair PR in late 2021 (and reviewers of that PR) when they wrote the BIP in 2015. -- Michael Folkson Email: michaelfolkson at [protonmail.com](http://protonmail.com/) Keybase: michaelfolkson PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 ------- Original Message ------- On Monday, January 31st, 2022 at 3:57 PM, Bastien TEINTURIER via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi Gloria, > > Many thanks for raising awareness on these issues and constantly pushing > towards finding a better model. This work will highly improve the > security of any multi-party contract trying to build on top of bitcoin > (because most multi-party contracts will need to have timeout conditions > and participants will need to make some transactions confirm before a > timeout happens - otherwise they may lose funds). > > For starters, let me quickly explain why the current rules are hard to > work with in the context of lightning (but I believe most L2 protocols > will have the same issues). Feel free to skip this part if you are > already convinced. > > ## Motivation > > The biggest pain point is BIP 125 rule 2. > If I need to increase the fees of a time-sensitive transaction because > the feerate has been rising since I broadcast it, I may need to also pay > high fees just to produce a confirmed utxo that I can use. I'm actually > paying a high fee twice instead of once (and needlessly using on-chain > space, our scarcest asset, because we could have avoided that additional > transaction!). > > It also has some annoying "non-determinism". > Imagine that my transaction has been evicted from my mempool because its > feerate was too low. I could think "Great, that means I don't have to > apply BIP 125 restrictions, I can just fund this transaction as if it > were a new one!". But actually I do, because my transaction could still > be in miner's mempools and I have no way of knowing it...this means that > whenever I have broadcast a transaction, I must assume that I will > always need to abide by whatever replacement rules the network applies. > > Fortunately, as far as I understand it, this rule only exists because of > a previous implementation detail of bitcoin core, so there's simply no > good reason to keep it. > > The second biggest pain point is rule 3. It prevents me from efficiently > using my capital while it's unconfirmed. Whenever I'm using a big utxo > to fund a transaction, I will get a big change output, and it would > really be a waste to be unable to use that change output to fund other > transactions. In order to be capital-efficient, I will end up creating > descendant trees for my time-sensitive transactions. But as Gloria > explained, replacing all my children will cost me an absurdly large > amount of fees. So what I'm actually planning to do instead is to RBF > one of the descendants high enough to get the whole tree confirmed. > But if those descendants' timeouts were far in the future, that's a > waste, I paid a lot more fees for them than I should have. I'd like to > just replace my transaction and republish the invalidated children > independently. > > Rule 4 doesn't hurt as much as the two previous ones, I don't have too > much to say about it. > > To be fair to the BIP 125 authors, all of these scenarios were very hard > to forecast at the time this BIP was created. We needed years to build > on those rules to get a better understanding of their limitations and if > the rationale behind them made sense in the long term. > > ## Proposals > > I believe that now is a good time to re-think those, and I really like > Gloria's categorization of the design constraints. > > I'd like to propose a different way of looking at descendants that makes > it easier to design the new rules. The way I understand it, limiting the > impact on descendant transactions is only important for DoS protection, > not for incentive compatibility. I would argue that after evictions, > descendant transactions will be submitted again (because they represent > transactions that people actually want to make), so evicting them does > not have a negative impact on mining incentives (in a world where blocks > are full most of the time). > > I'm curious to hear other people's thoughts on that. If it makes sense, > I would propose the following very simple rules: > > 1. The transaction's ancestor absolute fees must be X% higher than the > previous transaction's ancestor fees > 2. The transaction's ancestor feerate must be Y% higher than the > previous transaction's ancestor feerate > > I believe it's completely ok to require increasing both the fees and > feerate if we don't take descendants into account, because you control > your ancestor set - whereas the descendant set may be completely out of > your control. > > This is very easy to use by wallets, because the ancestor set is easy to > obtain. And an important point is that the ancestor set is the same in > every mempool, whereas the descendant set is not (your mempool may have > rejected the last descendants, while other people's mempools may still > contain them). > > Because of that reason, I'd like to avoid having a rule that relies on > some size of the replaced descendant set: it may be valid in your > mempool but invalid in someone else's, which makes it exploitable for > pinning attacks. > > I believe these rules are incentive compatible (again, if you accept > the fact that the descendants will be re-submitted and mined as well, > so their fees aren't lost). > > Can we choose X and Y so that these two rules are also DoS-resistant? > Unfortunately I'm not sure, so maybe we'll need to add a third rule to > address that. But before we do, can someone detail what it costs for a > node to evict a descendant tree? Given that bitcoin core doesn't allow > chains of more than 25 transactions, the maximum number of transactions > being replaced will be bounded by 25 * N (where N is the number of > outputs of the transaction being replaced). If it's just O(n) pruning of > a graph, maybe that's ok? Or maybe we make X or Y depend on the number > of outputs of the transaction being replaced (this would need very > careful thoughts)? > > If you made it this far, thanks for reading! > A couple of comments on the previous messages: > >> Currently, if we see a transaction >> that has the same txid as one in the mempool, we reject it as a >> duplicate, even if the feerate is much higher. It's unclear to me if >> we have a very strong reason to change this, but noting it as a >> limitation of our current replacement policy. > > I don't see a strong reason from an L2 protocol's point of view yet, but > there are many unkown unknowns. But from a miner incentive's point of > view, we should keep the transaction with the higher feerate, shouldn't > we? In that case it's also a more efficient use of on-chain space, which > is a win, right? > >> We might have a more-or-less long transition period during which we support both... > > Yes, this is a long term thing. > Even if bitcoin core releases a new version with updated RBF rules, as a > wallet you'll need to keep using the old rules for a long time if you > want to be safe. > > But it's all the more reason to try to ship this as soon as possible, > this way maybe our grand-children will be able to benefit from it ;) > (just kidding on the timespan obviously). > > Cheers, > Bastien > > Le lun. 31 janv. 2022 à 00:11, Antoine Riard via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> a écrit : > >> Hi Gloria, >> >> Thanks for this RBF sum up. Few thoughts and more context comments if it can help other readers. >> >>> For starters, the absolute fee pinning attack is especially >>> problematic if we apply the same rules (i.e. Rule #3 and #4) in >>> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >>> LN channel. The mempool is rather full, so their pre-negotiated >>> commitment transactions' feerates would not be considered high >>> priority by miners. Bob broadcasts his commitment transaction and >>> attaches a very large child (100KvB with 100,000sat in fees) to his >>> anchor output. Alice broadcasts her commitment transaction with a >>> fee-bumping child (200vB with 50,000sat fees which is a generous >>> 250sat/vB), but this does not meet the absolute fee requirement. She >>> would need to add another 50,000sat to replace Bob's commitment >>> transaction. >> >> Solving LN pinning attacks, what we're aiming for is enabling a fair feerate bid between the counterparties, thus either forcing the adversary to overbid or to disengage from the confirmation competition. If the replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob to >> pick up the first option. Though if he does, that's a winning outcome for Alice, as one of the commitment transactions confirms and her time-sensitive second-stage HTLC can be subsequently confirmed. >> >>> It's unclear to me if >>> we have a very strong reason to change this, but noting it as a >>> limitation of our current replacement policy. See [#24007][12]. >> >> Deployment of Taproot opens interesting possibilities in the vaults/payment channels design space, where the tapscripts can commit to different set of timelocks/quorum of keys. Even if the pre-signed states stay symmetric, whoever is the publisher, the feerate cost to spend can fluctuate. >> >>> While this isn't completely broken, and the user interface is >>> secondary to the safety of the mempool policy >> >> I think with L2s transaction broadcast backend, the stability and clarity of the RBF user interface is primary. What we could be worried about is a too-much complex interface easing the way for an attacker to trigger your L2 node to issue policy-invalid chain of transactions. Especially, when we consider that an attacker might have leverage on chain of transactions composition ("force broadcast of commitment A then commitment B, knowing they will share a CPFP") or even transactions size ("overload commitment A with HTLCs"). >> >>> * If the original transaction is in the top {0.75MvB, 1MvB} of the >>> mempool, apply the current rules (absolute fees must increase and >>> pay for the replacement transaction's new bandwidth). Otherwise, use a >>> feerate-only rule. >> >> How this new replacement rule would behave if you have a parent in the "replace-by-feerate" half but the child is in the "replace-by-fee" one ? >> >> If we allow the replacement of the parent based on the feerate, we might decrease the top block absolute fees. >> >> If we block the replacement of the parent based on the feerate because the replacement absolute fees aren't above the replaced package, we still preclude a pinning vector. The child might be low-feerate junk and even attached to a low ancestor-score branch. >> >> If I'm correct on this limitation, maybe we could turn off the "replace-by-fee" behavior as soon as the mempool is fulfilled with a few blocks ? >> >>> * Rate-limit how many replacements we allow per prevout. >> >> Depending on how it is implemented, though I would be concerned it introduces a new pinning vector in the context of shared-utxo. If it's a hardcoded constant, it could be exhausted by an adversary starting at the lowest acceptable feerate then slowly increasing while still not reaching >> the top of the mempool. Same if it's time-based or block-based, no guarantee the replacement slot is honestly used by your counterparty. >> >> Further, an above-the-average replacement frequency might just be the reflection of your confirmation strategy reacting to block schedule or mempools historical data. As long as the feerate penalty is paid, I lean to allow replacement. >> (One solution could be to associate per-user "tag" to the LN transactions, where each "tag" would have its own replacement slots, but privacy?) >> >>> * Rate-limit transaction validation in general, per peer. >> >> I think we could improve on the Core's new transaction requester logic. Maybe we could bind the peer announced flow based on the feerate score (modulo validation time) of the previously validated transactions from that peer ? That said, while related to RBF, it sounds to me that enhancing Core's rate-limiting transaction strategy is a whole discussion in itself [0]. Especially ensuring it's tolerant to the specific requirements of LN & consorts. >> >>> What should they be? We can do some arithmetic to see what happens if >>> you start with the biggest/lowest feerate transaction and do a bunch >>> of replacements. Maybe we end up with values that are high enough to >>> prevent abuse and make sense for applications/users that do RBF. >> >> That's a good question. >> >> One observation is that the attacker can always renew the set of DoSy utxos to pursue the attack. So maybe we could pick up constants scaled on the block size ? That way an attacker would have to burn fees, thus deterring them from launching an attack. Even if the attackers are miners, they have to renounce their income to acquire new DoSy utxos. If a low-fee period, we could scale up the constants ? >> >> Overall, I think there is the deployment issue to warn of. Moving to a new set of RBF rules implies for a lot of Bitcoin applications to rewrite their RBF logics. We might have a more-or-less long transition period during which we support both... >> >> Cheers, >> Antoine >> >> [0] https://github.com/bitcoin/bitcoin/pull/21224 >> >> Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> a écrit : >> >>> Hi everyone, >>> >>> This post discusses limitations of current Bitcoin Core RBF policy and >>> attempts to start a conversation about how we can improve it, >>> summarizing some ideas that have been discussed. Please reply if you >>> have any new input on issues to be solved and ideas for improvement! >>> >>> Just in case I've screwed up the text wrapping again, another copy can be >>> found here: https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff >>> >>> ## Background >>> >>> Please feel free to skip this section if you are already familiar >>> with RBF. >>> >>> Nodes may receive *conflicting* unconfirmed transactions, aka >>> "double spends" of the same inputs. Instead of always keeping the >>> first transaction, since v0.12, Bitcoin Core mempool policy has >>> included a set of Replace-by-Fee (RBF) criteria that allows the second >>> transaction to replace the first one and any descendants it may have. >>> >>> Bitcoin Core RBF policy was previously documented as BIP 125. >>> The current RBF policy is documented [here][1]. In summary: >>> >>> 1. The directly conflicting transactions all signal replaceability >>> explicitly. >>> >>> 2. The replacement transaction only includes an unconfirmed input if >>> that input was included in one of the directly conflicting >>> transactions. >>> >>> 3. The replacement transaction pays an absolute fee of at least the >>> sum paid by the original transactions. >>> >>> 4. The additional fees pays for the replacement transaction's >>> bandwidth at or above the rate set by the node's *incremental relay >>> feerate*. >>> >>> 5. The sum of all directly conflicting transactions' descendant counts >>> (number of transactions inclusive of itself and its descendants) >>> does not exceed 100. >>> >>> We can split these rules into 3 categories/goals: >>> >>> - **Allow Opting Out**: Some applications/businesses are unable to >>> handle transactions that are replaceable (e.g. merchants that use >>> zero-confirmation transactions). We (try to) help these businesses by >>> honoring BIP125 signaling; we won't replace transactions that have not >>> opted in. >>> >>> - **Incentive Compatibility**: Ensure that our RBF policy would not >>> accept replacement transactions which would decrease fee profits >>> of a miner. In general, if our mempool policy deviates from what is >>> economically rational, it's likely that the transactions in our >>> mempool will not match the ones in miners' mempools, making our >>> fee estimation, compact block relay, and other mempool-dependent >>> functions unreliable. Incentive-incompatible policy may also >>> encourage transaction submission through routes other than the p2p >>> network, harming censorship-resistance and privacy of Bitcoin payments. >>> >>> - **DoS Protection**: Limit two types of DoS attacks on the node's >>> mempool: (1) the number of times a transaction can be replaced and >>> (2) the volume of transactions that can be evicted during a >>> replacement. >>> >>> Even more abstract: our goal is to make a replacement policy that >>> results in a useful interface for users and safe policy for >>> node operators. >>> >>> ## Motivation >>> >>> There are a number of known problems with the current RBF policy. >>> Many of these shortcomings exist due to mempool limitations at the >>> time RBF was implemented or result from new types of Bitcoin usage; >>> they are not criticisms of the original design. >>> >>> ### Pinning Attacks >>> >>> The most pressing concern is that attackers may take advantage of >>> limitations in RBF policy to prevent other users' transactions from >>> being mined or getting accepted as a replacement. >>> >>> #### SIGHASH_ANYONECANPAY Pinning >>> >>> BIP125#2 can be bypassed by creating intermediary transactions to be >>> replaced together. Anyone can simply split a 1-input 1-output >>> transaction off from the replacement transaction, then broadcast the >>> transaction as is. This can always be done, and quite cheaply. More >>> details in [this comment][2]. >>> >>> In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, >>> anybody can just attach a low feerate parent to this transaction and >>> lower its ancestor feerate. Even if you require SIGHASH\_ALL which >>> prevents an attacker from changing any outputs, the input can be a >>> very low amount (e.g. just above the dust limit) from a low-fee >>> ancestor and still bring down the ancestor feerate of the transaction. >>> >>> TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and >>> signals replaceability, regardless of the feerate you broadcast at, an >>> attacker can lower its mining priority by adding an ancestor. >>> >>> #### Absolute Fee >>> >>> The restriction of requiring replacement transactions to increase the >>> absolute fee of the mempool has been described as "bonkers." If the >>> original transaction has a very large descendant that pays a large >>> amount of fees, even if it has a low feerate, the replacement >>> transaction must now pay those fees in order to meet Rule #3. >>> >>> #### Package RBF >>> >>> There are a number of reasons why, in order to enable Package RBF, we >>> cannot use the same criteria. >>> >>> For starters, the absolute fee pinning attack is especially >>> problematic if we apply the same rules (i.e. Rule #3 and #4) in >>> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >>> LN channel. The mempool is rather full, so their pre-negotiated >>> commitment transactions' feerates would not be considered high >>> priority by miners. Bob broadcasts his commitment transaction and >>> attaches a very large child (100KvB with 100,000sat in fees) to his >>> anchor output. Alice broadcasts her commitment transaction with a >>> fee-bumping child (200vB with 50,000sat fees which is a generous >>> 250sat/vB), but this does not meet the absolute fee requirement. She >>> would need to add another 50,000sat to replace Bob's commitment >>> transaction. >>> >>> Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be >>> broken for packages containing transactions already in the mempool, >>> explained [here][7]. >>> >>> Note: I originally [proposed][6] Package RBF using the same Rule #3 >>> and #4 before I realized how significant this pinning attack is. I'm >>> retracting that proposal, and a new set of Package RBF rules would >>> follow from whatever the new individual RBF rules end up being. >>> >>> #### Same Txid Different Witness >>> >>> Two transactions with the same non-witness data but different >>> witnesses have the same txid but different wtxid, and the same fee but >>> not necessarily the same feerate. Currently, if we see a transaction >>> that has the same txid as one in the mempool, we reject it as a >>> duplicate, even if the feerate is much higher. It's unclear to me if >>> we have a very strong reason to change this, but noting it as a >>> limitation of our current replacement policy. See [#24007][12]. >>> >>> ### User Interface >>> >>> #### Using Unconfirmed UTXOs to Fund Replacements >>> >>> The restriction of only allowing confirmed UTXOs for funding a >>> fee-bump (Rule #2) can hurt users trying to fee-bump their >>> transactions and complicate wallet implementations. If the original >>> transaction's output value isn't sufficient to fund a fee-bump and/or >>> all of the user's other UTXOs are unconfirmed, they might not be able >>> to fund a replacement transaction. Wallet developers also need to >>> treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which >>> adds complexity to wallet logic. For example, see BDK issues [#144][4] >>> and [#414][5]. >>> >>> #### Interface Not Suitable for Coin Selection >>> >>> Currently, a user cannot simply create a replacement transaction >>> targeting a specific feerate or meeting a minimum fee amount and >>> expect to meet the RBF criteria. The fee amount depends on the size of >>> the replacement transaction, and feerate is almost irrelevant. >>> >>> Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the >>> replacement. It [estimates][13] a feerate which is "wallet incremental >>> relay fee" (a conservative overestimation of the node's incremental >>> relay fee) higher than the original transaction, selects coins for >>> that feerate, and hopes that it meets the RBF rules. It never fails >>> Rule #3 and #4 because it uses all original inputs and refuses to >>> bump a transaction with mempool descendants. >>> >>> This is suboptimal, but is designed to work with the coin selection >>> engine: select a feerate first, and then add fees to cover it. >>> Following the exact RBF rules would require working the other way >>> around: based on how much fees we've added to the transaction and its >>> current size, calculate the feerate to see if we meet Rule #4. >>> >>> While this isn't completely broken, and the user interface is >>> secondary to the safety of the mempool policy, we can do much better. >>> A much more user-friendly interface would depend *only* on the >>> fee and size of the original transactions. >>> >>> ### Updates to Mempool and Mining >>> >>> Since RBF was first implemented, a number of improvements have been >>> made to mempool and mining logic. For example, we now use ancestor >>> feerates in mining (allowing CPFP), and keep track of ancestor >>> packages in the mempool. >>> >>> ## Ideas for Improvements >>> >>> ### Goals >>> >>> To summarize, these seem to be desired changes, in order of priority: >>> >>> 1. Remove Rule #3. The replacement should not be *required* to pay >>> higher absolute fees. >>> >>> 2. Make it impossible for a replacement transaction to have a lower >>> mining score than the original transaction(s). This would eliminate >>> the `SIGHASH\_ANYONECANPAY` pinning attack. >>> >>> 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. >>> >>> 4. Create a more helpful interface that helps wallet fund replacement >>> transactions that aim for a feerate and fee. >>> >>> ### A Different Model for Fees >>> >>> For incentive compatibility, I believe there are different >>> formulations we should consider. Most importantly, if we want to get >>> rid of the absolute fee rule, we can no longer think of it as "the >>> transaction needs to pay for its own bandwidth," since we won't always >>> be getting additional fees. That means we need a new method of >>> rate-limiting replacements that doesn't require additional fees every >>> time. >>> >>> While it makes sense to think about monetary costs when launching a >>> specific type of attack, given that the fees are paid to the miner and >>> not to the mempool operators, maybe it doesn't make much sense to >>> think about "paying for bandwidth". Maybe we should implement >>> transaction validation rate-limiting differently, e.g. building it >>> into the P2P layer instead of the mempool policy layer. >>> >>> Recently, Suhas gave a [formulation][8] for incentive compatibility >>> that made sense to me: "are the fees expected to be paid in the next >>> (N?) blocks higher or lower if we process this transaction?" >>> >>> I started by thinking about this where N=1 or `1 + p`. >>> Here, a rational miner is looking at what fees they would >>> collect in the next block, and then some proportion `p` of the rest of >>> the blocks based on their hashrate. We're assuming `p` isn't *so high* >>> that they would be okay with lower absolute fees in the next 1 block. >>> We're also assuming `p` isn't *so low* that the miner doesn't care >>> about what's left of the mempool after this block. >>> >>> A tweak to this formulation is "if we process this transaction, would >>> the fees in the next 1 block higher or lower, and is the feerate >>> density of the rest of the mempool higher or lower?" This is pretty >>> similar, where N=1, but we consider the rest of the mempool by feerate >>> rather than fees. >>> >>> ### Mining Score of a Mempool Transaction >>> >>> We are often interested in finding out what >>> the "mining score" of a transaction in the mempool is. That is, when >>> the transaction is considered in block template building, what is the >>> feerate it is considered at? >>> >>> Obviously, it's not the transaction's individual feerate. Bitcoin Core >>> [mining code sorts][14] transactions by their ancestor feerate and >>> includes them packages at a time, keeping track of how this affects the >>> package feerates of remaining transactions in the mempool. >>> >>> *ancestor feerate*: Ancestor feerate is easily accessible information, >>> but it's not accurate either, because it doesn't take into account the >>> fact that subsets of a transaction's ancestor set can be included >>> without it. For example, ancestors may have high feerates on their own >>> or we may have [high feerate siblings][8]. >>> >>> TLDR: *Looking at the current ancestor feerate of a transaction is >>> insufficient to tell us what feerate it will be considered at when >>> building a block template in the future.* >>> >>> *min(individual feerate, ancestor feerate)*: Another >>> heuristic that is simple to calculate based on current mempool tooling >>> is to use the [minimum of a transaction's individual score and its >>> ancestor score][10] as a conservative measure. But this can >>> overestimate as well (see the example below). >>> >>> *min ancestor feerate(tx + possible ancestor subsets)* We can also >>> take the minimum of every possible ancestor subset, but this can be >>> computationally expensive since there can be lots and lots of ancestor >>> subsets. >>> >>> *max ancestor feerate(tx + possible descendant subsets)*: Another idea >>> is to use the [maximum ancestor score of the transaction + each of its >>> descendants][9]. This doesn't work either; it has the same blindspot >>> of ancestor subsets being mined on their own. >>> >>> #### Mining Score Example >>> >>> Here's an example illustrating why mining score is tricky to >>> efficiently calculate for mempool transactions: >>> >>> Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), >>> C(9sat/vB), D(5sat/vB). >>> The layout is: grandparent A, parent B, and two children C and D. >>> >>> ``` >>> A >>> ^ >>> B >>> ^ ^ >>> C D >>> ``` >>> >>> A miner using ancestor packages to build block templates will first >>> include A with a mining score of 21. Next, the miner will include B and >>> C with a mining score of 6. This leaves D, with a mining score of 5. >>> >>> Note: in this case, mining by ancestor feerate results in the most >>> rational decisions, but [a candidate set-based approach][10] which >>> makes ancestor feerate much less relevant could >>> be more advantageous in other situations. >>> >>> Here is a chart showing the "true" mining score alongside the values >>> calculating using imperfect heuristics described above. All of them >>> can overestimate or underestimate. >>> >>> ``` >>> A B C D >>> mining score | 21 | 6 | 6 | 5 | >>> ancestor feerate | 21 | 11 | 10.3 | 9 | >>> min(individual, ancestor) | 21 | 1 | 9 | 5 | >>> min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | >>> max(tx + descendants subsets) | 21 | 9 | 9 | 5 | >>> >>> ``` >>> >>> Possibly the best solution for finding the "mining score" of a >>> transaction is to build a block template, see what feerate each >>> package is included at. Perhaps at some cutoff, remaining mempool >>> transactions can be estimated using some heuristic that leans >>> {overestimating, underestimating} depending on the situation. >>> >>> Mining score seems to be relevant in multiple places: Murch and I >>> recently [found][3] that it would be very important in >>> "ancestor-aware" funding of transactions (the wallet doesn't >>> incorporate ancestor fees when using unconfirmed transactions in coin >>> selection, which is a bug we want to fix). >>> >>> In general, it would be nice to know the exact mining priority of >>> one's unconfirmed transaction is. I can think of a few block/mempool >>> explorers who might want to display this information for users. >>> >>> ### RBF Improvement Proposals >>> >>> After speaking to quite a few people, here are some suggestions >>> for improvements that I have heard: >>> >>> * The ancestor score of the replacement must be {5, 10, N}% higher >>> than that of every original transaction. >>> >>> * The ancestor score of the replacement must be 1sat/vB higher than >>> that of every original transaction. >>> >>> * If the original transaction is in the top {0.75MvB, 1MvB} of the >>> mempool, apply the current rules (absolute fees must increase and >>> pay for the replacement transaction's new bandwidth). Otherwise, use a >>> feerate-only rule. >>> >>> * If fees don't increase, the size of the replacement transaction must >>> decrease by at least N%. >>> >>> * Rate-limit how many replacements we allow per prevout. >>> >>> * Rate-limit transaction validation in general, per peer. >>> >>> Perhaps some others on the mailing list can chime in to throw other >>> ideas into the ring and/or combine some of these rules into a sensible >>> policy. >>> >>> #### Replace by Feerate Only >>> >>> I don't think there's going to be a single-line feerate-based >>> rule that can incorporate everything we need. >>> On one hand, a feerate-only approach helps eliminate the issues >>> associated with Rule #3. On the other hand, I believe the main concern >>> with a feerate-only approach is how to rate limit replacements. We >>> don't want to enable an attack such as: >>> >>> 1. Attacker broadcasts large, low-feerate transaction, and attaches a >>> chain of descendants. >>> >>> 2. The attacker replaces the transaction with a smaller but higher >>> feerate transaction, attaching a new chain of descendants. >>> >>> 3. Repeat 1000 times. >>> >>> #### Fees in Next Block and Feerate for the Rest of the Mempool >>> >>> Perhaps we can look at replacements like this: >>> >>> 1. Calculate the directly conflicting transactions and, with their >>> descendants, the original transactions. Check signaling. Limit the >>> total volume (e.g. can't be more than 100 total or 1MvB or something). >>> >>> 2. Find which original transactions would be in the next ~1 block. The >>> replacement must pay at least this amount + X% in absolute fees. This >>> guarantees that the fees of the next block doesn't decrease. >>> >>> 3. Find which transactions would be left in the mempool after that ~1 >>> block. The replacement's feerate must be Y% higher than the maximum >>> mining score of these transactions. This guarantees that you now have >>> only *better* candidates in your after-this-block mempool than you did >>> before, even if the size and fees the transactions decrease. >>> >>> 4. Now you have two numbers: a minimum absolute fee amount and a >>> minimum feerate. Check to see if the replacement(s) meet these >>> minimums. Also, a wallet would be able to ask the node "What fee and >>> feerate would I need to put on a transaction replacing this?" and use >>> this information to fund a replacement transaction, without needing to >>> guess or overshoot. >>> >>> Obviously, there are some magic numbers missing here. X and Y are >>> TBD constants to ensure we have some kind of rate limiting for the >>> number of replacements allowed using some set of fees. >>> >>> What should they be? We can do some arithmetic to see what happens if >>> you start with the biggest/lowest feerate transaction and do a bunch >>> of replacements. Maybe we end up with values that are high enough to >>> prevent abuse and make sense for applications/users that do RBF. >>> >>> ### Mempool Changes Need for Implementation >>> >>> As described in the mining score section above, >>> we may want additional tooling to more accurately assess >>> the economic gain of replacing transactions in our mempool. >>> >>> A few options have been discussed: >>> >>> * Calculate block templates on the fly when we need to consider a >>> replacement. However, since replacements are [quite common][11] >>> and the information might be useful for other things as well, >>> it may be worth it to cache a block template. >>> >>> * Keep a persistent block template so that we know what transactions >>> we would put in the next block. We need to remember the feerate >>> at which each transaction was included in the template, because an >>> ancestor package may be included in the same block template in >>> multiple subsets. Transactions included earlier alter the ancestor >>> feerate of the remaining transactions in the package. We also need >>> to keep track of the new feerates of transactions left over. >>> >>> * Divide the mempool into two layers, "high feerate" and "low >>> feerate." The high feerate layer contains ~1 block of packages with >>> the highest ancestor feerates, and the low feerate layer contains >>> everything else. At the edge of a block, we have a Knapsacky problem >>> where the next highest ancestor feerate package might not fit, so we >>> would probably want the high feerate layer ~2MvB or something to avoid >>> underestimating the fees. >>> >>> ## Acknowledgements >>> >>> Thank you to everyone whose RBF-related suggestions, grievances, >>> criticisms and ideas were incorporated in this document: >>> Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, >>> Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, >>> Antoine Poinsot, Antoine Riard, Larry Ruane, >>> >>> S3RK and Bastien Teinturier. >>> >>> Thanks for reading! >>> >>> Best, >>> Gloria >>> [1]: https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md >>> [2]: https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 >>> [3]: https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea >>> [4]: https://github.com/bitcoindevkit/bdk/issues/144 >>> [5]: https://github.com/bitcoindevkit/bdk/issues/414 >>> [6]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html >>> [7]: https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 >>> [8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 >>> [9]: https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 >>> [10]: https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md >>> [11]: https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 >>> [12]: https://github.com/bitcoin/bitcoin/pull/24007 >>> [13]: https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 >>> >>> [14]: https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [-- Attachment #2: Type: text/html, Size: 44594 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-05 13:21 ` Michael Folkson @ 2022-02-07 10:22 ` Bastien TEINTURIER 2022-02-07 11:16 ` Gloria Zhao 0 siblings, 1 reply; 25+ messages in thread From: Bastien TEINTURIER @ 2022-02-07 10:22 UTC (permalink / raw) To: Michael Folkson, Anthony Towns; +Cc: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 42034 bytes --] Good morning, > The tricky question is what happens when X arrives on its own and it > might be that no one ever sends a replacement for B,C,D) It feels ok to me, but this is definitely arguable. It covers the fact that B,C,D could have been fake transactions whose sole purpose was to do a pinning attack: in that case the attacker would have found a way to ensure these transactions don't confirm anyway (or pay minimal/negligible fees). If these transactions were legitimate, I believe that their owners would remake them at some point (because these transactions reflect a business relationship that needed to happen, so it should very likely still happen). It's probably hard to verify because the new corresponding transactions may have nothing in common with the first, but I think the simplifications it offers for wallets is worth it (which is just my opinion and needs more scrutiny/feedback). > But if your backlog's feerate does drop off, *and* that matters, then > I don't think you can ignore the impact of the descendent transactions > that you might not get a replacement for. That is because you're only taking into account the current backlog, and not taking into account the fact that new items will be added to it soon to replace the evicted descendants. But I agree that this is a bet: we can't predict the future and guarantee these replacements will come. It is really a trade-off, ignoring descendents provides a much simpler contract that doesn't vary from one mempool to another, but when your backlog isn't full enough, you may lose some future profits if transactions don't come in later. > I think "Y% higher" rather than just "higher" is only useful for > rate-limiting, not incentive compatibility. (Though maybe it helps > stabilise a greedy algorithm in some cases?) That's true. I claimed these policies only address incentives, but using a percentage increase addresses rate-limiting a bit as well (I couldn't resist trying to do at least something for it!). I find it a very easy mechanism to implement, while choosing an absolute value is hard (it's always easier to think in relatives than absolutes). > This is why I think it is important to understand the rationales for introducing the rules in the first place I completely agree. As you mentioned, we are still in brainstorming phase, once (if?) we start to converge on what could be better policies, we do need to clearly explain each policy's expected goal. That will let future Bastien writing code in 2030 clearly highlight why the 2022 rules don't make sense anymore! Cheers, Bastien Le sam. 5 févr. 2022 à 14:22, Michael Folkson <michaelfolkson@protonmail.com> a écrit : > Thanks for this Bastien (and Gloria for initially posting about this). > > I sympathetically skimmed the eclair PR ( > https://github.com/ACINQ/eclair/pull/2113) dealing with replaceable > transactions fee bumping. > > There will continue to be a (hopefully) friendly tug of war on this > probably for the rest of Bitcoin's existence. I am sure people like Luke, > Prayank etc will (rightfully) continue to raise that Lightning and other > second layer protocols shouldn't demand that policy rules be changed if > there is a reason (e.g. DoS vector) for those rules on the base network. > But if there are rules that have no upside, introduce unnecessary > complexity for no reason and make Lightning implementers like Bastien's > life miserable attempting to deal with them I really hope we can make > progress on removing or simplifying them. > > This is why I think it is important to understand the rationales for > introducing the rules in the first place (and why it is safe to remove them > if indeed it is) and being as rigorous as possible on the rationales for > introducing additional rules. It sounds like from Gloria's initial post we > are still at a brainstorming phase (which is fine) but knowing what we know > today I really hope we can learn from the mistakes of the original BIP 125, > namely the Core implementation not matching the BIP and the sparse > rationales for the rules. As Bastien says this is not criticizing the > original BIP 125 authors, 7 years is a long time especially in Bitcoin > world and they probably weren't thinking about Bastien sitting down to > write an eclair PR in late 2021 (and reviewers of that PR) when they wrote > the BIP in 2015. > > -- > Michael Folkson > Email: michaelfolkson at protonmail.com > Keybase: michaelfolkson > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 > > > > ------- Original Message ------- > On Monday, January 31st, 2022 at 3:57 PM, Bastien TEINTURIER via > bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: > > Hi Gloria, > > Many thanks for raising awareness on these issues and constantly pushing > towards finding a better model. This work will highly improve the > security of any multi-party contract trying to build on top of bitcoin > (because most multi-party contracts will need to have timeout conditions > and participants will need to make some transactions confirm before a > timeout happens - otherwise they may lose funds). > > For starters, let me quickly explain why the current rules are hard to > work with in the context of lightning (but I believe most L2 protocols > will have the same issues). Feel free to skip this part if you are > already convinced. > > ## Motivation > > The biggest pain point is BIP 125 rule 2. > If I need to increase the fees of a time-sensitive transaction because > the feerate has been rising since I broadcast it, I may need to also pay > high fees just to produce a confirmed utxo that I can use. I'm actually > paying a high fee twice instead of once (and needlessly using on-chain > space, our scarcest asset, because we could have avoided that additional > transaction!). > > It also has some annoying "non-determinism". > Imagine that my transaction has been evicted from my mempool because its > feerate was too low. I could think "Great, that means I don't have to > apply BIP 125 restrictions, I can just fund this transaction as if it > were a new one!". But actually I do, because my transaction could still > be in miner's mempools and I have no way of knowing it...this means that > whenever I have broadcast a transaction, I must assume that I will > always need to abide by whatever replacement rules the network applies. > > Fortunately, as far as I understand it, this rule only exists because of > a previous implementation detail of bitcoin core, so there's simply no > good reason to keep it. > > The second biggest pain point is rule 3. It prevents me from efficiently > using my capital while it's unconfirmed. Whenever I'm using a big utxo > to fund a transaction, I will get a big change output, and it would > really be a waste to be unable to use that change output to fund other > transactions. In order to be capital-efficient, I will end up creating > descendant trees for my time-sensitive transactions. But as Gloria > explained, replacing all my children will cost me an absurdly large > amount of fees. So what I'm actually planning to do instead is to RBF > one of the descendants high enough to get the whole tree confirmed. > But if those descendants' timeouts were far in the future, that's a > waste, I paid a lot more fees for them than I should have. I'd like to > just replace my transaction and republish the invalidated children > independently. > > Rule 4 doesn't hurt as much as the two previous ones, I don't have too > much to say about it. > > To be fair to the BIP 125 authors, all of these scenarios were very hard > to forecast at the time this BIP was created. We needed years to build > on those rules to get a better understanding of their limitations and if > the rationale behind them made sense in the long term. > > ## Proposals > > I believe that now is a good time to re-think those, and I really like > Gloria's categorization of the design constraints. > > I'd like to propose a different way of looking at descendants that makes > it easier to design the new rules. The way I understand it, limiting the > impact on descendant transactions is only important for DoS protection, > not for incentive compatibility. I would argue that after evictions, > descendant transactions will be submitted again (because they represent > transactions that people actually want to make), so evicting them does > not have a negative impact on mining incentives (in a world where blocks > are full most of the time). > > I'm curious to hear other people's thoughts on that. If it makes sense, > I would propose the following very simple rules: > > 1. The transaction's ancestor absolute fees must be X% higher than the > previous transaction's ancestor fees > 2. The transaction's ancestor feerate must be Y% higher than the > previous transaction's ancestor feerate > > I believe it's completely ok to require increasing both the fees and > feerate if we don't take descendants into account, because you control > your ancestor set - whereas the descendant set may be completely out of > your control. > > This is very easy to use by wallets, because the ancestor set is easy to > obtain. And an important point is that the ancestor set is the same in > every mempool, whereas the descendant set is not (your mempool may have > rejected the last descendants, while other people's mempools may still > contain them). > > Because of that reason, I'd like to avoid having a rule that relies on > some size of the replaced descendant set: it may be valid in your > mempool but invalid in someone else's, which makes it exploitable for > pinning attacks. > > I believe these rules are incentive compatible (again, if you accept > the fact that the descendants will be re-submitted and mined as well, > so their fees aren't lost). > > Can we choose X and Y so that these two rules are also DoS-resistant? > Unfortunately I'm not sure, so maybe we'll need to add a third rule to > address that. But before we do, can someone detail what it costs for a > node to evict a descendant tree? Given that bitcoin core doesn't allow > chains of more than 25 transactions, the maximum number of transactions > being replaced will be bounded by 25 * N (where N is the number of > outputs of the transaction being replaced). If it's just O(n) pruning of > a graph, maybe that's ok? Or maybe we make X or Y depend on the number > of outputs of the transaction being replaced (this would need very > careful thoughts)? > > If you made it this far, thanks for reading! > A couple of comments on the previous messages: > > > Currently, if we see a transaction > > that has the same txid as one in the mempool, we reject it as a > > duplicate, even if the feerate is much higher. It's unclear to me if > > we have a very strong reason to change this, but noting it as a > > limitation of our current replacement policy. > > I don't see a strong reason from an L2 protocol's point of view yet, but > there are many unkown unknowns. But from a miner incentive's point of > view, we should keep the transaction with the higher feerate, shouldn't > we? In that case it's also a more efficient use of on-chain space, which > is a win, right? > > > We might have a more-or-less long transition period during which we > support both... > > Yes, this is a long term thing. > Even if bitcoin core releases a new version with updated RBF rules, as a > wallet you'll need to keep using the old rules for a long time if you > want to be safe. > > But it's all the more reason to try to ship this as soon as possible, > this way maybe our grand-children will be able to benefit from it ;) > (just kidding on the timespan obviously). > > Cheers, > Bastien > > Le lun. 31 janv. 2022 à 00:11, Antoine Riard via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> a écrit : > >> Hi Gloria, >> >> Thanks for this RBF sum up. Few thoughts and more context comments if it >> can help other readers. >> >> > For starters, the absolute fee pinning attack is especially >> > problematic if we apply the same rules (i.e. Rule #3 and #4) in >> > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >> > LN channel. The mempool is rather full, so their pre-negotiated >> > commitment transactions' feerates would not be considered high >> > priority by miners. Bob broadcasts his commitment transaction and >> > attaches a very large child (100KvB with 100,000sat in fees) to his >> > anchor output. Alice broadcasts her commitment transaction with a >> > fee-bumping child (200vB with 50,000sat fees which is a generous >> > 250sat/vB), but this does not meet the absolute fee requirement. She >> > would need to add another 50,000sat to replace Bob's commitment >> > transaction. >> >> Solving LN pinning attacks, what we're aiming for is enabling a fair >> feerate bid between the counterparties, thus either forcing the adversary >> to overbid or to disengage from the confirmation competition. If the >> replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob >> to >> pick up the first option. Though if he does, that's a winning outcome for >> Alice, as one of the commitment transactions confirms and her >> time-sensitive second-stage HTLC can be subsequently confirmed. >> >> > It's unclear to me if >> > we have a very strong reason to change this, but noting it as a >> > limitation of our current replacement policy. See [#24007][12]. >> >> Deployment of Taproot opens interesting possibilities in the >> vaults/payment channels design space, where the tapscripts can commit to >> different set of timelocks/quorum of keys. Even if the pre-signed states >> stay symmetric, whoever is the publisher, the feerate cost to spend can >> fluctuate. >> >> > While this isn't completely broken, and the user interface is >> > secondary to the safety of the mempool policy >> >> I think with L2s transaction broadcast backend, the stability and clarity >> of the RBF user interface is primary. What we could be worried about is a >> too-much complex interface easing the way for an attacker to trigger your >> L2 node to issue policy-invalid chain of transactions. Especially, when we >> consider that an attacker might have leverage on chain of transactions >> composition ("force broadcast of commitment A then commitment B, knowing >> they will share a CPFP") or even transactions size ("overload commitment A >> with HTLCs"). >> >> > * If the original transaction is in the top {0.75MvB, 1MvB} of the >> > mempool, apply the current rules (absolute fees must increase and >> > pay for the replacement transaction's new bandwidth). Otherwise, use a >> > feerate-only rule. >> >> How this new replacement rule would behave if you have a parent in the >> "replace-by-feerate" half but the child is in the "replace-by-fee" one ? >> >> If we allow the replacement of the parent based on the feerate, we might >> decrease the top block absolute fees. >> >> If we block the replacement of the parent based on the feerate because >> the replacement absolute fees aren't above the replaced package, we still >> preclude a pinning vector. The child might be low-feerate junk and even >> attached to a low ancestor-score branch. >> >> If I'm correct on this limitation, maybe we could turn off the >> "replace-by-fee" behavior as soon as the mempool is fulfilled with a few >> blocks ? >> >> > * Rate-limit how many replacements we allow per prevout. >> >> Depending on how it is implemented, though I would be concerned it >> introduces a new pinning vector in the context of shared-utxo. If it's a >> hardcoded constant, it could be exhausted by an adversary starting at the >> lowest acceptable feerate then slowly increasing while still not reaching >> the top of the mempool. Same if it's time-based or block-based, no >> guarantee the replacement slot is honestly used by your counterparty. >> >> Further, an above-the-average replacement frequency might just be the >> reflection of your confirmation strategy reacting to block schedule or >> mempools historical data. As long as the feerate penalty is paid, I lean to >> allow replacement. >> >> (One solution could be to associate per-user "tag" to the LN >> transactions, where each "tag" would have its own replacement slots, but >> privacy?) >> >> > * Rate-limit transaction validation in general, per peer. >> >> I think we could improve on the Core's new transaction requester logic. >> Maybe we could bind the peer announced flow based on the feerate score >> (modulo validation time) of the previously validated transactions from that >> peer ? That said, while related to RBF, it sounds to me that enhancing >> Core's rate-limiting transaction strategy is a whole discussion in itself >> [0]. Especially ensuring it's tolerant to the specific requirements of LN & >> consorts. >> >> > What should they be? We can do some arithmetic to see what happens if >> > you start with the biggest/lowest feerate transaction and do a bunch >> > of replacements. Maybe we end up with values that are high enough to >> > prevent abuse and make sense for applications/users that do RBF. >> >> That's a good question. >> >> One observation is that the attacker can always renew the set of DoSy >> utxos to pursue the attack. So maybe we could pick up constants scaled on >> the block size ? That way an attacker would have to burn fees, thus >> deterring them from launching an attack. Even if the attackers are miners, >> they have to renounce their income to acquire new DoSy utxos. If a low-fee >> period, we could scale up the constants ? >> >> >> Overall, I think there is the deployment issue to warn of. Moving to a >> new set of RBF rules implies for a lot of Bitcoin applications to rewrite >> their RBF logics. We might have a more-or-less long transition period >> during which we support both... >> >> Cheers, >> Antoine >> >> [0] https://github.com/bitcoin/bitcoin/pull/21224 >> >> Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> a écrit : >> >>> Hi everyone, >>> >>> This post discusses limitations of current Bitcoin Core RBF policy and >>> attempts to start a conversation about how we can improve it, >>> summarizing some ideas that have been discussed. Please reply if you >>> have any new input on issues to be solved and ideas for improvement! >>> >>> Just in case I've screwed up the text wrapping again, another copy can be >>> found here: >>> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff >>> >>> ## Background >>> >>> Please feel free to skip this section if you are already familiar >>> with RBF. >>> >>> Nodes may receive *conflicting* unconfirmed transactions, aka >>> "double spends" of the same inputs. Instead of always keeping the >>> first transaction, since v0.12, Bitcoin Core mempool policy has >>> included a set of Replace-by-Fee (RBF) criteria that allows the second >>> transaction to replace the first one and any descendants it may have. >>> >>> Bitcoin Core RBF policy was previously documented as BIP 125. >>> The current RBF policy is documented [here][1]. In summary: >>> >>> 1. The directly conflicting transactions all signal replaceability >>> explicitly. >>> >>> 2. The replacement transaction only includes an unconfirmed input if >>> that input was included in one of the directly conflicting >>> transactions. >>> >>> 3. The replacement transaction pays an absolute fee of at least the >>> sum paid by the original transactions. >>> >>> 4. The additional fees pays for the replacement transaction's >>> bandwidth at or above the rate set by the node's *incremental relay >>> feerate*. >>> >>> 5. The sum of all directly conflicting transactions' descendant counts >>> (number of transactions inclusive of itself and its descendants) >>> does not exceed 100. >>> >>> We can split these rules into 3 categories/goals: >>> >>> - **Allow Opting Out**: Some applications/businesses are unable to >>> handle transactions that are replaceable (e.g. merchants that use >>> zero-confirmation transactions). We (try to) help these businesses by >>> honoring BIP125 signaling; we won't replace transactions that have not >>> opted in. >>> >>> - **Incentive Compatibility**: Ensure that our RBF policy would not >>> accept replacement transactions which would decrease fee profits >>> of a miner. In general, if our mempool policy deviates from what is >>> economically rational, it's likely that the transactions in our >>> mempool will not match the ones in miners' mempools, making our >>> fee estimation, compact block relay, and other mempool-dependent >>> functions unreliable. Incentive-incompatible policy may also >>> encourage transaction submission through routes other than the p2p >>> network, harming censorship-resistance and privacy of Bitcoin payments. >>> >>> - **DoS Protection**: Limit two types of DoS attacks on the node's >>> mempool: (1) the number of times a transaction can be replaced and >>> (2) the volume of transactions that can be evicted during a >>> replacement. >>> >>> Even more abstract: our goal is to make a replacement policy that >>> results in a useful interface for users and safe policy for >>> node operators. >>> >>> ## Motivation >>> >>> There are a number of known problems with the current RBF policy. >>> Many of these shortcomings exist due to mempool limitations at the >>> time RBF was implemented or result from new types of Bitcoin usage; >>> they are not criticisms of the original design. >>> >>> ### Pinning Attacks >>> >>> The most pressing concern is that attackers may take advantage of >>> limitations in RBF policy to prevent other users' transactions from >>> being mined or getting accepted as a replacement. >>> >>> #### SIGHASH_ANYONECANPAY Pinning >>> >>> BIP125#2 can be bypassed by creating intermediary transactions to be >>> replaced together. Anyone can simply split a 1-input 1-output >>> transaction off from the replacement transaction, then broadcast the >>> transaction as is. This can always be done, and quite cheaply. More >>> details in [this comment][2]. >>> >>> In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, >>> anybody can just attach a low feerate parent to this transaction and >>> lower its ancestor feerate. Even if you require SIGHASH\_ALL which >>> prevents an attacker from changing any outputs, the input can be a >>> very low amount (e.g. just above the dust limit) from a low-fee >>> ancestor and still bring down the ancestor feerate of the transaction. >>> >>> TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and >>> signals replaceability, regardless of the feerate you broadcast at, an >>> attacker can lower its mining priority by adding an ancestor. >>> >>> #### Absolute Fee >>> >>> The restriction of requiring replacement transactions to increase the >>> absolute fee of the mempool has been described as "bonkers." If the >>> original transaction has a very large descendant that pays a large >>> amount of fees, even if it has a low feerate, the replacement >>> transaction must now pay those fees in order to meet Rule #3. >>> >>> #### Package RBF >>> >>> There are a number of reasons why, in order to enable Package RBF, we >>> cannot use the same criteria. >>> >>> For starters, the absolute fee pinning attack is especially >>> problematic if we apply the same rules (i.e. Rule #3 and #4) in >>> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >>> LN channel. The mempool is rather full, so their pre-negotiated >>> commitment transactions' feerates would not be considered high >>> priority by miners. Bob broadcasts his commitment transaction and >>> attaches a very large child (100KvB with 100,000sat in fees) to his >>> anchor output. Alice broadcasts her commitment transaction with a >>> fee-bumping child (200vB with 50,000sat fees which is a generous >>> 250sat/vB), but this does not meet the absolute fee requirement. She >>> would need to add another 50,000sat to replace Bob's commitment >>> transaction. >>> >>> Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be >>> broken for packages containing transactions already in the mempool, >>> explained [here][7]. >>> >>> Note: I originally [proposed][6] Package RBF using the same Rule #3 >>> and #4 before I realized how significant this pinning attack is. I'm >>> retracting that proposal, and a new set of Package RBF rules would >>> follow from whatever the new individual RBF rules end up being. >>> >>> #### Same Txid Different Witness >>> >>> Two transactions with the same non-witness data but different >>> witnesses have the same txid but different wtxid, and the same fee but >>> not necessarily the same feerate. Currently, if we see a transaction >>> that has the same txid as one in the mempool, we reject it as a >>> duplicate, even if the feerate is much higher. It's unclear to me if >>> we have a very strong reason to change this, but noting it as a >>> limitation of our current replacement policy. See [#24007][12]. >>> >>> ### User Interface >>> >>> #### Using Unconfirmed UTXOs to Fund Replacements >>> >>> The restriction of only allowing confirmed UTXOs for funding a >>> fee-bump (Rule #2) can hurt users trying to fee-bump their >>> transactions and complicate wallet implementations. If the original >>> transaction's output value isn't sufficient to fund a fee-bump and/or >>> all of the user's other UTXOs are unconfirmed, they might not be able >>> to fund a replacement transaction. Wallet developers also need to >>> treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which >>> adds complexity to wallet logic. For example, see BDK issues [#144][4] >>> and [#414][5]. >>> >>> #### Interface Not Suitable for Coin Selection >>> >>> Currently, a user cannot simply create a replacement transaction >>> targeting a specific feerate or meeting a minimum fee amount and >>> expect to meet the RBF criteria. The fee amount depends on the size of >>> the replacement transaction, and feerate is almost irrelevant. >>> >>> Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the >>> replacement. It [estimates][13] a feerate which is "wallet incremental >>> relay fee" (a conservative overestimation of the node's incremental >>> relay fee) higher than the original transaction, selects coins for >>> that feerate, and hopes that it meets the RBF rules. It never fails >>> Rule #3 and #4 because it uses all original inputs and refuses to >>> bump a transaction with mempool descendants. >>> >>> This is suboptimal, but is designed to work with the coin selection >>> engine: select a feerate first, and then add fees to cover it. >>> Following the exact RBF rules would require working the other way >>> around: based on how much fees we've added to the transaction and its >>> current size, calculate the feerate to see if we meet Rule #4. >>> >>> While this isn't completely broken, and the user interface is >>> secondary to the safety of the mempool policy, we can do much better. >>> A much more user-friendly interface would depend *only* on the >>> fee and size of the original transactions. >>> >>> ### Updates to Mempool and Mining >>> >>> Since RBF was first implemented, a number of improvements have been >>> made to mempool and mining logic. For example, we now use ancestor >>> feerates in mining (allowing CPFP), and keep track of ancestor >>> packages in the mempool. >>> >>> ## Ideas for Improvements >>> >>> ### Goals >>> >>> To summarize, these seem to be desired changes, in order of priority: >>> >>> 1. Remove Rule #3. The replacement should not be *required* to pay >>> higher absolute fees. >>> >>> 2. Make it impossible for a replacement transaction to have a lower >>> mining score than the original transaction(s). This would eliminate >>> the `SIGHASH\_ANYONECANPAY` pinning attack. >>> >>> 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. >>> >>> 4. Create a more helpful interface that helps wallet fund replacement >>> transactions that aim for a feerate and fee. >>> >>> ### A Different Model for Fees >>> >>> For incentive compatibility, I believe there are different >>> formulations we should consider. Most importantly, if we want to get >>> rid of the absolute fee rule, we can no longer think of it as "the >>> transaction needs to pay for its own bandwidth," since we won't always >>> be getting additional fees. That means we need a new method of >>> rate-limiting replacements that doesn't require additional fees every >>> time. >>> >>> While it makes sense to think about monetary costs when launching a >>> specific type of attack, given that the fees are paid to the miner and >>> not to the mempool operators, maybe it doesn't make much sense to >>> think about "paying for bandwidth". Maybe we should implement >>> transaction validation rate-limiting differently, e.g. building it >>> into the P2P layer instead of the mempool policy layer. >>> >>> Recently, Suhas gave a [formulation][8] for incentive compatibility >>> that made sense to me: "are the fees expected to be paid in the next >>> (N?) blocks higher or lower if we process this transaction?" >>> >>> I started by thinking about this where N=1 or `1 + p`. >>> Here, a rational miner is looking at what fees they would >>> collect in the next block, and then some proportion `p` of the rest of >>> the blocks based on their hashrate. We're assuming `p` isn't *so high* >>> that they would be okay with lower absolute fees in the next 1 block. >>> We're also assuming `p` isn't *so low* that the miner doesn't care >>> about what's left of the mempool after this block. >>> >>> A tweak to this formulation is "if we process this transaction, would >>> the fees in the next 1 block higher or lower, and is the feerate >>> density of the rest of the mempool higher or lower?" This is pretty >>> similar, where N=1, but we consider the rest of the mempool by feerate >>> rather than fees. >>> >>> ### Mining Score of a Mempool Transaction >>> >>> We are often interested in finding out what >>> the "mining score" of a transaction in the mempool is. That is, when >>> the transaction is considered in block template building, what is the >>> feerate it is considered at? >>> >>> Obviously, it's not the transaction's individual feerate. Bitcoin Core >>> [mining code sorts][14] transactions by their ancestor feerate and >>> includes them packages at a time, keeping track of how this affects the >>> package feerates of remaining transactions in the mempool. >>> >>> *ancestor feerate*: Ancestor feerate is easily accessible information, >>> but it's not accurate either, because it doesn't take into account the >>> fact that subsets of a transaction's ancestor set can be included >>> without it. For example, ancestors may have high feerates on their own >>> or we may have [high feerate siblings][8]. >>> >>> TLDR: *Looking at the current ancestor feerate of a transaction is >>> insufficient to tell us what feerate it will be considered at when >>> building a block template in the future.* >>> >>> *min(individual feerate, ancestor feerate)*: Another >>> heuristic that is simple to calculate based on current mempool tooling >>> is to use the [minimum of a transaction's individual score and its >>> ancestor score][10] as a conservative measure. But this can >>> overestimate as well (see the example below). >>> >>> *min ancestor feerate(tx + possible ancestor subsets)* We can also >>> take the minimum of every possible ancestor subset, but this can be >>> computationally expensive since there can be lots and lots of ancestor >>> subsets. >>> >>> *max ancestor feerate(tx + possible descendant subsets)*: Another idea >>> is to use the [maximum ancestor score of the transaction + each of its >>> descendants][9]. This doesn't work either; it has the same blindspot >>> of ancestor subsets being mined on their own. >>> >>> #### Mining Score Example >>> >>> Here's an example illustrating why mining score is tricky to >>> efficiently calculate for mempool transactions: >>> >>> Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), >>> C(9sat/vB), D(5sat/vB). >>> The layout is: grandparent A, parent B, and two children C and D. >>> >>> ``` >>> A >>> ^ >>> B >>> ^ ^ >>> C D >>> ``` >>> >>> A miner using ancestor packages to build block templates will first >>> include A with a mining score of 21. Next, the miner will include B and >>> C with a mining score of 6. This leaves D, with a mining score of 5. >>> >>> Note: in this case, mining by ancestor feerate results in the most >>> rational decisions, but [a candidate set-based approach][10] which >>> makes ancestor feerate much less relevant could >>> be more advantageous in other situations. >>> >>> Here is a chart showing the "true" mining score alongside the values >>> calculating using imperfect heuristics described above. All of them >>> can overestimate or underestimate. >>> >>> ``` >>> A B C D >>> mining score | 21 | 6 | 6 | 5 | >>> ancestor feerate | 21 | 11 | 10.3 | 9 | >>> min(individual, ancestor) | 21 | 1 | 9 | 5 | >>> min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | >>> max(tx + descendants subsets) | 21 | 9 | 9 | 5 | >>> >>> ``` >>> >>> Possibly the best solution for finding the "mining score" of a >>> transaction is to build a block template, see what feerate each >>> package is included at. Perhaps at some cutoff, remaining mempool >>> transactions can be estimated using some heuristic that leans >>> {overestimating, underestimating} depending on the situation. >>> >>> Mining score seems to be relevant in multiple places: Murch and I >>> recently [found][3] that it would be very important in >>> "ancestor-aware" funding of transactions (the wallet doesn't >>> incorporate ancestor fees when using unconfirmed transactions in coin >>> selection, which is a bug we want to fix). >>> >>> In general, it would be nice to know the exact mining priority of >>> one's unconfirmed transaction is. I can think of a few block/mempool >>> explorers who might want to display this information for users. >>> >>> ### RBF Improvement Proposals >>> >>> After speaking to quite a few people, here are some suggestions >>> for improvements that I have heard: >>> >>> * The ancestor score of the replacement must be {5, 10, N}% higher >>> than that of every original transaction. >>> >>> * The ancestor score of the replacement must be 1sat/vB higher than >>> that of every original transaction. >>> >>> * If the original transaction is in the top {0.75MvB, 1MvB} of the >>> mempool, apply the current rules (absolute fees must increase and >>> pay for the replacement transaction's new bandwidth). Otherwise, use a >>> feerate-only rule. >>> >>> * If fees don't increase, the size of the replacement transaction must >>> decrease by at least N%. >>> >>> * Rate-limit how many replacements we allow per prevout. >>> >>> * Rate-limit transaction validation in general, per peer. >>> >>> Perhaps some others on the mailing list can chime in to throw other >>> ideas into the ring and/or combine some of these rules into a sensible >>> policy. >>> >>> #### Replace by Feerate Only >>> >>> I don't think there's going to be a single-line feerate-based >>> rule that can incorporate everything we need. >>> On one hand, a feerate-only approach helps eliminate the issues >>> associated with Rule #3. On the other hand, I believe the main concern >>> with a feerate-only approach is how to rate limit replacements. We >>> don't want to enable an attack such as: >>> >>> 1. Attacker broadcasts large, low-feerate transaction, and attaches a >>> chain of descendants. >>> >>> 2. The attacker replaces the transaction with a smaller but higher >>> feerate transaction, attaching a new chain of descendants. >>> >>> 3. Repeat 1000 times. >>> >>> #### Fees in Next Block and Feerate for the Rest of the Mempool >>> >>> Perhaps we can look at replacements like this: >>> >>> 1. Calculate the directly conflicting transactions and, with their >>> descendants, the original transactions. Check signaling. Limit the >>> total volume (e.g. can't be more than 100 total or 1MvB or something). >>> >>> 2. Find which original transactions would be in the next ~1 block. The >>> replacement must pay at least this amount + X% in absolute fees. This >>> guarantees that the fees of the next block doesn't decrease. >>> >>> 3. Find which transactions would be left in the mempool after that ~1 >>> block. The replacement's feerate must be Y% higher than the maximum >>> mining score of these transactions. This guarantees that you now have >>> only *better* candidates in your after-this-block mempool than you did >>> before, even if the size and fees the transactions decrease. >>> >>> 4. Now you have two numbers: a minimum absolute fee amount and a >>> minimum feerate. Check to see if the replacement(s) meet these >>> minimums. Also, a wallet would be able to ask the node "What fee and >>> feerate would I need to put on a transaction replacing this?" and use >>> this information to fund a replacement transaction, without needing to >>> guess or overshoot. >>> >>> Obviously, there are some magic numbers missing here. X and Y are >>> TBD constants to ensure we have some kind of rate limiting for the >>> number of replacements allowed using some set of fees. >>> >>> What should they be? We can do some arithmetic to see what happens if >>> you start with the biggest/lowest feerate transaction and do a bunch >>> of replacements. Maybe we end up with values that are high enough to >>> prevent abuse and make sense for applications/users that do RBF. >>> >>> ### Mempool Changes Need for Implementation >>> >>> As described in the mining score section above, >>> we may want additional tooling to more accurately assess >>> the economic gain of replacing transactions in our mempool. >>> >>> A few options have been discussed: >>> >>> * Calculate block templates on the fly when we need to consider a >>> replacement. However, since replacements are [quite common][11] >>> and the information might be useful for other things as well, >>> it may be worth it to cache a block template. >>> >>> * Keep a persistent block template so that we know what transactions >>> we would put in the next block. We need to remember the feerate >>> at which each transaction was included in the template, because an >>> ancestor package may be included in the same block template in >>> multiple subsets. Transactions included earlier alter the ancestor >>> feerate of the remaining transactions in the package. We also need >>> to keep track of the new feerates of transactions left over. >>> >>> * Divide the mempool into two layers, "high feerate" and "low >>> feerate." The high feerate layer contains ~1 block of packages with >>> the highest ancestor feerates, and the low feerate layer contains >>> everything else. At the edge of a block, we have a Knapsacky problem >>> where the next highest ancestor feerate package might not fit, so we >>> would probably want the high feerate layer ~2MvB or something to avoid >>> underestimating the fees. >>> >>> ## Acknowledgements >>> >>> Thank you to everyone whose RBF-related suggestions, grievances, >>> criticisms and ideas were incorporated in this document: >>> Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, >>> Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, >>> Antoine Poinsot, Antoine Riard, Larry Ruane, >>> S3RK and Bastien Teinturier. >>> >>> Thanks for reading! >>> >>> Best, >>> Gloria >>> >>> [1]: >>> https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md >>> [2]: >>> https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 >>> [3]: >>> https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea >>> [4]: https://github.com/bitcoindevkit/bdk/issues/144 >>> [5]: https://github.com/bitcoindevkit/bdk/issues/414 >>> [6]: >>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html >>> [7]: >>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 >>> [8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 >>> [9]: >>> https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 >>> [10]: >>> https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md >>> [11]: >>> https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 >>> [12]: https://github.com/bitcoin/bitcoin/pull/24007 >>> [13]: >>> https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 >>> [14]: >>> https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > > [-- Attachment #2: Type: text/html, Size: 48283 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-07 10:22 ` Bastien TEINTURIER @ 2022-02-07 11:16 ` Gloria Zhao 2022-02-08 4:58 ` Anthony Towns 0 siblings, 1 reply; 25+ messages in thread From: Gloria Zhao @ 2022-02-07 11:16 UTC (permalink / raw) To: Bastien TEINTURIER, Bitcoin Protocol Discussion; +Cc: Anthony Towns [-- Attachment #1: Type: text/plain, Size: 49589 bytes --] Hi everyone, Thanks for giving your attention to the post! I haven't had time to write responses to everything, but sending my thoughts about what has been most noteworthy to me: @jeremy: > A final point is that a verifiable delay function could be used over, e.g., each of the N COutpoints individually to rate-limit transaction replacement. The VDF period can be made shorter / eliminated depending on the feerate increase. Thanks for the suggestion! In general, I don't think rate limiting by outpoint/prevout is a safe option, as it is particularly dangerous for L2 applications with shared prevouts. For example, the prevout that LN channel counterparties conflict on is the output from their shared funding tx. Any kind of limit on spending this prevout can be monopolized by a spammy attacker. For example, if you only allow 1 per minute, the attacker will just race to take up that slot every minute to prevent the honest party's transaction from being accepted. This is similar to the pinning attack based on monopolizing the transaction's descendant limit, except we can't carve out an exemption because we wouldn't know whose replacement we're looking at. @tbast: > The way I understand it, limiting the impact on descendant transactions is only important for DoS protection, not for incentive compatibility. > I believe it's completely ok to require increasing both the fees and feerate if we don't take descendants into account, because you control your ancestor set - whereas the descendant set may be completely out of your control. Ignoring descendants of direct conflicts would certainly make our lives much easier! Unfortunately, I don't think we can do this since they can be fee bumps, i.e., in AJ's example. Considering descendants is important for both incentive compatibility and DoS. If the replacement transaction has a higher feerate than its direct conflict, but the direct conflict also has high feerate descendants, we might end up with lower fees and/or feerates by accepting the replacement. @aj: > I wonder sometimes if it could be sufficient to just have a relay rate limit and prioritise by ancestor feerate though. Maybe something like: > > - instead of adding txs to each peers setInventoryTxToSend immediately, > set a mempool flag "relayed=false" > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to > each peer's setInventoryTxToSend and mark them as "relayed=true"; > calculate how much kB those txs were, and do this again after > SIZE/RATELIMIT seconds > > - don't include "relayed=false" txs when building blocks? Wow cool! I think outbound tx relay size-based rate-limiting and prioritizing tx relay by feerate are great ideas for preventing spammers from wasting bandwidth network-wide. I agree, this would slow the low feerate spam down, preventing a huge network-wide bandwidth spike. And it would allow high feerate transactions to propagate as they should, regardless of how busy traffic is. Combined with inbound tx request rate-limiting, might this be sufficient to prevent DoS regardless of the fee-based replacement policies? One point that I'm not 100% clear on: is it ok to prioritize the transactions by ancestor feerate in this scheme? As I described in the original post, this can be quite different from the actual feerate we would consider a transaction in a block for. The transaction could have a high feerate sibling bumping its ancestor. For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). If we just received C, it would be incorrect to give it a priority equal to its ancestor feerate (3sat/vB) because if we constructed a block template now, B would bump A, and C's new ancestor feerate is 5sat/vB. Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we also exclude C when building blocks, we're missing out on good fees. > - keep high-feerate evicted txs around for a while in case they get > mined by someone else to improve compact block relay, a la the > orphan pool? Replaced transactions are already added to vExtraTxnForCompact :D @ariard > Deployment of Taproot opens interesting possibilities in the vaults/payment channels design space, where the tapscripts can commit to different set of timelocks/quorum of keys. Even if the pre-signed states stay symmetric, whoever is the publisher, the feerate cost to spend can fluctuate. Indeed, perhaps with taproot we may legitimately have same-txid-different-witness transactions as a normal thing rather than rare edge case. But as with everything enabled by taproot, I wouldn't count our tapscript eggs until a concrete use case hatches and/or an application actually implements it. > How this new replacement rule would behave if you have a parent in the "replace-by-feerate" half but the child is in the "replace-by-fee" one ? Thanks for considering my suggestion! This particular scenario is not possible, since a child cannot be considered for the next block without its parent. But if the original transactions are found both in and outside the next block, I think it would be fine to just require both are met. > Overall, I think there is the deployment issue to warn of. Moving to a new set of RBF rules implies for a lot of Bitcoin applications to rewrite their RBF logics. I agree that transitioning as painlessly as possible would be a huge priority in any kind of upgrade to mempool policy. I'm very interested in hearing wallet devs' feedback on this. I'm also not actually clear on what backwards compatibility in this scenario would look like. I imagine it to mean we run both sets of RBF rules and accept the replacement if it passes either one. Or do we only accept the replacement if it passes both? For wallets, AJ's "All you need is for there to be *a* path that follows the new relay rules and gets from your node/wallet to perhaps 10% of hashpower" makes sense to me (which would be the former). For merchants who care more about making sure the original transaction isn't replaceable, would they prefer that either policy is sufficient to prevent a replacement (more in line with the latter)? Or is that covered by signaling / am I overthinking this? Thanks, Gloria On Mon, Feb 7, 2022 at 10:24 AM Bastien TEINTURIER via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Good morning, > > > The tricky question is what happens when X arrives on its own and it > > might be that no one ever sends a replacement for B,C,D) > > It feels ok to me, but this is definitely arguable. > > It covers the fact that B,C,D could have been fake transactions whose > sole purpose was to do a pinning attack: in that case the attacker would > have found a way to ensure these transactions don't confirm anyway (or > pay minimal/negligible fees). > > If these transactions were legitimate, I believe that their owners would > remake them at some point (because these transactions reflect a business > relationship that needed to happen, so it should very likely still > happen). It's probably hard to verify because the new corresponding > transactions may have nothing in common with the first, but I think the > simplifications it offers for wallets is worth it (which is just my > opinion and needs more scrutiny/feedback). > > > But if your backlog's feerate does drop off, *and* that matters, then > > I don't think you can ignore the impact of the descendent transactions > > that you might not get a replacement for. > > That is because you're only taking into account the current backlog, and > not taking into account the fact that new items will be added to it soon > to replace the evicted descendants. But I agree that this is a bet: we > can't predict the future and guarantee these replacements will come. > > It is really a trade-off, ignoring descendents provides a much simpler > contract that doesn't vary from one mempool to another, but when your > backlog isn't full enough, you may lose some future profits if > transactions don't come in later. > > > I think "Y% higher" rather than just "higher" is only useful for > > rate-limiting, not incentive compatibility. (Though maybe it helps > > stabilise a greedy algorithm in some cases?) > > That's true. I claimed these policies only address incentives, but using > a percentage increase addresses rate-limiting a bit as well (I couldn't > resist trying to do at least something for it!). I find it a very easy > mechanism to implement, while choosing an absolute value is hard (it's > always easier to think in relatives than absolutes). > > > This is why I think it is important to understand the rationales for > introducing the rules in the first place > > I completely agree. As you mentioned, we are still in brainstorming > phase, once (if?) we start to converge on what could be better policies, > we do need to clearly explain each policy's expected goal. That will let > future Bastien writing code in 2030 clearly highlight why the 2022 rules > don't make sense anymore! > > Cheers, > Bastien > > Le sam. 5 févr. 2022 à 14:22, Michael Folkson < > michaelfolkson@protonmail.com> a écrit : > >> Thanks for this Bastien (and Gloria for initially posting about this). >> >> I sympathetically skimmed the eclair PR ( >> https://github.com/ACINQ/eclair/pull/2113) dealing with replaceable >> transactions fee bumping. >> >> There will continue to be a (hopefully) friendly tug of war on this >> probably for the rest of Bitcoin's existence. I am sure people like Luke, >> Prayank etc will (rightfully) continue to raise that Lightning and other >> second layer protocols shouldn't demand that policy rules be changed if >> there is a reason (e.g. DoS vector) for those rules on the base network. >> But if there are rules that have no upside, introduce unnecessary >> complexity for no reason and make Lightning implementers like Bastien's >> life miserable attempting to deal with them I really hope we can make >> progress on removing or simplifying them. >> >> This is why I think it is important to understand the rationales for >> introducing the rules in the first place (and why it is safe to remove them >> if indeed it is) and being as rigorous as possible on the rationales for >> introducing additional rules. It sounds like from Gloria's initial post we >> are still at a brainstorming phase (which is fine) but knowing what we know >> today I really hope we can learn from the mistakes of the original BIP 125, >> namely the Core implementation not matching the BIP and the sparse >> rationales for the rules. As Bastien says this is not criticizing the >> original BIP 125 authors, 7 years is a long time especially in Bitcoin >> world and they probably weren't thinking about Bastien sitting down to >> write an eclair PR in late 2021 (and reviewers of that PR) when they wrote >> the BIP in 2015. >> >> -- >> Michael Folkson >> Email: michaelfolkson at protonmail.com >> Keybase: michaelfolkson >> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 >> >> >> >> ------- Original Message ------- >> On Monday, January 31st, 2022 at 3:57 PM, Bastien TEINTURIER via >> bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: >> >> Hi Gloria, >> >> Many thanks for raising awareness on these issues and constantly pushing >> towards finding a better model. This work will highly improve the >> security of any multi-party contract trying to build on top of bitcoin >> (because most multi-party contracts will need to have timeout conditions >> and participants will need to make some transactions confirm before a >> timeout happens - otherwise they may lose funds). >> >> For starters, let me quickly explain why the current rules are hard to >> work with in the context of lightning (but I believe most L2 protocols >> will have the same issues). Feel free to skip this part if you are >> already convinced. >> >> ## Motivation >> >> The biggest pain point is BIP 125 rule 2. >> If I need to increase the fees of a time-sensitive transaction because >> the feerate has been rising since I broadcast it, I may need to also pay >> high fees just to produce a confirmed utxo that I can use. I'm actually >> paying a high fee twice instead of once (and needlessly using on-chain >> space, our scarcest asset, because we could have avoided that additional >> transaction!). >> >> It also has some annoying "non-determinism". >> Imagine that my transaction has been evicted from my mempool because its >> feerate was too low. I could think "Great, that means I don't have to >> apply BIP 125 restrictions, I can just fund this transaction as if it >> were a new one!". But actually I do, because my transaction could still >> be in miner's mempools and I have no way of knowing it...this means that >> whenever I have broadcast a transaction, I must assume that I will >> always need to abide by whatever replacement rules the network applies. >> >> Fortunately, as far as I understand it, this rule only exists because of >> a previous implementation detail of bitcoin core, so there's simply no >> good reason to keep it. >> >> The second biggest pain point is rule 3. It prevents me from efficiently >> using my capital while it's unconfirmed. Whenever I'm using a big utxo >> to fund a transaction, I will get a big change output, and it would >> really be a waste to be unable to use that change output to fund other >> transactions. In order to be capital-efficient, I will end up creating >> descendant trees for my time-sensitive transactions. But as Gloria >> explained, replacing all my children will cost me an absurdly large >> amount of fees. So what I'm actually planning to do instead is to RBF >> one of the descendants high enough to get the whole tree confirmed. >> But if those descendants' timeouts were far in the future, that's a >> waste, I paid a lot more fees for them than I should have. I'd like to >> just replace my transaction and republish the invalidated children >> independently. >> >> Rule 4 doesn't hurt as much as the two previous ones, I don't have too >> much to say about it. >> >> To be fair to the BIP 125 authors, all of these scenarios were very hard >> to forecast at the time this BIP was created. We needed years to build >> on those rules to get a better understanding of their limitations and if >> the rationale behind them made sense in the long term. >> >> ## Proposals >> >> I believe that now is a good time to re-think those, and I really like >> Gloria's categorization of the design constraints. >> >> I'd like to propose a different way of looking at descendants that makes >> it easier to design the new rules. The way I understand it, limiting the >> impact on descendant transactions is only important for DoS protection, >> not for incentive compatibility. I would argue that after evictions, >> descendant transactions will be submitted again (because they represent >> transactions that people actually want to make), so evicting them does >> not have a negative impact on mining incentives (in a world where blocks >> are full most of the time). >> >> I'm curious to hear other people's thoughts on that. If it makes sense, >> I would propose the following very simple rules: >> >> 1. The transaction's ancestor absolute fees must be X% higher than the >> previous transaction's ancestor fees >> 2. The transaction's ancestor feerate must be Y% higher than the >> previous transaction's ancestor feerate >> >> I believe it's completely ok to require increasing both the fees and >> feerate if we don't take descendants into account, because you control >> your ancestor set - whereas the descendant set may be completely out of >> your control. >> >> This is very easy to use by wallets, because the ancestor set is easy to >> obtain. And an important point is that the ancestor set is the same in >> every mempool, whereas the descendant set is not (your mempool may have >> rejected the last descendants, while other people's mempools may still >> contain them). >> >> Because of that reason, I'd like to avoid having a rule that relies on >> some size of the replaced descendant set: it may be valid in your >> mempool but invalid in someone else's, which makes it exploitable for >> pinning attacks. >> >> I believe these rules are incentive compatible (again, if you accept >> the fact that the descendants will be re-submitted and mined as well, >> so their fees aren't lost). >> >> Can we choose X and Y so that these two rules are also DoS-resistant? >> Unfortunately I'm not sure, so maybe we'll need to add a third rule to >> address that. But before we do, can someone detail what it costs for a >> node to evict a descendant tree? Given that bitcoin core doesn't allow >> chains of more than 25 transactions, the maximum number of transactions >> being replaced will be bounded by 25 * N (where N is the number of >> outputs of the transaction being replaced). If it's just O(n) pruning of >> a graph, maybe that's ok? Or maybe we make X or Y depend on the number >> of outputs of the transaction being replaced (this would need very >> careful thoughts)? >> >> If you made it this far, thanks for reading! >> A couple of comments on the previous messages: >> >> > Currently, if we see a transaction >> > that has the same txid as one in the mempool, we reject it as a >> > duplicate, even if the feerate is much higher. It's unclear to me if >> > we have a very strong reason to change this, but noting it as a >> > limitation of our current replacement policy. >> >> I don't see a strong reason from an L2 protocol's point of view yet, but >> there are many unkown unknowns. But from a miner incentive's point of >> view, we should keep the transaction with the higher feerate, shouldn't >> we? In that case it's also a more efficient use of on-chain space, which >> is a win, right? >> >> > We might have a more-or-less long transition period during which we >> support both... >> >> Yes, this is a long term thing. >> Even if bitcoin core releases a new version with updated RBF rules, as a >> wallet you'll need to keep using the old rules for a long time if you >> want to be safe. >> >> But it's all the more reason to try to ship this as soon as possible, >> this way maybe our grand-children will be able to benefit from it ;) >> (just kidding on the timespan obviously). >> >> Cheers, >> Bastien >> >> Le lun. 31 janv. 2022 à 00:11, Antoine Riard via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> a écrit : >> >>> Hi Gloria, >>> >>> Thanks for this RBF sum up. Few thoughts and more context comments if it >>> can help other readers. >>> >>> > For starters, the absolute fee pinning attack is especially >>> > problematic if we apply the same rules (i.e. Rule #3 and #4) in >>> > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >>> > LN channel. The mempool is rather full, so their pre-negotiated >>> > commitment transactions' feerates would not be considered high >>> > priority by miners. Bob broadcasts his commitment transaction and >>> > attaches a very large child (100KvB with 100,000sat in fees) to his >>> > anchor output. Alice broadcasts her commitment transaction with a >>> > fee-bumping child (200vB with 50,000sat fees which is a generous >>> > 250sat/vB), but this does not meet the absolute fee requirement. She >>> > would need to add another 50,000sat to replace Bob's commitment >>> > transaction. >>> >>> Solving LN pinning attacks, what we're aiming for is enabling a fair >>> feerate bid between the counterparties, thus either forcing the adversary >>> to overbid or to disengage from the confirmation competition. If the >>> replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob >>> to >>> pick up the first option. Though if he does, that's a winning outcome >>> for Alice, as one of the commitment transactions confirms and her >>> time-sensitive second-stage HTLC can be subsequently confirmed. >>> >>> > It's unclear to me if >>> > we have a very strong reason to change this, but noting it as a >>> > limitation of our current replacement policy. See [#24007][12]. >>> >>> Deployment of Taproot opens interesting possibilities in the >>> vaults/payment channels design space, where the tapscripts can commit to >>> different set of timelocks/quorum of keys. Even if the pre-signed states >>> stay symmetric, whoever is the publisher, the feerate cost to spend can >>> fluctuate. >>> >>> > While this isn't completely broken, and the user interface is >>> > secondary to the safety of the mempool policy >>> >>> I think with L2s transaction broadcast backend, the stability and >>> clarity of the RBF user interface is primary. What we could be worried >>> about is a too-much complex interface easing the way for an attacker to >>> trigger your L2 node to issue policy-invalid chain of transactions. >>> Especially, when we consider that an attacker might have leverage on chain >>> of transactions composition ("force broadcast of commitment A then >>> commitment B, knowing they will share a CPFP") or even transactions size >>> ("overload commitment A with HTLCs"). >>> >>> > * If the original transaction is in the top {0.75MvB, 1MvB} of the >>> > mempool, apply the current rules (absolute fees must increase and >>> > pay for the replacement transaction's new bandwidth). Otherwise, use a >>> > feerate-only rule. >>> >>> How this new replacement rule would behave if you have a parent in the >>> "replace-by-feerate" half but the child is in the "replace-by-fee" one ? >>> >>> If we allow the replacement of the parent based on the feerate, we might >>> decrease the top block absolute fees. >>> >>> If we block the replacement of the parent based on the feerate because >>> the replacement absolute fees aren't above the replaced package, we still >>> preclude a pinning vector. The child might be low-feerate junk and even >>> attached to a low ancestor-score branch. >>> >>> If I'm correct on this limitation, maybe we could turn off the >>> "replace-by-fee" behavior as soon as the mempool is fulfilled with a few >>> blocks ? >>> >>> > * Rate-limit how many replacements we allow per prevout. >>> >>> Depending on how it is implemented, though I would be concerned it >>> introduces a new pinning vector in the context of shared-utxo. If it's a >>> hardcoded constant, it could be exhausted by an adversary starting at the >>> lowest acceptable feerate then slowly increasing while still not reaching >>> the top of the mempool. Same if it's time-based or block-based, no >>> guarantee the replacement slot is honestly used by your counterparty. >>> >>> Further, an above-the-average replacement frequency might just be the >>> reflection of your confirmation strategy reacting to block schedule or >>> mempools historical data. As long as the feerate penalty is paid, I lean to >>> allow replacement. >>> >>> (One solution could be to associate per-user "tag" to the LN >>> transactions, where each "tag" would have its own replacement slots, but >>> privacy?) >>> >>> > * Rate-limit transaction validation in general, per peer. >>> >>> I think we could improve on the Core's new transaction requester logic. >>> Maybe we could bind the peer announced flow based on the feerate score >>> (modulo validation time) of the previously validated transactions from that >>> peer ? That said, while related to RBF, it sounds to me that enhancing >>> Core's rate-limiting transaction strategy is a whole discussion in itself >>> [0]. Especially ensuring it's tolerant to the specific requirements of LN & >>> consorts. >>> >>> > What should they be? We can do some arithmetic to see what happens if >>> > you start with the biggest/lowest feerate transaction and do a bunch >>> > of replacements. Maybe we end up with values that are high enough to >>> > prevent abuse and make sense for applications/users that do RBF. >>> >>> That's a good question. >>> >>> One observation is that the attacker can always renew the set of DoSy >>> utxos to pursue the attack. So maybe we could pick up constants scaled on >>> the block size ? That way an attacker would have to burn fees, thus >>> deterring them from launching an attack. Even if the attackers are miners, >>> they have to renounce their income to acquire new DoSy utxos. If a low-fee >>> period, we could scale up the constants ? >>> >>> >>> Overall, I think there is the deployment issue to warn of. Moving to a >>> new set of RBF rules implies for a lot of Bitcoin applications to rewrite >>> their RBF logics. We might have a more-or-less long transition period >>> during which we support both... >>> >>> Cheers, >>> Antoine >>> >>> [0] https://github.com/bitcoin/bitcoin/pull/21224 >>> >>> Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev < >>> bitcoin-dev@lists.linuxfoundation.org> a écrit : >>> >>>> Hi everyone, >>>> >>>> This post discusses limitations of current Bitcoin Core RBF policy and >>>> attempts to start a conversation about how we can improve it, >>>> summarizing some ideas that have been discussed. Please reply if you >>>> have any new input on issues to be solved and ideas for improvement! >>>> >>>> Just in case I've screwed up the text wrapping again, another copy can >>>> be >>>> found here: >>>> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff >>>> >>>> ## Background >>>> >>>> Please feel free to skip this section if you are already familiar >>>> with RBF. >>>> >>>> Nodes may receive *conflicting* unconfirmed transactions, aka >>>> "double spends" of the same inputs. Instead of always keeping the >>>> first transaction, since v0.12, Bitcoin Core mempool policy has >>>> included a set of Replace-by-Fee (RBF) criteria that allows the second >>>> transaction to replace the first one and any descendants it may have. >>>> >>>> Bitcoin Core RBF policy was previously documented as BIP 125. >>>> The current RBF policy is documented [here][1]. In summary: >>>> >>>> 1. The directly conflicting transactions all signal replaceability >>>> explicitly. >>>> >>>> 2. The replacement transaction only includes an unconfirmed input if >>>> that input was included in one of the directly conflicting >>>> transactions. >>>> >>>> 3. The replacement transaction pays an absolute fee of at least the >>>> sum paid by the original transactions. >>>> >>>> 4. The additional fees pays for the replacement transaction's >>>> bandwidth at or above the rate set by the node's *incremental relay >>>> feerate*. >>>> >>>> 5. The sum of all directly conflicting transactions' descendant counts >>>> (number of transactions inclusive of itself and its descendants) >>>> does not exceed 100. >>>> >>>> We can split these rules into 3 categories/goals: >>>> >>>> - **Allow Opting Out**: Some applications/businesses are unable to >>>> handle transactions that are replaceable (e.g. merchants that use >>>> zero-confirmation transactions). We (try to) help these businesses by >>>> honoring BIP125 signaling; we won't replace transactions that have not >>>> opted in. >>>> >>>> - **Incentive Compatibility**: Ensure that our RBF policy would not >>>> accept replacement transactions which would decrease fee profits >>>> of a miner. In general, if our mempool policy deviates from what is >>>> economically rational, it's likely that the transactions in our >>>> mempool will not match the ones in miners' mempools, making our >>>> fee estimation, compact block relay, and other mempool-dependent >>>> functions unreliable. Incentive-incompatible policy may also >>>> encourage transaction submission through routes other than the p2p >>>> network, harming censorship-resistance and privacy of Bitcoin payments. >>>> >>>> - **DoS Protection**: Limit two types of DoS attacks on the node's >>>> mempool: (1) the number of times a transaction can be replaced and >>>> (2) the volume of transactions that can be evicted during a >>>> replacement. >>>> >>>> Even more abstract: our goal is to make a replacement policy that >>>> results in a useful interface for users and safe policy for >>>> node operators. >>>> >>>> ## Motivation >>>> >>>> There are a number of known problems with the current RBF policy. >>>> Many of these shortcomings exist due to mempool limitations at the >>>> time RBF was implemented or result from new types of Bitcoin usage; >>>> they are not criticisms of the original design. >>>> >>>> ### Pinning Attacks >>>> >>>> The most pressing concern is that attackers may take advantage of >>>> limitations in RBF policy to prevent other users' transactions from >>>> being mined or getting accepted as a replacement. >>>> >>>> #### SIGHASH_ANYONECANPAY Pinning >>>> >>>> BIP125#2 can be bypassed by creating intermediary transactions to be >>>> replaced together. Anyone can simply split a 1-input 1-output >>>> transaction off from the replacement transaction, then broadcast the >>>> transaction as is. This can always be done, and quite cheaply. More >>>> details in [this comment][2]. >>>> >>>> In general, if a transaction is signed with SIGHASH\_ANYONECANPAY, >>>> anybody can just attach a low feerate parent to this transaction and >>>> lower its ancestor feerate. Even if you require SIGHASH\_ALL which >>>> prevents an attacker from changing any outputs, the input can be a >>>> very low amount (e.g. just above the dust limit) from a low-fee >>>> ancestor and still bring down the ancestor feerate of the transaction. >>>> >>>> TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and >>>> signals replaceability, regardless of the feerate you broadcast at, an >>>> attacker can lower its mining priority by adding an ancestor. >>>> >>>> #### Absolute Fee >>>> >>>> The restriction of requiring replacement transactions to increase the >>>> absolute fee of the mempool has been described as "bonkers." If the >>>> original transaction has a very large descendant that pays a large >>>> amount of fees, even if it has a low feerate, the replacement >>>> transaction must now pay those fees in order to meet Rule #3. >>>> >>>> #### Package RBF >>>> >>>> There are a number of reasons why, in order to enable Package RBF, we >>>> cannot use the same criteria. >>>> >>>> For starters, the absolute fee pinning attack is especially >>>> problematic if we apply the same rules (i.e. Rule #3 and #4) in >>>> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a >>>> LN channel. The mempool is rather full, so their pre-negotiated >>>> commitment transactions' feerates would not be considered high >>>> priority by miners. Bob broadcasts his commitment transaction and >>>> attaches a very large child (100KvB with 100,000sat in fees) to his >>>> anchor output. Alice broadcasts her commitment transaction with a >>>> fee-bumping child (200vB with 50,000sat fees which is a generous >>>> 250sat/vB), but this does not meet the absolute fee requirement. She >>>> would need to add another 50,000sat to replace Bob's commitment >>>> transaction. >>>> >>>> Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be >>>> broken for packages containing transactions already in the mempool, >>>> explained [here][7]. >>>> >>>> Note: I originally [proposed][6] Package RBF using the same Rule #3 >>>> and #4 before I realized how significant this pinning attack is. I'm >>>> retracting that proposal, and a new set of Package RBF rules would >>>> follow from whatever the new individual RBF rules end up being. >>>> >>>> #### Same Txid Different Witness >>>> >>>> Two transactions with the same non-witness data but different >>>> witnesses have the same txid but different wtxid, and the same fee but >>>> not necessarily the same feerate. Currently, if we see a transaction >>>> that has the same txid as one in the mempool, we reject it as a >>>> duplicate, even if the feerate is much higher. It's unclear to me if >>>> we have a very strong reason to change this, but noting it as a >>>> limitation of our current replacement policy. See [#24007][12]. >>>> >>>> ### User Interface >>>> >>>> #### Using Unconfirmed UTXOs to Fund Replacements >>>> >>>> The restriction of only allowing confirmed UTXOs for funding a >>>> fee-bump (Rule #2) can hurt users trying to fee-bump their >>>> transactions and complicate wallet implementations. If the original >>>> transaction's output value isn't sufficient to fund a fee-bump and/or >>>> all of the user's other UTXOs are unconfirmed, they might not be able >>>> to fund a replacement transaction. Wallet developers also need to >>>> treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which >>>> adds complexity to wallet logic. For example, see BDK issues [#144][4] >>>> and [#414][5]. >>>> >>>> #### Interface Not Suitable for Coin Selection >>>> >>>> Currently, a user cannot simply create a replacement transaction >>>> targeting a specific feerate or meeting a minimum fee amount and >>>> expect to meet the RBF criteria. The fee amount depends on the size of >>>> the replacement transaction, and feerate is almost irrelevant. >>>> >>>> Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the >>>> replacement. It [estimates][13] a feerate which is "wallet incremental >>>> relay fee" (a conservative overestimation of the node's incremental >>>> relay fee) higher than the original transaction, selects coins for >>>> that feerate, and hopes that it meets the RBF rules. It never fails >>>> Rule #3 and #4 because it uses all original inputs and refuses to >>>> bump a transaction with mempool descendants. >>>> >>>> This is suboptimal, but is designed to work with the coin selection >>>> engine: select a feerate first, and then add fees to cover it. >>>> Following the exact RBF rules would require working the other way >>>> around: based on how much fees we've added to the transaction and its >>>> current size, calculate the feerate to see if we meet Rule #4. >>>> >>>> While this isn't completely broken, and the user interface is >>>> secondary to the safety of the mempool policy, we can do much better. >>>> A much more user-friendly interface would depend *only* on the >>>> fee and size of the original transactions. >>>> >>>> ### Updates to Mempool and Mining >>>> >>>> Since RBF was first implemented, a number of improvements have been >>>> made to mempool and mining logic. For example, we now use ancestor >>>> feerates in mining (allowing CPFP), and keep track of ancestor >>>> packages in the mempool. >>>> >>>> ## Ideas for Improvements >>>> >>>> ### Goals >>>> >>>> To summarize, these seem to be desired changes, in order of priority: >>>> >>>> 1. Remove Rule #3. The replacement should not be *required* to pay >>>> higher absolute fees. >>>> >>>> 2. Make it impossible for a replacement transaction to have a lower >>>> mining score than the original transaction(s). This would eliminate >>>> the `SIGHASH\_ANYONECANPAY` pinning attack. >>>> >>>> 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed. >>>> >>>> 4. Create a more helpful interface that helps wallet fund replacement >>>> transactions that aim for a feerate and fee. >>>> >>>> ### A Different Model for Fees >>>> >>>> For incentive compatibility, I believe there are different >>>> formulations we should consider. Most importantly, if we want to get >>>> rid of the absolute fee rule, we can no longer think of it as "the >>>> transaction needs to pay for its own bandwidth," since we won't always >>>> be getting additional fees. That means we need a new method of >>>> rate-limiting replacements that doesn't require additional fees every >>>> time. >>>> >>>> While it makes sense to think about monetary costs when launching a >>>> specific type of attack, given that the fees are paid to the miner and >>>> not to the mempool operators, maybe it doesn't make much sense to >>>> think about "paying for bandwidth". Maybe we should implement >>>> transaction validation rate-limiting differently, e.g. building it >>>> into the P2P layer instead of the mempool policy layer. >>>> >>>> Recently, Suhas gave a [formulation][8] for incentive compatibility >>>> that made sense to me: "are the fees expected to be paid in the next >>>> (N?) blocks higher or lower if we process this transaction?" >>>> >>>> I started by thinking about this where N=1 or `1 + p`. >>>> Here, a rational miner is looking at what fees they would >>>> collect in the next block, and then some proportion `p` of the rest of >>>> the blocks based on their hashrate. We're assuming `p` isn't *so high* >>>> that they would be okay with lower absolute fees in the next 1 block. >>>> We're also assuming `p` isn't *so low* that the miner doesn't care >>>> about what's left of the mempool after this block. >>>> >>>> A tweak to this formulation is "if we process this transaction, would >>>> the fees in the next 1 block higher or lower, and is the feerate >>>> density of the rest of the mempool higher or lower?" This is pretty >>>> similar, where N=1, but we consider the rest of the mempool by feerate >>>> rather than fees. >>>> >>>> ### Mining Score of a Mempool Transaction >>>> >>>> We are often interested in finding out what >>>> the "mining score" of a transaction in the mempool is. That is, when >>>> the transaction is considered in block template building, what is the >>>> feerate it is considered at? >>>> >>>> Obviously, it's not the transaction's individual feerate. Bitcoin Core >>>> [mining code sorts][14] transactions by their ancestor feerate and >>>> includes them packages at a time, keeping track of how this affects the >>>> package feerates of remaining transactions in the mempool. >>>> >>>> *ancestor feerate*: Ancestor feerate is easily accessible information, >>>> but it's not accurate either, because it doesn't take into account the >>>> fact that subsets of a transaction's ancestor set can be included >>>> without it. For example, ancestors may have high feerates on their own >>>> or we may have [high feerate siblings][8]. >>>> >>>> TLDR: *Looking at the current ancestor feerate of a transaction is >>>> insufficient to tell us what feerate it will be considered at when >>>> building a block template in the future.* >>>> >>>> *min(individual feerate, ancestor feerate)*: Another >>>> heuristic that is simple to calculate based on current mempool tooling >>>> is to use the [minimum of a transaction's individual score and its >>>> ancestor score][10] as a conservative measure. But this can >>>> overestimate as well (see the example below). >>>> >>>> *min ancestor feerate(tx + possible ancestor subsets)* We can also >>>> take the minimum of every possible ancestor subset, but this can be >>>> computationally expensive since there can be lots and lots of ancestor >>>> subsets. >>>> >>>> *max ancestor feerate(tx + possible descendant subsets)*: Another idea >>>> is to use the [maximum ancestor score of the transaction + each of its >>>> descendants][9]. This doesn't work either; it has the same blindspot >>>> of ancestor subsets being mined on their own. >>>> >>>> #### Mining Score Example >>>> >>>> Here's an example illustrating why mining score is tricky to >>>> efficiently calculate for mempool transactions: >>>> >>>> Let's say you have same-size transactions A (21sat/vB), B (1sat/vB), >>>> C(9sat/vB), D(5sat/vB). >>>> The layout is: grandparent A, parent B, and two children C and D. >>>> >>>> ``` >>>> A >>>> ^ >>>> B >>>> ^ ^ >>>> C D >>>> ``` >>>> >>>> A miner using ancestor packages to build block templates will first >>>> include A with a mining score of 21. Next, the miner will include B and >>>> C with a mining score of 6. This leaves D, with a mining score of 5. >>>> >>>> Note: in this case, mining by ancestor feerate results in the most >>>> rational decisions, but [a candidate set-based approach][10] which >>>> makes ancestor feerate much less relevant could >>>> be more advantageous in other situations. >>>> >>>> Here is a chart showing the "true" mining score alongside the values >>>> calculating using imperfect heuristics described above. All of them >>>> can overestimate or underestimate. >>>> >>>> ``` >>>> A B C D >>>> mining score | 21 | 6 | 6 | 5 | >>>> ancestor feerate | 21 | 11 | 10.3 | 9 | >>>> min(individual, ancestor) | 21 | 1 | 9 | 5 | >>>> min(tx + ancestor subsets) | 21 | 1 | 5 | 3 | >>>> max(tx + descendants subsets) | 21 | 9 | 9 | 5 | >>>> >>>> ``` >>>> >>>> Possibly the best solution for finding the "mining score" of a >>>> transaction is to build a block template, see what feerate each >>>> package is included at. Perhaps at some cutoff, remaining mempool >>>> transactions can be estimated using some heuristic that leans >>>> {overestimating, underestimating} depending on the situation. >>>> >>>> Mining score seems to be relevant in multiple places: Murch and I >>>> recently [found][3] that it would be very important in >>>> "ancestor-aware" funding of transactions (the wallet doesn't >>>> incorporate ancestor fees when using unconfirmed transactions in coin >>>> selection, which is a bug we want to fix). >>>> >>>> In general, it would be nice to know the exact mining priority of >>>> one's unconfirmed transaction is. I can think of a few block/mempool >>>> explorers who might want to display this information for users. >>>> >>>> ### RBF Improvement Proposals >>>> >>>> After speaking to quite a few people, here are some suggestions >>>> for improvements that I have heard: >>>> >>>> * The ancestor score of the replacement must be {5, 10, N}% higher >>>> than that of every original transaction. >>>> >>>> * The ancestor score of the replacement must be 1sat/vB higher than >>>> that of every original transaction. >>>> >>>> * If the original transaction is in the top {0.75MvB, 1MvB} of the >>>> mempool, apply the current rules (absolute fees must increase and >>>> pay for the replacement transaction's new bandwidth). Otherwise, use a >>>> feerate-only rule. >>>> >>>> * If fees don't increase, the size of the replacement transaction must >>>> decrease by at least N%. >>>> >>>> * Rate-limit how many replacements we allow per prevout. >>>> >>>> * Rate-limit transaction validation in general, per peer. >>>> >>>> Perhaps some others on the mailing list can chime in to throw other >>>> ideas into the ring and/or combine some of these rules into a sensible >>>> policy. >>>> >>>> #### Replace by Feerate Only >>>> >>>> I don't think there's going to be a single-line feerate-based >>>> rule that can incorporate everything we need. >>>> On one hand, a feerate-only approach helps eliminate the issues >>>> associated with Rule #3. On the other hand, I believe the main concern >>>> with a feerate-only approach is how to rate limit replacements. We >>>> don't want to enable an attack such as: >>>> >>>> 1. Attacker broadcasts large, low-feerate transaction, and attaches a >>>> chain of descendants. >>>> >>>> 2. The attacker replaces the transaction with a smaller but higher >>>> feerate transaction, attaching a new chain of descendants. >>>> >>>> 3. Repeat 1000 times. >>>> >>>> #### Fees in Next Block and Feerate for the Rest of the Mempool >>>> >>>> Perhaps we can look at replacements like this: >>>> >>>> 1. Calculate the directly conflicting transactions and, with their >>>> descendants, the original transactions. Check signaling. Limit the >>>> total volume (e.g. can't be more than 100 total or 1MvB or something). >>>> >>>> 2. Find which original transactions would be in the next ~1 block. The >>>> replacement must pay at least this amount + X% in absolute fees. This >>>> guarantees that the fees of the next block doesn't decrease. >>>> >>>> 3. Find which transactions would be left in the mempool after that ~1 >>>> block. The replacement's feerate must be Y% higher than the maximum >>>> mining score of these transactions. This guarantees that you now have >>>> only *better* candidates in your after-this-block mempool than you did >>>> before, even if the size and fees the transactions decrease. >>>> >>>> 4. Now you have two numbers: a minimum absolute fee amount and a >>>> minimum feerate. Check to see if the replacement(s) meet these >>>> minimums. Also, a wallet would be able to ask the node "What fee and >>>> feerate would I need to put on a transaction replacing this?" and use >>>> this information to fund a replacement transaction, without needing to >>>> guess or overshoot. >>>> >>>> Obviously, there are some magic numbers missing here. X and Y are >>>> TBD constants to ensure we have some kind of rate limiting for the >>>> number of replacements allowed using some set of fees. >>>> >>>> What should they be? We can do some arithmetic to see what happens if >>>> you start with the biggest/lowest feerate transaction and do a bunch >>>> of replacements. Maybe we end up with values that are high enough to >>>> prevent abuse and make sense for applications/users that do RBF. >>>> >>>> ### Mempool Changes Need for Implementation >>>> >>>> As described in the mining score section above, >>>> we may want additional tooling to more accurately assess >>>> the economic gain of replacing transactions in our mempool. >>>> >>>> A few options have been discussed: >>>> >>>> * Calculate block templates on the fly when we need to consider a >>>> replacement. However, since replacements are [quite common][11] >>>> and the information might be useful for other things as well, >>>> it may be worth it to cache a block template. >>>> >>>> * Keep a persistent block template so that we know what transactions >>>> we would put in the next block. We need to remember the feerate >>>> at which each transaction was included in the template, because an >>>> ancestor package may be included in the same block template in >>>> multiple subsets. Transactions included earlier alter the ancestor >>>> feerate of the remaining transactions in the package. We also need >>>> to keep track of the new feerates of transactions left over. >>>> >>>> * Divide the mempool into two layers, "high feerate" and "low >>>> feerate." The high feerate layer contains ~1 block of packages with >>>> the highest ancestor feerates, and the low feerate layer contains >>>> everything else. At the edge of a block, we have a Knapsacky problem >>>> where the next highest ancestor feerate package might not fit, so we >>>> would probably want the high feerate layer ~2MvB or something to avoid >>>> underestimating the fees. >>>> >>>> ## Acknowledgements >>>> >>>> Thank you to everyone whose RBF-related suggestions, grievances, >>>> criticisms and ideas were incorporated in this document: >>>> Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker, >>>> Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery, >>>> Antoine Poinsot, Antoine Riard, Larry Ruane, >>>> S3RK and Bastien Teinturier. >>>> >>>> Thanks for reading! >>>> >>>> Best, >>>> Gloria >>>> >>>> [1]: >>>> https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md >>>> [2]: >>>> https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999 >>>> [3]: >>>> https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea >>>> [4]: https://github.com/bitcoindevkit/bdk/issues/144 >>>> [5]: https://github.com/bitcoindevkit/bdk/issues/414 >>>> [6]: >>>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html >>>> [7]: >>>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2 >>>> [8]: >>>> https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366 >>>> [9]: >>>> https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922 >>>> [10]: >>>> https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md >>>> [11]: >>>> https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670 >>>> [12]: https://github.com/bitcoin/bitcoin/pull/24007 >>>> [13]: >>>> https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114 >>>> [14]: >>>> https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320 >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> >> _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 55813 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-07 11:16 ` Gloria Zhao @ 2022-02-08 4:58 ` Anthony Towns 2022-03-09 15:09 ` Gloria Zhao 0 siblings, 1 reply; 25+ messages in thread From: Anthony Towns @ 2022-02-08 4:58 UTC (permalink / raw) To: Bitcoin Protocol Discussion On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: > @aj: > > I wonder sometimes if it could be sufficient to just have a relay rate > > limit and prioritise by ancestor feerate though. Maybe something like: > > - instead of adding txs to each peers setInventoryTxToSend immediately, > > set a mempool flag "relayed=false" > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to > > each peer's setInventoryTxToSend and mark them as "relayed=true"; > > calculate how much kB those txs were, and do this again after > > SIZE/RATELIMIT seconds > > - don't include "relayed=false" txs when building blocks? The "?" was me not being sure that point is a good suggestion... Miners might reasonably decide to have no rate limit, and always relay, and never exclude txs -- but the question then becomes is whether they hear about the tx at all, so rate limiting behaviour could still be a potential problem for whoever made the tx. > Wow cool! I think outbound tx relay size-based rate-limiting and > prioritizing tx relay by feerate are great ideas for preventing spammers > from wasting bandwidth network-wide. I agree, this would slow the low > feerate spam down, preventing a huge network-wide bandwidth spike. And it > would allow high feerate transactions to propagate as they should, > regardless of how busy traffic is. Combined with inbound tx request > rate-limiting, might this be sufficient to prevent DoS regardless of the > fee-based replacement policies? I think you only want to do outbound rate limits, ie, how often you send INV, GETDATA and TX messages? Once you receive any of those, I think you have to immediately process / ignore it, you can't really sensibly defer it (beyond the existing queues we have that just build up while we're busy processing other things first)? > One point that I'm not 100% clear on: is it ok to prioritize the > transactions by ancestor feerate in this scheme? As I described in the > original post, this can be quite different from the actual feerate we would > consider a transaction in a block for. The transaction could have a high > feerate sibling bumping its ancestor. > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). If > we just received C, it would be incorrect to give it a priority equal to > its ancestor feerate (3sat/vB) because if we constructed a block template > now, B would bump A, and C's new ancestor feerate is 5sat/vB. > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we > also exclude C when building blocks, we're missing out on good fees. I think you're right that this would be ugly. It's something of a special case: a) you really care about C getting into the next block; but b) you're trusting B not being replaced by a higher fee tx that doesn't have A as a parent; and c) there's a lot of txs bidding the floor of the next block up to a level in-between the ancestor fee rate of 3sat/vB and the tx fee rate of 5sat/vB Without (a), maybe you don't care about it getting to a miner quickly. If your trust in (b) was misplaced, then your tx's effective fee rate will drop and (because of (c)), you'll lose anyway. And if the spam ends up outside of (c)'s range, either the rate limiting won't take effect (spam's too cheap) and you'll be fine, or you'll miss out on the block anyway (spam's paying more than your tx rate) and you never had any hope of making it in. Note that we already rate limit via INVENTORY_BROADCAST_MAX / *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs per 10 minutes for outbound connections. This would be a weight based rate limit instead-of/in-addition-to that, I guess. As far as a non-ugly approach goes, I think you'd have to be smarter about tracking the "effective fee rate" than the ancestor fee rate manages; maybe that's something that could fall out of Murch and Clara's candidate set blockbuilding ideas [0] ? Perhaps that same work would also make it possible to come up with a better answer to "do I care that this replacement would invalidate these descendents?" [0] https://github.com/Xekyo/blockbuilding > > - keep high-feerate evicted txs around for a while in case they get > > mined by someone else to improve compact block relay, a la the > > orphan pool? > Replaced transactions are already added to vExtraTxnForCompact :D I guess I was thinking that it's just a 100 tx LRU cache, which might not be good enough? Maybe it would be more on point to have a rate limit apply only to replacement transactions? > For wallets, AJ's "All you need is for there to be *a* path that follows > the new relay rules and gets from your node/wallet to perhaps 10% of > hashpower" makes sense to me (which would be the former). Perhaps a corollarly of that is that it's *better* to have the mempool acceptance rule only consider economic incentives, and have the spam prevention only be about "shall I tell my peers about this?" If you don't have that split; then the anti-spam rules can prevent you from getting the tx in the mempool at all; whereas if you do have the split, then even if the bitcoind anti-spam rules are blocking you at every turn, you can still send your tx to miners by some other route, and then they can add it to their mempool directly without any hassle. Cheers, aj ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-02-08 4:58 ` Anthony Towns @ 2022-03-09 15:09 ` Gloria Zhao 2022-03-11 16:22 ` Billy Tetrud 2022-03-17 2:02 ` Antoine Riard 0 siblings, 2 replies; 25+ messages in thread From: Gloria Zhao @ 2022-03-09 15:09 UTC (permalink / raw) To: Anthony Towns; +Cc: Bitcoin Protocol Discussion [-- Attachment #1: Type: text/plain, Size: 10750 bytes --] Hi RBF friends, Posting a summary of RBF discussions at coredev (mostly on transaction relay rate-limiting), user-elected descendant limit as a short term solution to unblock package RBF, and mining score, all open for feedback: One big concept discussed was baking DoS protection into the p2p level rather than policy level. TLDR: The fees are not paid to the node operator, but to the miner. While we can use fees to reason about the cost of an attack, if we're ultimately interested in preventing resource exhaustion, maybe we want to "stop the bleeding" when it happens and bound the amount of resources used in general. There were two main ideas: 1. Transaction relay rate limiting (i.e. the one you proposed above or some variation) with a feerate-based priority queue 2. Staggered broadcast of replacement transactions: within some time interval, maybe accept multiple replacements for the same prevout, but only relay the original transaction. Looking to solicit feedback on these ideas and the concept in general. Is it a good idea (separate from RBF) to add rate-limiting in transaction relay? And is it the right direction to think about RBF DoS protection this way? A lingering concern that I have about this idea is it would then be possible to impact the propagation of another person’s transaction, i.e., an attacker can censor somebody’s transaction from ever being announced by a node if they send enough transactions to fill up the rate limit. Obviously this would be expensive since they're spending a lot on fees, but I imagine it could be profitable in some situations to spend a few thousand dollars to prevent anyone from hearing about a transaction for a few hours. This might be a non-issue in practice if the rate limit is generous and traffic isn’t horrendous, but is this a problem? And if we don't require an increase in (i.e. addition of "new") absolute fees, users are essentially allowed to “recycle” fees. In the scenario where we prioritize relay based on feerate, users could potentially be placed higher in the queue, ahead of other users’ transactions, multiple times, without ever adding more fees to the transaction. Again, maybe this isn’t a huge deal in practice if we set the parameters right, but it seems… not great, in principle. --------- It's probably also a good idea to point out that there's been some discussion happening on the gist containing my original post on this thread (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). Suhas and Matt [proposed][0] adding a policy rule allowing users to specify descendant limits on their transactions. For example, some nth bit of nSequence with nVersion 3 means "this transaction won't have more than X vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It solves the pinning problem with package RBF where the attacker's package contains a very large and high-fee descendant. We could add this policy and deploy it with package RBF/package relay so that LN can use it by setting the user-elected descendant limit flag on commitment transactions. (Otherwise package RBF is blocked until we find a more comprehensive solution to the pinning attack). It's simple to [implement][1] as a mempool policy, but adds some complexity for wallets that use it, since it limits their use of UTXOs from transactions with this bit set. --------- Also, coming back to the idea of "we can't just use {individual, ancestor} feerate," I'm interested in soliciting feedback on adding a “mining score” calculator. I've implemented one [here][2] which takes the transaction in question, grabs all of the connected mempool transactions (including siblings, coparents, etc., as they wouldn’t be in the ancestor nor descendant sets), and builds a “block template” using our current mining algorithm. The mining score of a transaction is the ancestor feerate at which it is included. This would be helpful for something like ancestor-aware funding and fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited priority queue for transaction relay, we'd want to use something like this as the priority value. And for RBF, we probably want to require that a replacement have a higher mining score than the original transactions. This could be computationally expensive to do all the time; it could be good to cache it but that could make mempool bookkeeping more complicated. Also, if we end up trying to switch to a candidate set-based algorithm for mining, we'd of course need a new calculator. [0]: https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score [3]: https://github.com/bitcoin/bitcoin/issues/9645 [4]: https://github.com/bitcoin/bitcoin/issues/15553 Best, Gloria On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: > On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: > > @aj: > > > I wonder sometimes if it could be sufficient to just have a relay rate > > > limit and prioritise by ancestor feerate though. Maybe something like: > > > - instead of adding txs to each peers setInventoryTxToSend immediately, > > > set a mempool flag "relayed=false" > > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to > > > each peer's setInventoryTxToSend and mark them as "relayed=true"; > > > calculate how much kB those txs were, and do this again after > > > SIZE/RATELIMIT seconds > > > > - don't include "relayed=false" txs when building blocks? > > The "?" was me not being sure that point is a good suggestion... > > Miners might reasonably decide to have no rate limit, and always relay, > and never exclude txs -- but the question then becomes is whether they > hear about the tx at all, so rate limiting behaviour could still be a > potential problem for whoever made the tx. > > > Wow cool! I think outbound tx relay size-based rate-limiting and > > prioritizing tx relay by feerate are great ideas for preventing spammers > > from wasting bandwidth network-wide. I agree, this would slow the low > > feerate spam down, preventing a huge network-wide bandwidth spike. And it > > would allow high feerate transactions to propagate as they should, > > regardless of how busy traffic is. Combined with inbound tx request > > rate-limiting, might this be sufficient to prevent DoS regardless of the > > fee-based replacement policies? > > I think you only want to do outbound rate limits, ie, how often you send > INV, GETDATA and TX messages? Once you receive any of those, I think > you have to immediately process / ignore it, you can't really sensibly > defer it (beyond the existing queues we have that just build up while > we're busy processing other things first)? > > > One point that I'm not 100% clear on: is it ok to prioritize the > > transactions by ancestor feerate in this scheme? As I described in the > > original post, this can be quite different from the actual feerate we > would > > consider a transaction in a block for. The transaction could have a high > > feerate sibling bumping its ancestor. > > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). If > > we just received C, it would be incorrect to give it a priority equal to > > its ancestor feerate (3sat/vB) because if we constructed a block template > > now, B would bump A, and C's new ancestor feerate is 5sat/vB. > > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we > > also exclude C when building blocks, we're missing out on good fees. > > I think you're right that this would be ugly. It's something of a > special case: > > a) you really care about C getting into the next block; but > b) you're trusting B not being replaced by a higher fee tx that > doesn't have A as a parent; and > c) there's a lot of txs bidding the floor of the next block up to a > level in-between the ancestor fee rate of 3sat/vB and the tx fee > rate of 5sat/vB > > Without (a), maybe you don't care about it getting to a miner quickly. > If your trust in (b) was misplaced, then your tx's effective fee rate > will drop and (because of (c)), you'll lose anyway. And if the spam ends > up outside of (c)'s range, either the rate limiting won't take effect > (spam's too cheap) and you'll be fine, or you'll miss out on the block > anyway (spam's paying more than your tx rate) and you never had any hope > of making it in. > > Note that we already rate limit via INVENTORY_BROADCAST_MAX / > *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs > per 10 minutes for outbound connections. This would be a weight based > rate limit instead-of/in-addition-to that, I guess. > > As far as a non-ugly approach goes, I think you'd have to be smarter about > tracking the "effective fee rate" than the ancestor fee rate manages; > maybe that's something that could fall out of Murch and Clara's candidate > set blockbuilding ideas [0] ? > > Perhaps that same work would also make it possible to come up with > a better answer to "do I care that this replacement would invalidate > these descendents?" > > [0] https://github.com/Xekyo/blockbuilding > > > > - keep high-feerate evicted txs around for a while in case they get > > > mined by someone else to improve compact block relay, a la the > > > orphan pool? > > Replaced transactions are already added to vExtraTxnForCompact :D > > I guess I was thinking that it's just a 100 tx LRU cache, which might > not be good enough? > > Maybe it would be more on point to have a rate limit apply only to > replacement transactions? > > > For wallets, AJ's "All you need is for there to be *a* path that follows > > the new relay rules and gets from your node/wallet to perhaps 10% of > > hashpower" makes sense to me (which would be the former). > > Perhaps a corollarly of that is that it's *better* to have the mempool > acceptance rule only consider economic incentives, and have the spam > prevention only be about "shall I tell my peers about this?" > > If you don't have that split; then the anti-spam rules can prevent you > from getting the tx in the mempool at all; whereas if you do have the > split, then even if the bitcoind anti-spam rules are blocking you at > every turn, you can still send your tx to miners by some other route, > and then they can add it to their mempool directly without any hassle. > > Cheers, > aj > > [-- Attachment #2: Type: text/html, Size: 12551 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-09 15:09 ` Gloria Zhao @ 2022-03-11 16:22 ` Billy Tetrud 2022-03-12 8:18 ` Billy Tetrud 2022-03-17 2:02 ` Antoine Riard 1 sibling, 1 reply; 25+ messages in thread From: Billy Tetrud @ 2022-03-11 16:22 UTC (permalink / raw) To: Gloria Zhao, Bitcoin Protocol Discussion; +Cc: Anthony Towns [-- Attachment #1: Type: text/plain, Size: 16777 bytes --] Hi Gloria, > 1. Transaction relay rate limiting I have a similar concern as yours, that this could prevent higher fee-rate transactions from being broadcast. > 2. Staggered broadcast of replacement transactions: within some time interval, maybe accept multiple replacements for the same prevout, but only relay the original transaction. By this do you mean basically having a batching window where, on receiving a replacement transaction, a node will wait for a period of time, potentially receiving many replacements for the same transaction (or many separate conflicting transactions), and only broadcasting the "best" one(s) at the end of that time window? Its an interesting idea, but it would produce a problem. Every hop that replacement transaction takes would be delayed by this staggered window. If the window were 3 minutes and transactions generally take 20 hops to get to the majority of miners, a "best-case average" delay might be 3.75 minutes (noting that among your 8 nodes, its quite likely one of them would have a window ending much sooner than 3 minutes). Some (maybe 3% of) nodes would experience delays of more than 20 minutes. That kind of delay isn't great. However it made me think of another idea: a transaction replacement broadcast cooldown. What if nodes kept track of the time they broadcasted the last replacement for a package and had a relay cooldown after the last replacement was broadcasted? A node receiving a replacement would relay the replacement immediately if the package its replacing was broadcasted more than X seconds ago, and otherwise it would wait until the time when that package was broadcasted at least X seconds ago to broadcast it. Any replacements it receives during that waiting period would replace as normal, meaning the unrebroadcasted replacement would never be broadcasted, and only the highest value replacement would be broadcasted at the end of the cooldown. This wouldn't prevent a higher-fee-rate transaction from being broadcasted (like rate limiting could), but would still be effective at limiting unnecessary data transmission. Another benefit is that in the non-adversarial case, replacement transactions wouldn't be subject to any delay at all (while in the staggered broadcast idea, most replacements would experience some delay). And in the adversarial case, where a malicious actor broadcasts a low-as-possible-value replacement just before yours, the worst case delay is just whatever the cooldown period is. I would imagine that maybe 1 minute would be a reasonable worst-case delay. This would limit spam for a transaction that makes it into a block to ~10x (9 to 1). I don't see much of a downside to doing this beyond just the slight additional complexity of relay rules (and considering it could save substantial additional code complexity, even that is a benefit). All a node would need to do is keep a timestamp on each transaction they receive for when it was broadcasted and check it when a replacement comes in. If now-broadcastDate < cooldown, set a timer for cooldown - (now-broadcastDate) to broadcast it. If another replacement comes in, clear that timer and repeat using the original broadcast date (since the unbroadcast transaction doesn't have a broadcast date yet). I think it might also be useful to note that eliminating "extra data" caused by careless or malicious actors (spam or whatever you want to call it) should not be the goal. It is impossible to prevent all spam. What we should be aiming for is more specific: we should attempt to design a system where spam is manageable. Eg if our goal is to ensure that a bitcoin node uses no more than 10% of the bandwidth of a "normal" user, if current non-spam traffic only requires 1% of a "normal" users's bandwidth, then the network can bear a 9 to 1 ratio of spam. When a node spins up, there is a lot more data to download and process. So we know that all full nodes can handle at least as much traffic as they handle during IBD. What's the difference between those amounts? I'm not sure, but I would guess that IBD is at least a couple times more demanding than a fully synced node. So I might suggest that as long as spam can be kept below a ratio of maybe 2 to 1, we should consider the design acceptable (and therefore more complexity unnecessary). The 1 minute broadcast cooldown I mentioned before wouldn't be quite sufficient to achieve that ratio. But a 3.33 minute cooldown would be. Whether this is "too much" is something that would have to be discussed, I suspect a worst-case adversarial 3.33 minute delay would not be "too much". Doing this could basically eliminate any risk of actual service denial via replacement transactions. However, I do think that these DOS concerns are quite overblown. I wrote up a comment on your rbf-improvements.md <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing my thought process on that. The summary is that as long as the fee-rate relay rule is maintained, any "spam" is actually paid for, either by the "first" transaction in the spam chain, or by the "spam" itself. Even without something like a minimum RBF relay delay limiting how much spam could be created, the economics of the fee-rate rule already sufficiently mitigate the issue of spam. On Wed, Mar 9, 2022 at 9:37 AM Gloria Zhao via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi RBF friends, > > Posting a summary of RBF discussions at coredev (mostly on transaction > relay rate-limiting), user-elected descendant limit as a short term > solution to unblock package RBF, and mining score, all open for feedback: > > One big concept discussed was baking DoS protection into the p2p level > rather than policy level. TLDR: The fees are not paid to the node operator, > but to the miner. While we can use fees to reason about the cost of an > attack, if we're ultimately interested in preventing resource exhaustion, > maybe we want to "stop the bleeding" when it happens and bound the amount > of resources used in general. There were two main ideas: > > 1. Transaction relay rate limiting (i.e. the one you proposed above or > some variation) with a feerate-based priority queue > 2. Staggered broadcast of replacement transactions: within some time > interval, maybe accept multiple replacements for the same prevout, but only > relay the original transaction. > > Looking to solicit feedback on these ideas and the concept in general. Is > it a good idea (separate from RBF) to add rate-limiting in transaction > relay? And is it the right direction to think about RBF DoS protection this > way? > > A lingering concern that I have about this idea is it would then be > possible to impact the propagation of another person’s transaction, i.e., > an attacker can censor somebody’s transaction from ever being announced by > a node if they send enough transactions to fill up the rate limit. > Obviously this would be expensive since they're spending a lot on fees, but > I imagine it could be profitable in some situations to spend a few thousand > dollars to prevent anyone from hearing about a transaction for a few hours. > This might be a non-issue in practice if the rate limit is generous and > traffic isn’t horrendous, but is this a problem? > > And if we don't require an increase in (i.e. addition of "new") absolute > fees, users are essentially allowed to “recycle” fees. In the scenario > where we prioritize relay based on feerate, users could potentially be > placed higher in the queue, ahead of other users’ transactions, multiple > times, without ever adding more fees to the transaction. Again, maybe this > isn’t a huge deal in practice if we set the parameters right, but it seems… > not great, in principle. > > --------- > > It's probably also a good idea to point out that there's been some > discussion happening on the gist containing my original post on this thread > (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). > > Suhas and Matt [proposed][0] adding a policy rule allowing users to > specify descendant limits on their transactions. For example, some nth bit > of nSequence with nVersion 3 means "this transaction won't have more than X > vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It > solves the pinning problem with package RBF where the attacker's package > contains a very large and high-fee descendant. > > We could add this policy and deploy it with package RBF/package relay so > that LN can use it by setting the user-elected descendant limit flag on > commitment transactions. (Otherwise package RBF is blocked until we find a > more comprehensive solution to the pinning attack). > > It's simple to [implement][1] as a mempool policy, but adds some > complexity for wallets that use it, since it limits their use of UTXOs from > transactions with this bit set. > > --------- > > Also, coming back to the idea of "we can't just use {individual, ancestor} > feerate," I'm interested in soliciting feedback on adding a “mining score” > calculator. I've implemented one [here][2] which takes the transaction in > question, grabs all of the connected mempool transactions (including > siblings, coparents, etc., as they wouldn’t be in the ancestor nor > descendant sets), and builds a “block template” using our current mining > algorithm. The mining score of a transaction is the ancestor feerate at > which it is included. > > This would be helpful for something like ancestor-aware funding and > fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited > priority queue for transaction relay, we'd want to use something like this > as the priority value. And for RBF, we probably want to require that a > replacement have a higher mining score than the original transactions. This > could be computationally expensive to do all the time; it could be good to > cache it but that could make mempool bookkeeping more complicated. Also, if > we end up trying to switch to a candidate set-based algorithm for mining, > we'd of course need a new calculator. > > [0]: > https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 > [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit > [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score > [3]: https://github.com/bitcoin/bitcoin/issues/9645 > [4]: https://github.com/bitcoin/bitcoin/issues/15553 > > Best, > Gloria > > On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: > >> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >> > @aj: >> > > I wonder sometimes if it could be sufficient to just have a relay rate >> > > limit and prioritise by ancestor feerate though. Maybe something like: >> > > - instead of adding txs to each peers setInventoryTxToSend >> immediately, >> > > set a mempool flag "relayed=false" >> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to >> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >> > > calculate how much kB those txs were, and do this again after >> > > SIZE/RATELIMIT seconds >> >> > > - don't include "relayed=false" txs when building blocks? >> >> The "?" was me not being sure that point is a good suggestion... >> >> Miners might reasonably decide to have no rate limit, and always relay, >> and never exclude txs -- but the question then becomes is whether they >> hear about the tx at all, so rate limiting behaviour could still be a >> potential problem for whoever made the tx. >> >> > Wow cool! I think outbound tx relay size-based rate-limiting and >> > prioritizing tx relay by feerate are great ideas for preventing spammers >> > from wasting bandwidth network-wide. I agree, this would slow the low >> > feerate spam down, preventing a huge network-wide bandwidth spike. And >> it >> > would allow high feerate transactions to propagate as they should, >> > regardless of how busy traffic is. Combined with inbound tx request >> > rate-limiting, might this be sufficient to prevent DoS regardless of the >> > fee-based replacement policies? >> >> I think you only want to do outbound rate limits, ie, how often you send >> INV, GETDATA and TX messages? Once you receive any of those, I think >> you have to immediately process / ignore it, you can't really sensibly >> defer it (beyond the existing queues we have that just build up while >> we're busy processing other things first)? >> >> > One point that I'm not 100% clear on: is it ok to prioritize the >> > transactions by ancestor feerate in this scheme? As I described in the >> > original post, this can be quite different from the actual feerate we >> would >> > consider a transaction in a block for. The transaction could have a high >> > feerate sibling bumping its ancestor. >> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). >> If >> > we just received C, it would be incorrect to give it a priority equal to >> > its ancestor feerate (3sat/vB) because if we constructed a block >> template >> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we >> > also exclude C when building blocks, we're missing out on good fees. >> >> I think you're right that this would be ugly. It's something of a >> special case: >> >> a) you really care about C getting into the next block; but >> b) you're trusting B not being replaced by a higher fee tx that >> doesn't have A as a parent; and >> c) there's a lot of txs bidding the floor of the next block up to a >> level in-between the ancestor fee rate of 3sat/vB and the tx fee >> rate of 5sat/vB >> >> Without (a), maybe you don't care about it getting to a miner quickly. >> If your trust in (b) was misplaced, then your tx's effective fee rate >> will drop and (because of (c)), you'll lose anyway. And if the spam ends >> up outside of (c)'s range, either the rate limiting won't take effect >> (spam's too cheap) and you'll be fine, or you'll miss out on the block >> anyway (spam's paying more than your tx rate) and you never had any hope >> of making it in. >> >> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >> per 10 minutes for outbound connections. This would be a weight based >> rate limit instead-of/in-addition-to that, I guess. >> >> As far as a non-ugly approach goes, I think you'd have to be smarter about >> tracking the "effective fee rate" than the ancestor fee rate manages; >> maybe that's something that could fall out of Murch and Clara's candidate >> set blockbuilding ideas [0] ? >> >> Perhaps that same work would also make it possible to come up with >> a better answer to "do I care that this replacement would invalidate >> these descendents?" >> >> [0] https://github.com/Xekyo/blockbuilding >> >> > > - keep high-feerate evicted txs around for a while in case they get >> > > mined by someone else to improve compact block relay, a la the >> > > orphan pool? >> > Replaced transactions are already added to vExtraTxnForCompact :D >> >> I guess I was thinking that it's just a 100 tx LRU cache, which might >> not be good enough? >> >> Maybe it would be more on point to have a rate limit apply only to >> replacement transactions? >> >> > For wallets, AJ's "All you need is for there to be *a* path that follows >> > the new relay rules and gets from your node/wallet to perhaps 10% of >> > hashpower" makes sense to me (which would be the former). >> >> Perhaps a corollarly of that is that it's *better* to have the mempool >> acceptance rule only consider economic incentives, and have the spam >> prevention only be about "shall I tell my peers about this?" >> >> If you don't have that split; then the anti-spam rules can prevent you >> from getting the tx in the mempool at all; whereas if you do have the >> split, then even if the bitcoind anti-spam rules are blocking you at >> every turn, you can still send your tx to miners by some other route, >> and then they can add it to their mempool directly without any hassle. >> >> Cheers, >> aj >> >> _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 19343 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-11 16:22 ` Billy Tetrud @ 2022-03-12 8:18 ` Billy Tetrud 2022-03-14 10:29 ` Gloria Zhao 0 siblings, 1 reply; 25+ messages in thread From: Billy Tetrud @ 2022-03-12 8:18 UTC (permalink / raw) To: Gloria Zhao, Bitcoin Protocol Discussion; +Cc: Anthony Towns [-- Attachment #1: Type: text/plain, Size: 17656 bytes --] In reading through more of the discussion, it seems the idea I presented above might basically be a reformulation of t-bast's rate-limiting idea presented in this comment <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4081349#gistcomment-4081349>. Perhaps he could comment on whether that characterization is accurate. On Fri, Mar 11, 2022 at 10:22 AM Billy Tetrud <billy.tetrud@gmail.com> wrote: > Hi Gloria, > > > 1. Transaction relay rate limiting > > I have a similar concern as yours, that this could prevent higher fee-rate > transactions from being broadcast. > > > 2. Staggered broadcast of replacement transactions: within some time > interval, maybe accept multiple replacements for the same prevout, but only > relay the original transaction. > > By this do you mean basically having a batching window where, on receiving > a replacement transaction, a node will wait for a period of time, > potentially receiving many replacements for the same transaction (or many > separate conflicting transactions), and only broadcasting the "best" one(s) > at the end of that time window? > > Its an interesting idea, but it would produce a problem. Every hop that > replacement transaction takes would be delayed by this staggered window. If > the window were 3 minutes and transactions generally take 20 hops to get to > the majority of miners, a "best-case average" delay might be 3.75 minutes > (noting that among your 8 nodes, its quite likely one of them would have a > window ending much sooner than 3 minutes). Some (maybe 3% of) nodes would > experience delays of more than 20 minutes. That kind of delay isn't great. > > However it made me think of another idea: a transaction replacement > broadcast cooldown. What if nodes kept track of the time they broadcasted > the last replacement for a package and had a relay cooldown after the last > replacement was broadcasted? A node receiving a replacement would relay the > replacement immediately if the package its replacing was broadcasted more > than X seconds ago, and otherwise it would wait until the time when that > package was broadcasted at least X seconds ago to broadcast it. Any > replacements it receives during that waiting period would replace as > normal, meaning the unrebroadcasted replacement would never be > broadcasted, and only the highest value replacement would be broadcasted at > the end of the cooldown. > > This wouldn't prevent a higher-fee-rate transaction from being broadcasted > (like rate limiting could), but would still be effective at limiting > unnecessary data transmission. Another benefit is that in the > non-adversarial case, replacement transactions wouldn't be subject to any > delay at all (while in the staggered broadcast idea, most replacements > would experience some delay). And in the adversarial case, where a > malicious actor broadcasts a low-as-possible-value replacement just before > yours, the worst case delay is just whatever the cooldown period is. I > would imagine that maybe 1 minute would be a reasonable worst-case delay. > This would limit spam for a transaction that makes it into a block to ~10x > (9 to 1). I don't see much of a downside to doing this beyond just the > slight additional complexity of relay rules (and considering it could save > substantial additional code complexity, even that is a benefit). > > All a node would need to do is keep a timestamp on each transaction they > receive for when it was broadcasted and check it when a replacement comes > in. If now-broadcastDate < cooldown, set a timer for cooldown - > (now-broadcastDate) to broadcast it. If another replacement comes in, clear > that timer and repeat using the original broadcast date (since the > unbroadcast transaction doesn't have a broadcast date yet). > > I think it might also be useful to note that eliminating "extra data" > caused by careless or malicious actors (spam or whatever you want to call > it) should not be the goal. It is impossible to prevent all spam. What we > should be aiming for is more specific: we should attempt to design a system > where spam is manageable. Eg if our goal is to ensure that a bitcoin node > uses no more than 10% of the bandwidth of a "normal" user, if current > non-spam traffic only requires 1% of a "normal" users's bandwidth, then the > network can bear a 9 to 1 ratio of spam. When a node spins up, there is a > lot more data to download and process. So we know that all full nodes can > handle at least as much traffic as they handle during IBD. What's the > difference between those amounts? I'm not sure, but I would guess that IBD > is at least a couple times more demanding than a fully synced node. So I > might suggest that as long as spam can be kept below a ratio of maybe 2 to > 1, we should consider the design acceptable (and therefore more complexity > unnecessary). > > The 1 minute broadcast cooldown I mentioned before wouldn't be quite > sufficient to achieve that ratio. But a 3.33 minute cooldown would be. > Whether this is "too much" is something that would have to be discussed, I > suspect a worst-case adversarial 3.33 minute delay would not be "too much". > Doing this could basically eliminate any risk of actual service denial via > replacement transactions. > > However, I do think that these DOS concerns are quite overblown. I wrote > up a comment on your rbf-improvements.md > <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing > my thought process on that. The summary is that as long as the fee-rate > relay rule is maintained, any "spam" is actually paid for, either by the > "first" transaction in the spam chain, or by the "spam" itself. Even > without something like a minimum RBF relay delay limiting how much spam > could be created, the economics of the fee-rate rule already sufficiently > mitigate the issue of spam. > On Wed, Mar 9, 2022 at 9:37 AM Gloria Zhao via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> Hi RBF friends, >> >> Posting a summary of RBF discussions at coredev (mostly on transaction >> relay rate-limiting), user-elected descendant limit as a short term >> solution to unblock package RBF, and mining score, all open for feedback: >> >> One big concept discussed was baking DoS protection into the p2p level >> rather than policy level. TLDR: The fees are not paid to the node operator, >> but to the miner. While we can use fees to reason about the cost of an >> attack, if we're ultimately interested in preventing resource exhaustion, >> maybe we want to "stop the bleeding" when it happens and bound the amount >> of resources used in general. There were two main ideas: >> >> 1. Transaction relay rate limiting (i.e. the one you proposed above or >> some variation) with a feerate-based priority queue >> 2. Staggered broadcast of replacement transactions: within some time >> interval, maybe accept multiple replacements for the same prevout, but only >> relay the original transaction. >> >> Looking to solicit feedback on these ideas and the concept in general. Is >> it a good idea (separate from RBF) to add rate-limiting in transaction >> relay? And is it the right direction to think about RBF DoS protection this >> way? >> >> A lingering concern that I have about this idea is it would then be >> possible to impact the propagation of another person’s transaction, i.e., >> an attacker can censor somebody’s transaction from ever being announced by >> a node if they send enough transactions to fill up the rate limit. >> Obviously this would be expensive since they're spending a lot on fees, but >> I imagine it could be profitable in some situations to spend a few thousand >> dollars to prevent anyone from hearing about a transaction for a few hours. >> This might be a non-issue in practice if the rate limit is generous and >> traffic isn’t horrendous, but is this a problem? >> >> And if we don't require an increase in (i.e. addition of "new") absolute >> fees, users are essentially allowed to “recycle” fees. In the scenario >> where we prioritize relay based on feerate, users could potentially be >> placed higher in the queue, ahead of other users’ transactions, multiple >> times, without ever adding more fees to the transaction. Again, maybe this >> isn’t a huge deal in practice if we set the parameters right, but it seems… >> not great, in principle. >> >> --------- >> >> It's probably also a good idea to point out that there's been some >> discussion happening on the gist containing my original post on this thread >> (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). >> >> Suhas and Matt [proposed][0] adding a policy rule allowing users to >> specify descendant limits on their transactions. For example, some nth bit >> of nSequence with nVersion 3 means "this transaction won't have more than X >> vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It >> solves the pinning problem with package RBF where the attacker's package >> contains a very large and high-fee descendant. >> >> We could add this policy and deploy it with package RBF/package relay so >> that LN can use it by setting the user-elected descendant limit flag on >> commitment transactions. (Otherwise package RBF is blocked until we find a >> more comprehensive solution to the pinning attack). >> >> It's simple to [implement][1] as a mempool policy, but adds some >> complexity for wallets that use it, since it limits their use of UTXOs from >> transactions with this bit set. >> >> --------- >> >> Also, coming back to the idea of "we can't just use {individual, >> ancestor} feerate," I'm interested in soliciting feedback on adding a >> “mining score” calculator. I've implemented one [here][2] which takes the >> transaction in question, grabs all of the connected mempool transactions >> (including siblings, coparents, etc., as they wouldn’t be in the ancestor >> nor descendant sets), and builds a “block template” using our current >> mining algorithm. The mining score of a transaction is the ancestor feerate >> at which it is included. >> >> This would be helpful for something like ancestor-aware funding and >> fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited >> priority queue for transaction relay, we'd want to use something like this >> as the priority value. And for RBF, we probably want to require that a >> replacement have a higher mining score than the original transactions. This >> could be computationally expensive to do all the time; it could be good to >> cache it but that could make mempool bookkeeping more complicated. Also, if >> we end up trying to switch to a candidate set-based algorithm for mining, >> we'd of course need a new calculator. >> >> [0]: >> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 >> [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit >> [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score >> [3]: https://github.com/bitcoin/bitcoin/issues/9645 >> [4]: https://github.com/bitcoin/bitcoin/issues/15553 >> >> Best, >> Gloria >> >> On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: >> >>> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >>> > @aj: >>> > > I wonder sometimes if it could be sufficient to just have a relay >>> rate >>> > > limit and prioritise by ancestor feerate though. Maybe something >>> like: >>> > > - instead of adding txs to each peers setInventoryTxToSend >>> immediately, >>> > > set a mempool flag "relayed=false" >>> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to >>> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >>> > > calculate how much kB those txs were, and do this again after >>> > > SIZE/RATELIMIT seconds >>> >>> > > - don't include "relayed=false" txs when building blocks? >>> >>> The "?" was me not being sure that point is a good suggestion... >>> >>> Miners might reasonably decide to have no rate limit, and always relay, >>> and never exclude txs -- but the question then becomes is whether they >>> hear about the tx at all, so rate limiting behaviour could still be a >>> potential problem for whoever made the tx. >>> >>> > Wow cool! I think outbound tx relay size-based rate-limiting and >>> > prioritizing tx relay by feerate are great ideas for preventing >>> spammers >>> > from wasting bandwidth network-wide. I agree, this would slow the low >>> > feerate spam down, preventing a huge network-wide bandwidth spike. And >>> it >>> > would allow high feerate transactions to propagate as they should, >>> > regardless of how busy traffic is. Combined with inbound tx request >>> > rate-limiting, might this be sufficient to prevent DoS regardless of >>> the >>> > fee-based replacement policies? >>> >>> I think you only want to do outbound rate limits, ie, how often you send >>> INV, GETDATA and TX messages? Once you receive any of those, I think >>> you have to immediately process / ignore it, you can't really sensibly >>> defer it (beyond the existing queues we have that just build up while >>> we're busy processing other things first)? >>> >>> > One point that I'm not 100% clear on: is it ok to prioritize the >>> > transactions by ancestor feerate in this scheme? As I described in the >>> > original post, this can be quite different from the actual feerate we >>> would >>> > consider a transaction in a block for. The transaction could have a >>> high >>> > feerate sibling bumping its ancestor. >>> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). >>> If >>> > we just received C, it would be incorrect to give it a priority equal >>> to >>> > its ancestor feerate (3sat/vB) because if we constructed a block >>> template >>> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >>> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we >>> > also exclude C when building blocks, we're missing out on good fees. >>> >>> I think you're right that this would be ugly. It's something of a >>> special case: >>> >>> a) you really care about C getting into the next block; but >>> b) you're trusting B not being replaced by a higher fee tx that >>> doesn't have A as a parent; and >>> c) there's a lot of txs bidding the floor of the next block up to a >>> level in-between the ancestor fee rate of 3sat/vB and the tx fee >>> rate of 5sat/vB >>> >>> Without (a), maybe you don't care about it getting to a miner quickly. >>> If your trust in (b) was misplaced, then your tx's effective fee rate >>> will drop and (because of (c)), you'll lose anyway. And if the spam ends >>> up outside of (c)'s range, either the rate limiting won't take effect >>> (spam's too cheap) and you'll be fine, or you'll miss out on the block >>> anyway (spam's paying more than your tx rate) and you never had any hope >>> of making it in. >>> >>> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >>> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >>> per 10 minutes for outbound connections. This would be a weight based >>> rate limit instead-of/in-addition-to that, I guess. >>> >>> As far as a non-ugly approach goes, I think you'd have to be smarter >>> about >>> tracking the "effective fee rate" than the ancestor fee rate manages; >>> maybe that's something that could fall out of Murch and Clara's candidate >>> set blockbuilding ideas [0] ? >>> >>> Perhaps that same work would also make it possible to come up with >>> a better answer to "do I care that this replacement would invalidate >>> these descendents?" >>> >>> [0] https://github.com/Xekyo/blockbuilding >>> >>> > > - keep high-feerate evicted txs around for a while in case they get >>> > > mined by someone else to improve compact block relay, a la the >>> > > orphan pool? >>> > Replaced transactions are already added to vExtraTxnForCompact :D >>> >>> I guess I was thinking that it's just a 100 tx LRU cache, which might >>> not be good enough? >>> >>> Maybe it would be more on point to have a rate limit apply only to >>> replacement transactions? >>> >>> > For wallets, AJ's "All you need is for there to be *a* path that >>> follows >>> > the new relay rules and gets from your node/wallet to perhaps 10% of >>> > hashpower" makes sense to me (which would be the former). >>> >>> Perhaps a corollarly of that is that it's *better* to have the mempool >>> acceptance rule only consider economic incentives, and have the spam >>> prevention only be about "shall I tell my peers about this?" >>> >>> If you don't have that split; then the anti-spam rules can prevent you >>> from getting the tx in the mempool at all; whereas if you do have the >>> split, then even if the bitcoind anti-spam rules are blocking you at >>> every turn, you can still send your tx to miners by some other route, >>> and then they can add it to their mempool directly without any hassle. >>> >>> Cheers, >>> aj >>> >>> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > [-- Attachment #2: Type: text/html, Size: 20081 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-12 8:18 ` Billy Tetrud @ 2022-03-14 10:29 ` Gloria Zhao 2022-03-15 1:43 ` Billy Tetrud 0 siblings, 1 reply; 25+ messages in thread From: Gloria Zhao @ 2022-03-14 10:29 UTC (permalink / raw) To: Billy Tetrud; +Cc: Bitcoin Protocol Discussion, Anthony Towns [-- Attachment #1: Type: text/plain, Size: 25249 bytes --] Hi Billy, > We should expect miners will be using a more complex, more optimal way of determining what blocks they're working on [...] we should instead run with the assumption that miners keep all potentially relevant transactions in their mempools, including potentially many conflicting transctions, in order to create the most profitable blocks. And therefore we shouldn't put the constraint on normal non-mining full nodes to do that same more-complex mempool behavior or add any complexity for the purpose of denying transaction replacements. > I think a lot of the complexity in these ideas is because of the attempt to match relay rules with miner inclusion rules. I think the assumption that miners are using a completely different implementation of mempool and block template building is false. IIUC, most miners use Bitcoin Core and perhaps configure their node differently (e.g. larger mempool and different minfeerates), but also use `getblocktemplate` which means the same ancestor package-based mining algorithm. Of course, I'm not a miner, so if anybody is a miner or has seen miners' setups, please correct me if I'm wrong. In either case, we would want our mining algorithm to result in block templates that are as close as possible to perfectly incentive compatibile. Fundamentally, I believe default mempool policy (which perhaps naturally creates a network-wide transaction relay policy) should be as close to the mining code as possible. Imagine node A only keeps 1 block's worth of transactions, and node B keeps a (default) 300MB mempool. The contents of node A's mempool should be as close as possible to a block template generated from node B's mempool. Otherwise, node A's mempool is not very useful - their fee estimation is flawed and compact block relay won't do them much good if they need to re-request a lot of block transactions. Next, imagine that node B is a miner. It would be very suboptimal if the mining code was ancestor package-based (i.e. supports CPFP), but the mempool policy only cared about individual feerate, and evicted low-fee parents despite their high-fee children. It's easy to see why this would be suboptimal. Attempting to match mempool policy with the mining algorithm is also arguably the point of package relay. Our mining code uses ancestor packages which is good, but we only submit transactions one at a time to the mempool, so a transaction's high-fee children can't be considered until they are all already in the mempool. Package relay allows us to start thinking about ancestor packages immediately when evaluating transactions for submission to the mempool. The attempt to match policy with miner inclusion rules is deliberate and necessary. > I want to echo James O'Beirne's opinion on this that this may be the wrong path to go down (a path of more complexity without much gain). He said: "Special consideration for "what should be in the next block" and/or the caching of block templates seems like an imposing dependency, dragging in a bunch of state and infrastructure to a question that should be solely limited to mempool feerate aggregates and the feerate of the particular txn package a wallet is concerned with." It seems that I under-explained the purpose of building/caching block templates in my original post, since both you and James have the same misunderstanding. Since RBF's introduction, we have improved to an ancestor package-based mining algorithm. This supports CPFP (incentive compatible) and it is now common to see more complex "families" of transactions as users fee-bump transactions (market is working, yay). On the other hand, we no longer have an accurate way of determining a transaction's "mining score," i.e., the feerate of this transaction's ancestor package when it is included in a block template using our current mining algorithm. This limitation is a big blocker in proposing new fee/feerate RBF rules. For example, if we say "the transaction needs a better feerate," this is obviously flawed, since the original transactions may have very high-feerate children, and the replacement transaction may have low feerate parents. So what we really want is "the transaction needs to be more incentive compatible to mine based on our mining algorithm," but we have no way of getting that information right now. In my original post, I [described 4 heuristics to get transaction's "mining score"][1] using the current data cached in the mempool (e.g. max ancestor feerate of descendant set), as well as why they don't work. As such, the best way to calculate a transaction's mining score AFAICT is to grab all of the related transactions and build a mini "block template" with them. The [implementation][2] I sent last week also cuts out some of the fluff, so the pseudocode looks like this: // Get ALL connected entries (ancestors, descendants, siblings, cousins, coparents, etc.) vector<TxMempoolEntry> cluster = mempool.GetAllTransactionsRelatedTo(txid); sort(cluster, ancestorfeerate); // For deducting ancestors when they are included separately vector<ModifiedTxMempoolEntry> modified_entries; while (!cluster.empty() and !modified_entries.empty()) { iter = BetterAncestorFeerateOf(cluster, modified_entries); best_ancestor_package = GetAncestorSet(iter); mining_score = Feerate(best_ancestor_package); for (entry : best_ancestor_package) { mining_scores[entry] = mining_score; for (descendant : GetAllDescendants(entry) { modified_entries[descendant].DeductAncestorFromAccounting(entry); } } } Perhaps somebody will come up with a better way to do this, but this is my solution, and I hope I've now sufficiently described why I've made an effort to think about this. It's not because I want to make things more fancy or complicated, but because we currently have no way of accurately determining a transaction's mining score. The reason I proposed a persistent block template is so we can efficiently query the mining score of all transactions in the mempool. > However, I do think that these DOS concerns are quite overblown. I wrote up a comment on your rbf-improvements.md <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing my thought process on that. The summary is that as long as the fee-rate relay rule is maintained, any "spam" is actually paid for, either by the "first" transaction in the spam chain, or by the "spam" itself. The DoS concern is not overblown. I recommend you re-read the [current RBF policy][3]; Rule 3 and 4 *are* the feerate relay rules. Removing Rule 3 means allowing recycled fees, so new transactions are not necessarily "paying" anything. [1]: https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#mining-score-of-a-mempool-transaction [2]: https://github.com/glozow/bitcoin/tree/2022-02-mining-score [3]: https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md Best, Gloria On Sat, Mar 12, 2022 at 8:18 AM Billy Tetrud <billy.tetrud@gmail.com> wrote: > In reading through more of the discussion, it seems the idea I presented > above might basically be a reformulation of t-bast's rate-limiting idea > presented in this comment > <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4081349#gistcomment-4081349>. > Perhaps he could comment on whether that characterization is accurate. > > On Fri, Mar 11, 2022 at 10:22 AM Billy Tetrud <billy.tetrud@gmail.com> > wrote: > >> Hi Gloria, >> >> > 1. Transaction relay rate limiting >> >> I have a similar concern as yours, that this could prevent higher >> fee-rate transactions from being broadcast. >> >> > 2. Staggered broadcast of replacement transactions: within some time >> interval, maybe accept multiple replacements for the same prevout, but only >> relay the original transaction. >> >> By this do you mean basically having a batching window where, on >> receiving a replacement transaction, a node will wait for a period of time, >> potentially receiving many replacements for the same transaction (or many >> separate conflicting transactions), and only broadcasting the "best" one(s) >> at the end of that time window? >> >> Its an interesting idea, but it would produce a problem. Every hop that >> replacement transaction takes would be delayed by this staggered window. If >> the window were 3 minutes and transactions generally take 20 hops to get to >> the majority of miners, a "best-case average" delay might be 3.75 minutes >> (noting that among your 8 nodes, its quite likely one of them would have a >> window ending much sooner than 3 minutes). Some (maybe 3% of) nodes would >> experience delays of more than 20 minutes. That kind of delay isn't great. >> >> However it made me think of another idea: a transaction replacement >> broadcast cooldown. What if nodes kept track of the time they broadcasted >> the last replacement for a package and had a relay cooldown after the last >> replacement was broadcasted? A node receiving a replacement would relay the >> replacement immediately if the package its replacing was broadcasted more >> than X seconds ago, and otherwise it would wait until the time when that >> package was broadcasted at least X seconds ago to broadcast it. Any >> replacements it receives during that waiting period would replace as >> normal, meaning the unrebroadcasted replacement would never be >> broadcasted, and only the highest value replacement would be broadcasted at >> the end of the cooldown. >> >> This wouldn't prevent a higher-fee-rate transaction from being >> broadcasted (like rate limiting could), but would still be effective at >> limiting unnecessary data transmission. Another benefit is that in the >> non-adversarial case, replacement transactions wouldn't be subject to any >> delay at all (while in the staggered broadcast idea, most replacements >> would experience some delay). And in the adversarial case, where a >> malicious actor broadcasts a low-as-possible-value replacement just before >> yours, the worst case delay is just whatever the cooldown period is. I >> would imagine that maybe 1 minute would be a reasonable worst-case delay. >> This would limit spam for a transaction that makes it into a block to ~10x >> (9 to 1). I don't see much of a downside to doing this beyond just the >> slight additional complexity of relay rules (and considering it could save >> substantial additional code complexity, even that is a benefit). >> >> All a node would need to do is keep a timestamp on each transaction they >> receive for when it was broadcasted and check it when a replacement comes >> in. If now-broadcastDate < cooldown, set a timer for cooldown - >> (now-broadcastDate) to broadcast it. If another replacement comes in, clear >> that timer and repeat using the original broadcast date (since the >> unbroadcast transaction doesn't have a broadcast date yet). >> >> I think it might also be useful to note that eliminating "extra data" >> caused by careless or malicious actors (spam or whatever you want to call >> it) should not be the goal. It is impossible to prevent all spam. What we >> should be aiming for is more specific: we should attempt to design a system >> where spam is manageable. Eg if our goal is to ensure that a bitcoin node >> uses no more than 10% of the bandwidth of a "normal" user, if current >> non-spam traffic only requires 1% of a "normal" users's bandwidth, then the >> network can bear a 9 to 1 ratio of spam. When a node spins up, there is a >> lot more data to download and process. So we know that all full nodes can >> handle at least as much traffic as they handle during IBD. What's the >> difference between those amounts? I'm not sure, but I would guess that IBD >> is at least a couple times more demanding than a fully synced node. So I >> might suggest that as long as spam can be kept below a ratio of maybe 2 to >> 1, we should consider the design acceptable (and therefore more complexity >> unnecessary). >> >> The 1 minute broadcast cooldown I mentioned before wouldn't be quite >> sufficient to achieve that ratio. But a 3.33 minute cooldown would be. >> Whether this is "too much" is something that would have to be discussed, I >> suspect a worst-case adversarial 3.33 minute delay would not be "too much". >> Doing this could basically eliminate any risk of actual service denial via >> replacement transactions. >> >> However, I do think that these DOS concerns are quite overblown. I wrote >> up a comment on your rbf-improvements.md >> <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing >> my thought process on that. The summary is that as long as the fee-rate >> relay rule is maintained, any "spam" is actually paid for, either by the >> "first" transaction in the spam chain, or by the "spam" itself. Even >> without something like a minimum RBF relay delay limiting how much spam >> could be created, the economics of the fee-rate rule already sufficiently >> mitigate the issue of spam. >> On Wed, Mar 9, 2022 at 9:37 AM Gloria Zhao via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> >>> Hi RBF friends, >>> >>> Posting a summary of RBF discussions at coredev (mostly on transaction >>> relay rate-limiting), user-elected descendant limit as a short term >>> solution to unblock package RBF, and mining score, all open for feedback: >>> >>> One big concept discussed was baking DoS protection into the p2p level >>> rather than policy level. TLDR: The fees are not paid to the node operator, >>> but to the miner. While we can use fees to reason about the cost of an >>> attack, if we're ultimately interested in preventing resource exhaustion, >>> maybe we want to "stop the bleeding" when it happens and bound the amount >>> of resources used in general. There were two main ideas: >>> >>> 1. Transaction relay rate limiting (i.e. the one you proposed above or >>> some variation) with a feerate-based priority queue >>> 2. Staggered broadcast of replacement transactions: within some time >>> interval, maybe accept multiple replacements for the same prevout, but only >>> relay the original transaction. >>> >>> Looking to solicit feedback on these ideas and the concept in general. >>> Is it a good idea (separate from RBF) to add rate-limiting in transaction >>> relay? And is it the right direction to think about RBF DoS protection this >>> way? >>> >>> A lingering concern that I have about this idea is it would then be >>> possible to impact the propagation of another person’s transaction, i.e., >>> an attacker can censor somebody’s transaction from ever being announced by >>> a node if they send enough transactions to fill up the rate limit. >>> Obviously this would be expensive since they're spending a lot on fees, but >>> I imagine it could be profitable in some situations to spend a few thousand >>> dollars to prevent anyone from hearing about a transaction for a few hours. >>> This might be a non-issue in practice if the rate limit is generous and >>> traffic isn’t horrendous, but is this a problem? >>> >>> And if we don't require an increase in (i.e. addition of "new") absolute >>> fees, users are essentially allowed to “recycle” fees. In the scenario >>> where we prioritize relay based on feerate, users could potentially be >>> placed higher in the queue, ahead of other users’ transactions, multiple >>> times, without ever adding more fees to the transaction. Again, maybe this >>> isn’t a huge deal in practice if we set the parameters right, but it seems… >>> not great, in principle. >>> >>> --------- >>> >>> It's probably also a good idea to point out that there's been some >>> discussion happening on the gist containing my original post on this thread >>> (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). >>> >>> Suhas and Matt [proposed][0] adding a policy rule allowing users to >>> specify descendant limits on their transactions. For example, some nth bit >>> of nSequence with nVersion 3 means "this transaction won't have more than X >>> vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It >>> solves the pinning problem with package RBF where the attacker's package >>> contains a very large and high-fee descendant. >>> >>> We could add this policy and deploy it with package RBF/package relay so >>> that LN can use it by setting the user-elected descendant limit flag on >>> commitment transactions. (Otherwise package RBF is blocked until we find a >>> more comprehensive solution to the pinning attack). >>> >>> It's simple to [implement][1] as a mempool policy, but adds some >>> complexity for wallets that use it, since it limits their use of UTXOs from >>> transactions with this bit set. >>> >>> --------- >>> >>> Also, coming back to the idea of "we can't just use {individual, >>> ancestor} feerate," I'm interested in soliciting feedback on adding a >>> “mining score” calculator. I've implemented one [here][2] which takes the >>> transaction in question, grabs all of the connected mempool transactions >>> (including siblings, coparents, etc., as they wouldn’t be in the ancestor >>> nor descendant sets), and builds a “block template” using our current >>> mining algorithm. The mining score of a transaction is the ancestor feerate >>> at which it is included. >>> >>> This would be helpful for something like ancestor-aware funding and >>> fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited >>> priority queue for transaction relay, we'd want to use something like this >>> as the priority value. And for RBF, we probably want to require that a >>> replacement have a higher mining score than the original transactions. This >>> could be computationally expensive to do all the time; it could be good to >>> cache it but that could make mempool bookkeeping more complicated. Also, if >>> we end up trying to switch to a candidate set-based algorithm for mining, >>> we'd of course need a new calculator. >>> >>> [0]: >>> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 >>> [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit >>> [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score >>> [3]: https://github.com/bitcoin/bitcoin/issues/9645 >>> [4]: https://github.com/bitcoin/bitcoin/issues/15553 >>> >>> Best, >>> Gloria >>> >>> On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: >>> >>>> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >>>> > @aj: >>>> > > I wonder sometimes if it could be sufficient to just have a relay >>>> rate >>>> > > limit and prioritise by ancestor feerate though. Maybe something >>>> like: >>>> > > - instead of adding txs to each peers setInventoryTxToSend >>>> immediately, >>>> > > set a mempool flag "relayed=false" >>>> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs >>>> to >>>> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >>>> > > calculate how much kB those txs were, and do this again after >>>> > > SIZE/RATELIMIT seconds >>>> >>>> > > - don't include "relayed=false" txs when building blocks? >>>> >>>> The "?" was me not being sure that point is a good suggestion... >>>> >>>> Miners might reasonably decide to have no rate limit, and always relay, >>>> and never exclude txs -- but the question then becomes is whether they >>>> hear about the tx at all, so rate limiting behaviour could still be a >>>> potential problem for whoever made the tx. >>>> >>>> > Wow cool! I think outbound tx relay size-based rate-limiting and >>>> > prioritizing tx relay by feerate are great ideas for preventing >>>> spammers >>>> > from wasting bandwidth network-wide. I agree, this would slow the low >>>> > feerate spam down, preventing a huge network-wide bandwidth spike. >>>> And it >>>> > would allow high feerate transactions to propagate as they should, >>>> > regardless of how busy traffic is. Combined with inbound tx request >>>> > rate-limiting, might this be sufficient to prevent DoS regardless of >>>> the >>>> > fee-based replacement policies? >>>> >>>> I think you only want to do outbound rate limits, ie, how often you send >>>> INV, GETDATA and TX messages? Once you receive any of those, I think >>>> you have to immediately process / ignore it, you can't really sensibly >>>> defer it (beyond the existing queues we have that just build up while >>>> we're busy processing other things first)? >>>> >>>> > One point that I'm not 100% clear on: is it ok to prioritize the >>>> > transactions by ancestor feerate in this scheme? As I described in the >>>> > original post, this can be quite different from the actual feerate we >>>> would >>>> > consider a transaction in a block for. The transaction could have a >>>> high >>>> > feerate sibling bumping its ancestor. >>>> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C >>>> (5sat/vB). If >>>> > we just received C, it would be incorrect to give it a priority equal >>>> to >>>> > its ancestor feerate (3sat/vB) because if we constructed a block >>>> template >>>> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >>>> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If >>>> we >>>> > also exclude C when building blocks, we're missing out on good fees. >>>> >>>> I think you're right that this would be ugly. It's something of a >>>> special case: >>>> >>>> a) you really care about C getting into the next block; but >>>> b) you're trusting B not being replaced by a higher fee tx that >>>> doesn't have A as a parent; and >>>> c) there's a lot of txs bidding the floor of the next block up to a >>>> level in-between the ancestor fee rate of 3sat/vB and the tx fee >>>> rate of 5sat/vB >>>> >>>> Without (a), maybe you don't care about it getting to a miner quickly. >>>> If your trust in (b) was misplaced, then your tx's effective fee rate >>>> will drop and (because of (c)), you'll lose anyway. And if the spam ends >>>> up outside of (c)'s range, either the rate limiting won't take effect >>>> (spam's too cheap) and you'll be fine, or you'll miss out on the block >>>> anyway (spam's paying more than your tx rate) and you never had any hope >>>> of making it in. >>>> >>>> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >>>> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >>>> per 10 minutes for outbound connections. This would be a weight based >>>> rate limit instead-of/in-addition-to that, I guess. >>>> >>>> As far as a non-ugly approach goes, I think you'd have to be smarter >>>> about >>>> tracking the "effective fee rate" than the ancestor fee rate manages; >>>> maybe that's something that could fall out of Murch and Clara's >>>> candidate >>>> set blockbuilding ideas [0] ? >>>> >>>> Perhaps that same work would also make it possible to come up with >>>> a better answer to "do I care that this replacement would invalidate >>>> these descendents?" >>>> >>>> [0] https://github.com/Xekyo/blockbuilding >>>> >>>> > > - keep high-feerate evicted txs around for a while in case they get >>>> > > mined by someone else to improve compact block relay, a la the >>>> > > orphan pool? >>>> > Replaced transactions are already added to vExtraTxnForCompact :D >>>> >>>> I guess I was thinking that it's just a 100 tx LRU cache, which might >>>> not be good enough? >>>> >>>> Maybe it would be more on point to have a rate limit apply only to >>>> replacement transactions? >>>> >>>> > For wallets, AJ's "All you need is for there to be *a* path that >>>> follows >>>> > the new relay rules and gets from your node/wallet to perhaps 10% of >>>> > hashpower" makes sense to me (which would be the former). >>>> >>>> Perhaps a corollarly of that is that it's *better* to have the mempool >>>> acceptance rule only consider economic incentives, and have the spam >>>> prevention only be about "shall I tell my peers about this?" >>>> >>>> If you don't have that split; then the anti-spam rules can prevent you >>>> from getting the tx in the mempool at all; whereas if you do have the >>>> split, then even if the bitcoind anti-spam rules are blocking you at >>>> every turn, you can still send your tx to miners by some other route, >>>> and then they can add it to their mempool directly without any hassle. >>>> >>>> Cheers, >>>> aj >>>> >>>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> [-- Attachment #2: Type: text/html, Size: 28407 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-14 10:29 ` Gloria Zhao @ 2022-03-15 1:43 ` Billy Tetrud 0 siblings, 0 replies; 25+ messages in thread From: Billy Tetrud @ 2022-03-15 1:43 UTC (permalink / raw) To: Gloria Zhao; +Cc: Bitcoin Protocol Discussion, Anthony Towns [-- Attachment #1: Type: text/plain, Size: 26281 bytes --] Hi Gloria, It seems you're responding to what I wrote on github. Happy to respond, but perhaps we should keep it there so the conversation is kept linearly together? I'm curious what you think of my thoughts on what you brought up most recently in this thread about rate limiting / staggered window dos protection stuff. Cheers, BT On Mon, Mar 14, 2022 at 5:29 AM Gloria Zhao <gloriajzhao@gmail.com> wrote: > Hi Billy, > > > We should expect miners will be using a more complex, more optimal way > of determining what blocks they're working on [...] we should instead run > with the assumption that miners keep all potentially relevant transactions > in their mempools, including potentially many conflicting transctions, in > order to create the most profitable blocks. And therefore we shouldn't put > the constraint on normal non-mining full nodes to do that same more-complex > mempool behavior or add any complexity for the purpose of denying > transaction replacements. > > > I think a lot of the complexity in these ideas is because of the attempt > to match relay rules with miner > inclusion rules. > > I think the assumption that miners are using a completely different > implementation of mempool and block template building is false. IIUC, most > miners use Bitcoin Core and perhaps configure their node differently (e.g. > larger mempool and different minfeerates), but also use `getblocktemplate` > which means the same ancestor package-based mining algorithm. > > Of course, I'm not a miner, so if anybody is a miner or has seen miners' > setups, please correct me if I'm wrong. > > In either case, we would want our mining algorithm to result in block > templates that are as close as possible to perfectly incentive compatibile. > > Fundamentally, I believe default mempool policy (which perhaps naturally > creates a network-wide transaction relay policy) should be as close to the > mining code as possible. Imagine node A only keeps 1 block's worth of > transactions, and node B keeps a (default) 300MB mempool. The contents of > node A's mempool should be as close as possible to a block template > generated from node B's mempool. Otherwise, node A's mempool is not very > useful - their fee estimation is flawed and compact block relay won't do > them much good if they need to re-request a lot of block transactions. > Next, imagine that node B is a miner. It would be very suboptimal if the > mining code was ancestor package-based (i.e. supports CPFP), but the > mempool policy only cared about individual feerate, and evicted low-fee > parents despite their high-fee children. It's easy to see why this would be > suboptimal. > Attempting to match mempool policy with the mining algorithm is also > arguably the point of package relay. Our mining code uses ancestor packages > which is good, but we only submit transactions one at a time to the > mempool, so a transaction's high-fee children can't be considered until > they are all already in the mempool. Package relay allows us to start > thinking about ancestor packages immediately when evaluating transactions > for submission to the mempool. > > The attempt to match policy with miner inclusion rules is deliberate and > necessary. > > > I want to echo James O'Beirne's opinion on this that this may be the > wrong path to go down (a path of more complexity without much gain). He > said: "Special consideration for "what should be in the next block" and/or > the caching of block templates seems like an imposing dependency, dragging > in a bunch of state and infrastructure to a question that should be solely > limited to mempool feerate aggregates and the feerate of the particular txn > package a wallet is concerned with." > > It seems that I under-explained the purpose of building/caching block > templates in my original post, since both you and James have the same > misunderstanding. Since RBF's introduction, we have improved to an ancestor > package-based mining algorithm. This supports CPFP (incentive compatible) > and it is now common to see more complex "families" of transactions as > users fee-bump transactions (market is working, yay). On the other hand, we > no longer have an accurate way of determining a transaction's "mining > score," i.e., the feerate of this transaction's ancestor package when it is > included in a block template using our current mining algorithm. > > This limitation is a big blocker in proposing new fee/feerate RBF rules. > For example, if we say "the transaction needs a better feerate," this is > obviously flawed, since the original transactions may have very > high-feerate children, and the replacement transaction may have low feerate > parents. So what we really want is "the transaction needs to be more > incentive compatible to mine based on our mining algorithm," but we have no > way of getting that information right now. > > In my original post, I [described 4 heuristics to get transaction's > "mining score"][1] using the current data cached in the mempool (e.g. max > ancestor feerate of descendant set), as well as why they don't work. As > such, the best way to calculate a transaction's mining score AFAICT is to > grab all of the related transactions and build a mini "block template" with > them. The [implementation][2] I sent last week also cuts out some of the > fluff, so the pseudocode looks like this: > > // Get ALL connected entries (ancestors, descendants, siblings, cousins, > coparents, etc.) > vector<TxMempoolEntry> cluster = mempool.GetAllTransactionsRelatedTo(txid); > sort(cluster, ancestorfeerate); > > // For deducting ancestors when they are included separately > vector<ModifiedTxMempoolEntry> modified_entries; > > while (!cluster.empty() and !modified_entries.empty()) { > iter = BetterAncestorFeerateOf(cluster, modified_entries); > best_ancestor_package = GetAncestorSet(iter); > mining_score = Feerate(best_ancestor_package); > for (entry : best_ancestor_package) { > mining_scores[entry] = mining_score; > for (descendant : GetAllDescendants(entry) { > > modified_entries[descendant].DeductAncestorFromAccounting(entry); > } > } > } > > Perhaps somebody will come up with a better way to do this, but this is my > solution, and I hope I've now sufficiently described why I've made an > effort to think about this. It's not because I want to make things more > fancy or complicated, but because we currently have no way of accurately > determining a transaction's mining score. The reason I proposed a > persistent block template is so we can efficiently query the mining score > of all transactions in the mempool. > > > However, I do think that these DOS concerns are quite overblown. I wrote > up a comment on your rbf-improvements.md > <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing > my thought process on that. The summary is that as long as the fee-rate > relay rule is maintained, any "spam" is actually paid for, either by the > "first" transaction in the spam chain, or by the "spam" itself. > > The DoS concern is not overblown. I recommend you re-read the [current RBF > policy][3]; Rule 3 and 4 *are* the feerate relay rules. Removing Rule 3 > means allowing recycled fees, so new transactions are not necessarily > "paying" anything. > > [1]: > https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#mining-score-of-a-mempool-transaction > [2]: https://github.com/glozow/bitcoin/tree/2022-02-mining-score > [3]: > https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md > > Best, > Gloria > > On Sat, Mar 12, 2022 at 8:18 AM Billy Tetrud <billy.tetrud@gmail.com> > wrote: > >> In reading through more of the discussion, it seems the idea I presented >> above might basically be a reformulation of t-bast's rate-limiting idea >> presented in this comment >> <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4081349#gistcomment-4081349>. >> Perhaps he could comment on whether that characterization is accurate. >> >> On Fri, Mar 11, 2022 at 10:22 AM Billy Tetrud <billy.tetrud@gmail.com> >> wrote: >> >>> Hi Gloria, >>> >>> > 1. Transaction relay rate limiting >>> >>> I have a similar concern as yours, that this could prevent higher >>> fee-rate transactions from being broadcast. >>> >>> > 2. Staggered broadcast of replacement transactions: within some time >>> interval, maybe accept multiple replacements for the same prevout, but only >>> relay the original transaction. >>> >>> By this do you mean basically having a batching window where, on >>> receiving a replacement transaction, a node will wait for a period of time, >>> potentially receiving many replacements for the same transaction (or many >>> separate conflicting transactions), and only broadcasting the "best" one(s) >>> at the end of that time window? >>> >>> Its an interesting idea, but it would produce a problem. Every hop that >>> replacement transaction takes would be delayed by this staggered window. If >>> the window were 3 minutes and transactions generally take 20 hops to get to >>> the majority of miners, a "best-case average" delay might be 3.75 minutes >>> (noting that among your 8 nodes, its quite likely one of them would have a >>> window ending much sooner than 3 minutes). Some (maybe 3% of) nodes would >>> experience delays of more than 20 minutes. That kind of delay isn't great. >>> >>> However it made me think of another idea: a transaction replacement >>> broadcast cooldown. What if nodes kept track of the time they broadcasted >>> the last replacement for a package and had a relay cooldown after the last >>> replacement was broadcasted? A node receiving a replacement would relay the >>> replacement immediately if the package its replacing was broadcasted more >>> than X seconds ago, and otherwise it would wait until the time when that >>> package was broadcasted at least X seconds ago to broadcast it. Any >>> replacements it receives during that waiting period would replace as >>> normal, meaning the unrebroadcasted replacement would never be >>> broadcasted, and only the highest value replacement would be broadcasted at >>> the end of the cooldown. >>> >>> This wouldn't prevent a higher-fee-rate transaction from being >>> broadcasted (like rate limiting could), but would still be effective at >>> limiting unnecessary data transmission. Another benefit is that in the >>> non-adversarial case, replacement transactions wouldn't be subject to any >>> delay at all (while in the staggered broadcast idea, most replacements >>> would experience some delay). And in the adversarial case, where a >>> malicious actor broadcasts a low-as-possible-value replacement just before >>> yours, the worst case delay is just whatever the cooldown period is. I >>> would imagine that maybe 1 minute would be a reasonable worst-case delay. >>> This would limit spam for a transaction that makes it into a block to ~10x >>> (9 to 1). I don't see much of a downside to doing this beyond just the >>> slight additional complexity of relay rules (and considering it could save >>> substantial additional code complexity, even that is a benefit). >>> >>> All a node would need to do is keep a timestamp on each transaction they >>> receive for when it was broadcasted and check it when a replacement comes >>> in. If now-broadcastDate < cooldown, set a timer for cooldown - >>> (now-broadcastDate) to broadcast it. If another replacement comes in, clear >>> that timer and repeat using the original broadcast date (since the >>> unbroadcast transaction doesn't have a broadcast date yet). >>> >>> I think it might also be useful to note that eliminating "extra data" >>> caused by careless or malicious actors (spam or whatever you want to call >>> it) should not be the goal. It is impossible to prevent all spam. What we >>> should be aiming for is more specific: we should attempt to design a system >>> where spam is manageable. Eg if our goal is to ensure that a bitcoin node >>> uses no more than 10% of the bandwidth of a "normal" user, if current >>> non-spam traffic only requires 1% of a "normal" users's bandwidth, then the >>> network can bear a 9 to 1 ratio of spam. When a node spins up, there is a >>> lot more data to download and process. So we know that all full nodes can >>> handle at least as much traffic as they handle during IBD. What's the >>> difference between those amounts? I'm not sure, but I would guess that IBD >>> is at least a couple times more demanding than a fully synced node. So I >>> might suggest that as long as spam can be kept below a ratio of maybe 2 to >>> 1, we should consider the design acceptable (and therefore more complexity >>> unnecessary). >>> >>> The 1 minute broadcast cooldown I mentioned before wouldn't be quite >>> sufficient to achieve that ratio. But a 3.33 minute cooldown would be. >>> Whether this is "too much" is something that would have to be discussed, I >>> suspect a worst-case adversarial 3.33 minute delay would not be "too much". >>> Doing this could basically eliminate any risk of actual service denial via >>> replacement transactions. >>> >>> However, I do think that these DOS concerns are quite overblown. I wrote >>> up a comment on your rbf-improvements.md >>> <https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4093100#gistcomment-4093100> detailing >>> my thought process on that. The summary is that as long as the fee-rate >>> relay rule is maintained, any "spam" is actually paid for, either by the >>> "first" transaction in the spam chain, or by the "spam" itself. Even >>> without something like a minimum RBF relay delay limiting how much spam >>> could be created, the economics of the fee-rate rule already sufficiently >>> mitigate the issue of spam. >>> On Wed, Mar 9, 2022 at 9:37 AM Gloria Zhao via bitcoin-dev < >>> bitcoin-dev@lists.linuxfoundation.org> wrote: >>> >>>> Hi RBF friends, >>>> >>>> Posting a summary of RBF discussions at coredev (mostly on transaction >>>> relay rate-limiting), user-elected descendant limit as a short term >>>> solution to unblock package RBF, and mining score, all open for feedback: >>>> >>>> One big concept discussed was baking DoS protection into the p2p level >>>> rather than policy level. TLDR: The fees are not paid to the node operator, >>>> but to the miner. While we can use fees to reason about the cost of an >>>> attack, if we're ultimately interested in preventing resource exhaustion, >>>> maybe we want to "stop the bleeding" when it happens and bound the amount >>>> of resources used in general. There were two main ideas: >>>> >>>> 1. Transaction relay rate limiting (i.e. the one you proposed above or >>>> some variation) with a feerate-based priority queue >>>> 2. Staggered broadcast of replacement transactions: within some time >>>> interval, maybe accept multiple replacements for the same prevout, but only >>>> relay the original transaction. >>>> >>>> Looking to solicit feedback on these ideas and the concept in general. >>>> Is it a good idea (separate from RBF) to add rate-limiting in transaction >>>> relay? And is it the right direction to think about RBF DoS protection this >>>> way? >>>> >>>> A lingering concern that I have about this idea is it would then be >>>> possible to impact the propagation of another person’s transaction, i.e., >>>> an attacker can censor somebody’s transaction from ever being announced by >>>> a node if they send enough transactions to fill up the rate limit. >>>> Obviously this would be expensive since they're spending a lot on fees, but >>>> I imagine it could be profitable in some situations to spend a few thousand >>>> dollars to prevent anyone from hearing about a transaction for a few hours. >>>> This might be a non-issue in practice if the rate limit is generous and >>>> traffic isn’t horrendous, but is this a problem? >>>> >>>> And if we don't require an increase in (i.e. addition of "new") >>>> absolute fees, users are essentially allowed to “recycle” fees. In the >>>> scenario where we prioritize relay based on feerate, users could >>>> potentially be placed higher in the queue, ahead of other users’ >>>> transactions, multiple times, without ever adding more fees to the >>>> transaction. Again, maybe this isn’t a huge deal in practice if we set the >>>> parameters right, but it seems… not great, in principle. >>>> >>>> --------- >>>> >>>> It's probably also a good idea to point out that there's been some >>>> discussion happening on the gist containing my original post on this thread >>>> (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). >>>> >>>> Suhas and Matt [proposed][0] adding a policy rule allowing users to >>>> specify descendant limits on their transactions. For example, some nth bit >>>> of nSequence with nVersion 3 means "this transaction won't have more than X >>>> vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It >>>> solves the pinning problem with package RBF where the attacker's package >>>> contains a very large and high-fee descendant. >>>> >>>> We could add this policy and deploy it with package RBF/package relay >>>> so that LN can use it by setting the user-elected descendant limit flag on >>>> commitment transactions. (Otherwise package RBF is blocked until we find a >>>> more comprehensive solution to the pinning attack). >>>> >>>> It's simple to [implement][1] as a mempool policy, but adds some >>>> complexity for wallets that use it, since it limits their use of UTXOs from >>>> transactions with this bit set. >>>> >>>> --------- >>>> >>>> Also, coming back to the idea of "we can't just use {individual, >>>> ancestor} feerate," I'm interested in soliciting feedback on adding a >>>> “mining score” calculator. I've implemented one [here][2] which takes the >>>> transaction in question, grabs all of the connected mempool transactions >>>> (including siblings, coparents, etc., as they wouldn’t be in the ancestor >>>> nor descendant sets), and builds a “block template” using our current >>>> mining algorithm. The mining score of a transaction is the ancestor feerate >>>> at which it is included. >>>> >>>> This would be helpful for something like ancestor-aware funding and >>>> fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited >>>> priority queue for transaction relay, we'd want to use something like this >>>> as the priority value. And for RBF, we probably want to require that a >>>> replacement have a higher mining score than the original transactions. This >>>> could be computationally expensive to do all the time; it could be good to >>>> cache it but that could make mempool bookkeeping more complicated. Also, if >>>> we end up trying to switch to a candidate set-based algorithm for mining, >>>> we'd of course need a new calculator. >>>> >>>> [0]: >>>> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 >>>> [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit >>>> [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score >>>> [3]: https://github.com/bitcoin/bitcoin/issues/9645 >>>> [4]: https://github.com/bitcoin/bitcoin/issues/15553 >>>> >>>> Best, >>>> Gloria >>>> >>>> On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: >>>> >>>>> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >>>>> > @aj: >>>>> > > I wonder sometimes if it could be sufficient to just have a relay >>>>> rate >>>>> > > limit and prioritise by ancestor feerate though. Maybe something >>>>> like: >>>>> > > - instead of adding txs to each peers setInventoryTxToSend >>>>> immediately, >>>>> > > set a mempool flag "relayed=false" >>>>> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs >>>>> to >>>>> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >>>>> > > calculate how much kB those txs were, and do this again after >>>>> > > SIZE/RATELIMIT seconds >>>>> >>>>> > > - don't include "relayed=false" txs when building blocks? >>>>> >>>>> The "?" was me not being sure that point is a good suggestion... >>>>> >>>>> Miners might reasonably decide to have no rate limit, and always relay, >>>>> and never exclude txs -- but the question then becomes is whether they >>>>> hear about the tx at all, so rate limiting behaviour could still be a >>>>> potential problem for whoever made the tx. >>>>> >>>>> > Wow cool! I think outbound tx relay size-based rate-limiting and >>>>> > prioritizing tx relay by feerate are great ideas for preventing >>>>> spammers >>>>> > from wasting bandwidth network-wide. I agree, this would slow the low >>>>> > feerate spam down, preventing a huge network-wide bandwidth spike. >>>>> And it >>>>> > would allow high feerate transactions to propagate as they should, >>>>> > regardless of how busy traffic is. Combined with inbound tx request >>>>> > rate-limiting, might this be sufficient to prevent DoS regardless of >>>>> the >>>>> > fee-based replacement policies? >>>>> >>>>> I think you only want to do outbound rate limits, ie, how often you >>>>> send >>>>> INV, GETDATA and TX messages? Once you receive any of those, I think >>>>> you have to immediately process / ignore it, you can't really sensibly >>>>> defer it (beyond the existing queues we have that just build up while >>>>> we're busy processing other things first)? >>>>> >>>>> > One point that I'm not 100% clear on: is it ok to prioritize the >>>>> > transactions by ancestor feerate in this scheme? As I described in >>>>> the >>>>> > original post, this can be quite different from the actual feerate >>>>> we would >>>>> > consider a transaction in a block for. The transaction could have a >>>>> high >>>>> > feerate sibling bumping its ancestor. >>>>> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C >>>>> (5sat/vB). If >>>>> > we just received C, it would be incorrect to give it a priority >>>>> equal to >>>>> > its ancestor feerate (3sat/vB) because if we constructed a block >>>>> template >>>>> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >>>>> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If >>>>> we >>>>> > also exclude C when building blocks, we're missing out on good fees. >>>>> >>>>> I think you're right that this would be ugly. It's something of a >>>>> special case: >>>>> >>>>> a) you really care about C getting into the next block; but >>>>> b) you're trusting B not being replaced by a higher fee tx that >>>>> doesn't have A as a parent; and >>>>> c) there's a lot of txs bidding the floor of the next block up to a >>>>> level in-between the ancestor fee rate of 3sat/vB and the tx fee >>>>> rate of 5sat/vB >>>>> >>>>> Without (a), maybe you don't care about it getting to a miner quickly. >>>>> If your trust in (b) was misplaced, then your tx's effective fee rate >>>>> will drop and (because of (c)), you'll lose anyway. And if the spam >>>>> ends >>>>> up outside of (c)'s range, either the rate limiting won't take effect >>>>> (spam's too cheap) and you'll be fine, or you'll miss out on the block >>>>> anyway (spam's paying more than your tx rate) and you never had any >>>>> hope >>>>> of making it in. >>>>> >>>>> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >>>>> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >>>>> per 10 minutes for outbound connections. This would be a weight based >>>>> rate limit instead-of/in-addition-to that, I guess. >>>>> >>>>> As far as a non-ugly approach goes, I think you'd have to be smarter >>>>> about >>>>> tracking the "effective fee rate" than the ancestor fee rate manages; >>>>> maybe that's something that could fall out of Murch and Clara's >>>>> candidate >>>>> set blockbuilding ideas [0] ? >>>>> >>>>> Perhaps that same work would also make it possible to come up with >>>>> a better answer to "do I care that this replacement would invalidate >>>>> these descendents?" >>>>> >>>>> [0] https://github.com/Xekyo/blockbuilding >>>>> >>>>> > > - keep high-feerate evicted txs around for a while in case they get >>>>> > > mined by someone else to improve compact block relay, a la the >>>>> > > orphan pool? >>>>> > Replaced transactions are already added to vExtraTxnForCompact :D >>>>> >>>>> I guess I was thinking that it's just a 100 tx LRU cache, which might >>>>> not be good enough? >>>>> >>>>> Maybe it would be more on point to have a rate limit apply only to >>>>> replacement transactions? >>>>> >>>>> > For wallets, AJ's "All you need is for there to be *a* path that >>>>> follows >>>>> > the new relay rules and gets from your node/wallet to perhaps 10% of >>>>> > hashpower" makes sense to me (which would be the former). >>>>> >>>>> Perhaps a corollarly of that is that it's *better* to have the mempool >>>>> acceptance rule only consider economic incentives, and have the spam >>>>> prevention only be about "shall I tell my peers about this?" >>>>> >>>>> If you don't have that split; then the anti-spam rules can prevent you >>>>> from getting the tx in the mempool at all; whereas if you do have the >>>>> split, then even if the bitcoind anti-spam rules are blocking you at >>>>> every turn, you can still send your tx to miners by some other route, >>>>> and then they can add it to their mempool directly without any hassle. >>>>> >>>>> Cheers, >>>>> aj >>>>> >>>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>> [-- Attachment #2: Type: text/html, Size: 29295 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-09 15:09 ` Gloria Zhao 2022-03-11 16:22 ` Billy Tetrud @ 2022-03-17 2:02 ` Antoine Riard 2022-03-17 15:59 ` Billy Tetrud 1 sibling, 1 reply; 25+ messages in thread From: Antoine Riard @ 2022-03-17 2:02 UTC (permalink / raw) To: Gloria Zhao, Bitcoin Protocol Discussion; +Cc: Anthony Towns [-- Attachment #1: Type: text/plain, Size: 19666 bytes --] Hi Mempoololic Anonymous fellow, > 2. Staggered broadcast of replacement transactions: within some time > interval, maybe accept multiple replacements for the same prevout, but only > relay the original transaction. If the goal of replacement staggering is to save on bandwidth, I'm not sure it's going to be effective if you consider replacement done from a shared-utxo. E.g, Alice broadcasts a package to confirm her commitment, relay is staggered until T. At the same time, Bob broadcasts a package to confirm his version of the commitment at a slightly better feerate, relay is staggered until T. At T, package A gradually floods from Alice's peers and package B does the same from Bob's peers. When there is an intersection. B overrides A and starts to replace package A in the network mempools nearest to Alice. I think those peers won't have bandwidth saving from adopting a replacement staggering strategy. Or maybe that's something completely different if you have in mind ? I think it's worth more staggering detail to guess if it's robust against all the replacement propagations patterns. Though if we aim to save on replacement bandwidth I wonder if a "diff-only" strategy, assuming some new p2p mechanism, would be more interesting (as discussed in the recent "Thoughts on fee bumping thread"). > A lingering concern that I have about this idea is it would then be > possible to impact the propagation of another person’s transaction, i.e., > an attacker can censor somebody’s transaction from ever being announced by > a node if they send enough transactions to fill up the rate limit. > Obviously this would be expensive since they're spending a lot on fees, but > I imagine it could be profitable in some situations to spend a few thousand > dollars to prevent anyone from hearing about a transaction for a few hours. > This might be a non-issue in practice if the rate limit is generous and > traffic isn’t horrendous, but is this a problem? I think I share the concern too about an attacker exhausting a node transaction relay ressources to prevent another person's transaction to propagate, especially if the transaction targeted is a L2's time-sensitive one. In that latter context, an attacker would aim to delay the relay of a time-sensitive transaction (e.g a HTLC-success) to the miners, until the timelock expires. The malicious delay period should swallow the go-to-chain HTLC deadline ("the deadline for received HTLCs this node fulfilled" in bolt 2 parlance), in that current example 18 blocks. Let's say we allocate 10 MB of bandwidth per-block period. Once the 10 MB are exhausted, there is no more bandwidth allocated until the next block is issued. If the top mempool feerate is 1 sat/vb, such naive design would allow an attacker to buy all the p2p network bandwidth period for 0.1 BTC. If an attacker aims to jam a HTLC transaction for the 18 blocks period, the cost is of 1,8 BTC. If the attacker is a LN counterparty to a HTLC worth more than 1.8 BTC, the attack sounds economically profitable. Worst, the p2p network bandwidth is a public resource while a HTLC is a private, off-chain contract. An attacker could be counterparty to many HTLCs, where each HTLC individual value is far inferior to the global p2p bandwidth cost but the sum, only known to the attacker, is superior to. Therefore, it sounds to me that p2p network bandwidth might be attractive if the stealing are batched. Is the attacker scenario described credible ? Are the numbers sketched out realistic ? If yes, I think one design insight for eventual transaction relay rate limiting would be to make them "dynamic", and not naively fixed for a period. By making them dynamic, an attacker would have to compete with the effective feerate proposed by the victim transaction. E.g, if the HTLC-success feerate is of 10 sat/vb, an attacker would have to propose a stream of malicious transaction of more than 10 sat/vb during the whole HTLC deadline period for the transaction-relay jamming to be effective. Further, the attack might be invisible from the victim standpoint, the malicious flow of feerate competitive transactions can be hard to dissociate from an honest one. Thus, you can expect the HTLC transaction issuer to only slowly increase the feerate at each block, and those moves to be anticipated by the attacker. Even if the transaction issuer adopts a scorched-earth approach for the latest blocks of the deadline, the absolute value of the HTLC burnt in fees might still be less than the transaction relay bandwidth exhaustion paid by the attacker because the attack is batched by the attacker. I'm not sure if this reasoning is correct. Though if yes, the issue sounds really similar to "flood&loot" attack affecting LN previously researched on [0]. What worries me more with this "exhaust&loot" is that if we introduce bounded transaction relay rate limiting, it sounds a cheaper public ressource to buy than the mempool.. [0] https://arxiv.org/pdf/2006.08513.pdf Anyway, I would say it's worthy to investigate more transaction relay rate limiting designs and especially carefully weigh the implications for L2s. Those ones might have to adjust their fee-bumping and transaction rebroadcast strategies in consequence. > Suhas and Matt [proposed][0] adding a policy rule allowing users to specify > descendant limits on their transactions. For example, some nth bit of > nSequence with nVersion 3 means "this transaction won't have more than X > vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It > solves the pinning problem with package RBF where the attacker's package > contains a very large and high-fee descendant. Hey, what if the pinning transaction has a parent with a junk feerate ? Let's say you have commitment tx for a HTLC of value 500000 sats, with top mempool feerate of 50 sat/vbyte. The commitment tx is pinned by a malicious tx of size 1000 vbytes, matching top mempool feerate. This malicious tx has a second unconfirmed parent (in addition to the commitment) of size MAX_STANDARD_TX_WEIGHT offering a 1 sat/vb. I think the pinning transaction ancestor score would be less than 2 sat/vb and thus considered irrelevant for block template inclusion ? At the same time, as the pinning transaction is attached with a top mempool feerate, the honest user wouldn't be able to replace it with a better-feerate proposal ? Unless adopting a scorched-earth approach, although economically I don't think this fee-bumping strategy is safe in case of batch-pinning. It might be fixable if we make one additional requirement "The child transaction subject to the user-elected descendant limit must have only one unconfirmed parent" (here the commitment transaction) ? Though I'm not even sure of the robustness of this fix. The commitment transaction itself could be used as a junk parent to downgrade the pinning transaction ancestor score. E.g, using a revoked commitment transaction with `max_accepted_htlcs` on both sides, pre-signed with a feerate of 1 sat/vb. We might restrict the maximum number of pending HTLCs network-wise to make the worst commitment transaction size reasonable, though not sure if my LN colleagues are going to like the idea.. Is that reasoning correct and conform to our Ancestor Set Based algorithm approach ? Maybe more details are needed. > Also, coming back to the idea of "we can't just use {individual, ancestor} > feerate," I'm interested in soliciting feedback on adding a “mining score” > calculator. I've implemented one [here][2] which takes the transaction in > question, grabs all of the connected mempool transactions (including > siblings, coparents, etc., as they wouldn’t be in the ancestor nor > descendant sets), and builds a “block template” using our current mining > algorithm. The mining score of a transaction is the ancestor feerate at > which it is included. I don't have a strong opinion there yet, though if we make this "block template" construction the default one, I would be really conservative to avoid malicious child attachment on multi-party transactions downgrading the block inclusion efficiency. Antoine Le mer. 9 mars 2022 à 10:37, Gloria Zhao via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> a écrit : > Hi RBF friends, > > Posting a summary of RBF discussions at coredev (mostly on transaction > relay rate-limiting), user-elected descendant limit as a short term > solution to unblock package RBF, and mining score, all open for feedback: > > One big concept discussed was baking DoS protection into the p2p level > rather than policy level. TLDR: The fees are not paid to the node operator, > but to the miner. While we can use fees to reason about the cost of an > attack, if we're ultimately interested in preventing resource exhaustion, > maybe we want to "stop the bleeding" when it happens and bound the amount > of resources used in general. There were two main ideas: > > 1. Transaction relay rate limiting (i.e. the one you proposed above or > some variation) with a feerate-based priority queue > 2. Staggered broadcast of replacement transactions: within some time > interval, maybe accept multiple replacements for the same prevout, but only > relay the original transaction. > > Looking to solicit feedback on these ideas and the concept in general. Is > it a good idea (separate from RBF) to add rate-limiting in transaction > relay? And is it the right direction to think about RBF DoS protection this > way? > > A lingering concern that I have about this idea is it would then be > possible to impact the propagation of another person’s transaction, i.e., > an attacker can censor somebody’s transaction from ever being announced by > a node if they send enough transactions to fill up the rate limit. > Obviously this would be expensive since they're spending a lot on fees, but > I imagine it could be profitable in some situations to spend a few thousand > dollars to prevent anyone from hearing about a transaction for a few hours. > This might be a non-issue in practice if the rate limit is generous and > traffic isn’t horrendous, but is this a problem? > > And if we don't require an increase in (i.e. addition of "new") absolute > fees, users are essentially allowed to “recycle” fees. In the scenario > where we prioritize relay based on feerate, users could potentially be > placed higher in the queue, ahead of other users’ transactions, multiple > times, without ever adding more fees to the transaction. Again, maybe this > isn’t a huge deal in practice if we set the parameters right, but it seems… > not great, in principle. > > --------- > > It's probably also a good idea to point out that there's been some > discussion happening on the gist containing my original post on this thread > (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). > > Suhas and Matt [proposed][0] adding a policy rule allowing users to > specify descendant limits on their transactions. For example, some nth bit > of nSequence with nVersion 3 means "this transaction won't have more than X > vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It > solves the pinning problem with package RBF where the attacker's package > contains a very large and high-fee descendant. > > We could add this policy and deploy it with package RBF/package relay so > that LN can use it by setting the user-elected descendant limit flag on > commitment transactions. (Otherwise package RBF is blocked until we find a > more comprehensive solution to the pinning attack). > > It's simple to [implement][1] as a mempool policy, but adds some > complexity for wallets that use it, since it limits their use of UTXOs from > transactions with this bit set. > > --------- > > Also, coming back to the idea of "we can't just use {individual, ancestor} > feerate," I'm interested in soliciting feedback on adding a “mining score” > calculator. I've implemented one [here][2] which takes the transaction in > question, grabs all of the connected mempool transactions (including > siblings, coparents, etc., as they wouldn’t be in the ancestor nor > descendant sets), and builds a “block template” using our current mining > algorithm. The mining score of a transaction is the ancestor feerate at > which it is included. > > This would be helpful for something like ancestor-aware funding and > fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited > priority queue for transaction relay, we'd want to use something like this > as the priority value. And for RBF, we probably want to require that a > replacement have a higher mining score than the original transactions. This > could be computationally expensive to do all the time; it could be good to > cache it but that could make mempool bookkeeping more complicated. Also, if > we end up trying to switch to a candidate set-based algorithm for mining, > we'd of course need a new calculator. > > [0]: > https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 > [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit > [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score > [3]: https://github.com/bitcoin/bitcoin/issues/9645 > [4]: https://github.com/bitcoin/bitcoin/issues/15553 > > Best, > Gloria > > On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: > >> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >> > @aj: >> > > I wonder sometimes if it could be sufficient to just have a relay rate >> > > limit and prioritise by ancestor feerate though. Maybe something like: >> > > - instead of adding txs to each peers setInventoryTxToSend >> immediately, >> > > set a mempool flag "relayed=false" >> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to >> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >> > > calculate how much kB those txs were, and do this again after >> > > SIZE/RATELIMIT seconds >> >> > > - don't include "relayed=false" txs when building blocks? >> >> The "?" was me not being sure that point is a good suggestion... >> >> Miners might reasonably decide to have no rate limit, and always relay, >> and never exclude txs -- but the question then becomes is whether they >> hear about the tx at all, so rate limiting behaviour could still be a >> potential problem for whoever made the tx. >> >> > Wow cool! I think outbound tx relay size-based rate-limiting and >> > prioritizing tx relay by feerate are great ideas for preventing spammers >> > from wasting bandwidth network-wide. I agree, this would slow the low >> > feerate spam down, preventing a huge network-wide bandwidth spike. And >> it >> > would allow high feerate transactions to propagate as they should, >> > regardless of how busy traffic is. Combined with inbound tx request >> > rate-limiting, might this be sufficient to prevent DoS regardless of the >> > fee-based replacement policies? >> >> I think you only want to do outbound rate limits, ie, how often you send >> INV, GETDATA and TX messages? Once you receive any of those, I think >> you have to immediately process / ignore it, you can't really sensibly >> defer it (beyond the existing queues we have that just build up while >> we're busy processing other things first)? >> >> > One point that I'm not 100% clear on: is it ok to prioritize the >> > transactions by ancestor feerate in this scheme? As I described in the >> > original post, this can be quite different from the actual feerate we >> would >> > consider a transaction in a block for. The transaction could have a high >> > feerate sibling bumping its ancestor. >> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). >> If >> > we just received C, it would be incorrect to give it a priority equal to >> > its ancestor feerate (3sat/vB) because if we constructed a block >> template >> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we >> > also exclude C when building blocks, we're missing out on good fees. >> >> I think you're right that this would be ugly. It's something of a >> special case: >> >> a) you really care about C getting into the next block; but >> b) you're trusting B not being replaced by a higher fee tx that >> doesn't have A as a parent; and >> c) there's a lot of txs bidding the floor of the next block up to a >> level in-between the ancestor fee rate of 3sat/vB and the tx fee >> rate of 5sat/vB >> >> Without (a), maybe you don't care about it getting to a miner quickly. >> If your trust in (b) was misplaced, then your tx's effective fee rate >> will drop and (because of (c)), you'll lose anyway. And if the spam ends >> up outside of (c)'s range, either the rate limiting won't take effect >> (spam's too cheap) and you'll be fine, or you'll miss out on the block >> anyway (spam's paying more than your tx rate) and you never had any hope >> of making it in. >> >> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >> per 10 minutes for outbound connections. This would be a weight based >> rate limit instead-of/in-addition-to that, I guess. >> >> As far as a non-ugly approach goes, I think you'd have to be smarter about >> tracking the "effective fee rate" than the ancestor fee rate manages; >> maybe that's something that could fall out of Murch and Clara's candidate >> set blockbuilding ideas [0] ? >> >> Perhaps that same work would also make it possible to come up with >> a better answer to "do I care that this replacement would invalidate >> these descendents?" >> >> [0] https://github.com/Xekyo/blockbuilding >> >> > > - keep high-feerate evicted txs around for a while in case they get >> > > mined by someone else to improve compact block relay, a la the >> > > orphan pool? >> > Replaced transactions are already added to vExtraTxnForCompact :D >> >> I guess I was thinking that it's just a 100 tx LRU cache, which might >> not be good enough? >> >> Maybe it would be more on point to have a rate limit apply only to >> replacement transactions? >> >> > For wallets, AJ's "All you need is for there to be *a* path that follows >> > the new relay rules and gets from your node/wallet to perhaps 10% of >> > hashpower" makes sense to me (which would be the former). >> >> Perhaps a corollarly of that is that it's *better* to have the mempool >> acceptance rule only consider economic incentives, and have the spam >> prevention only be about "shall I tell my peers about this?" >> >> If you don't have that split; then the anti-spam rules can prevent you >> from getting the tx in the mempool at all; whereas if you do have the >> split, then even if the bitcoind anti-spam rules are blocking you at >> every turn, you can still send your tx to miners by some other route, >> and then they can add it to their mempool directly without any hassle. >> >> Cheers, >> aj >> >> _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 22174 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [bitcoin-dev] Improving RBF Policy 2022-03-17 2:02 ` Antoine Riard @ 2022-03-17 15:59 ` Billy Tetrud 0 siblings, 0 replies; 25+ messages in thread From: Billy Tetrud @ 2022-03-17 15:59 UTC (permalink / raw) To: Antoine Riard, Bitcoin Protocol Discussion; +Cc: Anthony Towns [-- Attachment #1: Type: text/plain, Size: 22909 bytes --] @Antoine > B overrides A and starts to replace package A in the network mempools nearest to Alice. I think those peers won't have bandwidth saving from adopting a replacement staggering strategy. That's an interesting point, but even with that fact, the method would be effective at limiting spam. While yes, considering just a single unit of potential spam, only the nodes that didn't relay the spam in the first place would save bandwidth. However, the point is not to prevent a single unit, but to prevent thousands of units of spam. Even if in the situation you describe Bob and Alice sent 100 replacement transaction per seconds, it would lead to only 1 transaction sent by Bob's peers and only 2 transactions sent by Alice's peers (within a given stagger/cooldown window). That seems pretty effective to me. > I wonder if a "diff-only" strategy.. would be more interesting I think a diff-only relay strategy is definitely interesting. But its completely orthogonal. A diff only strategy isn't really a spam reduction mechanism, but rather a relay optimization that reduces bandwidth on all relay. Both can be done - seems like it could be argued that both should be done. >> For example, some nth bit of nSequence with nVersion 3 means "this transaction won't have more than X vbytes of descendants" >what if the pinning transaction has a parent with a junk feerate ? I think you're right that this scheme would also be susceptible to pinning. One thing I've identified as pretty much always suboptimal in any kind of policy is cliffs. Hard cut offs very often cause problems. You see this in welfare cliffs where the cliff disincentivizes people from earning more money for example. Its almost always better to build in a smooth gradient. Rate limiting where a node would relay replacement transaction data up to a certain point and then stop is a cliff like this. The descendant byte limit is a cliff like this. If such things were to be actually done, I'd recommend building in some kind of gradient where, say, every X vbytes of descendents requires Y additional feerate, or something to that effect. That way you can always add more descendents as long as you're willing to pay a higher and higher feerate for them. However, I think simply removing the absolute feerate rule is a better solution to that kind of RBF pinning. On Thu, Mar 17, 2022 at 4:32 AM Antoine Riard via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi Mempoololic Anonymous fellow, > > > 2. Staggered broadcast of replacement transactions: within some time > > interval, maybe accept multiple replacements for the same prevout, but > only > > relay the original transaction. > > If the goal of replacement staggering is to save on bandwidth, I'm not > sure it's going to be effective if you consider replacement done from a > shared-utxo. E.g, Alice broadcasts a package to confirm her commitment, > relay is staggered until T. At the same time, Bob broadcasts a package to > confirm his version of the commitment at a slightly better feerate, relay > is staggered until T. > > At T, package A gradually floods from Alice's peers and package B does the > same from Bob's peers. When there is an intersection. B overrides A and > starts to replace package A in the network mempools nearest to Alice. I > think those peers won't have bandwidth saving from adopting a replacement > staggering strategy. > > Or maybe that's something completely different if you have in mind ? I > think it's worth more staggering detail to guess if it's robust against all > the replacement propagations patterns. > > Though if we aim to save on replacement bandwidth I wonder if a > "diff-only" strategy, assuming some new p2p mechanism, would be more > interesting (as discussed in the recent "Thoughts on fee bumping thread"). > > > A lingering concern that I have about this idea is it would then be > > possible to impact the propagation of another person’s transaction, i.e., > > an attacker can censor somebody’s transaction from ever being announced > by > > a node if they send enough transactions to fill up the rate limit. > > Obviously this would be expensive since they're spending a lot on fees, > but > > I imagine it could be profitable in some situations to spend a few > thousand > > dollars to prevent anyone from hearing about a transaction for a few > hours. > > This might be a non-issue in practice if the rate limit is generous and > > traffic isn’t horrendous, but is this a problem? > > I think I share the concern too about an attacker exhausting a node > transaction relay ressources to prevent another person's transaction to > propagate, especially if the transaction targeted is a L2's time-sensitive > one. In that latter context, an attacker would aim to delay the relay of a > time-sensitive transaction (e.g a HTLC-success) to the miners, until the > timelock expires. The malicious delay period should swallow the go-to-chain > HTLC deadline ("the deadline for received HTLCs this node fulfilled" in > bolt 2 parlance), in that current example 18 blocks. > > Let's say we allocate 10 MB of bandwidth per-block period. Once the 10 MB > are exhausted, there is no more bandwidth allocated until the next block is > issued. If the top mempool feerate is 1 sat/vb, such naive design would > allow an attacker to buy all the p2p network bandwidth period for 0.1 BTC. > If an attacker aims to jam a HTLC transaction for the 18 blocks period, the > cost is of 1,8 BTC. If the attacker is a LN counterparty to a HTLC worth > more than 1.8 BTC, the attack sounds economically profitable. > > Worst, the p2p network bandwidth is a public resource while a HTLC is a > private, off-chain contract. An attacker could be counterparty to many > HTLCs, where each HTLC individual value is far inferior to the global p2p > bandwidth cost but the sum, only known to the attacker, is superior to. > Therefore, it sounds to me that p2p network bandwidth might be attractive > if the stealing are batched. > > Is the attacker scenario described credible ? Are the numbers sketched out > realistic ? > > If yes, I think one design insight for eventual transaction relay rate > limiting would be to make them "dynamic", and not naively fixed for a > period. By making them dynamic, an attacker would have to compete with the > effective feerate proposed by the victim transaction. E.g, if the > HTLC-success feerate is of 10 sat/vb, an attacker would have to propose a > stream of malicious transaction of more than 10 sat/vb during the whole > HTLC deadline period for the transaction-relay jamming to be effective. > > Further, the attack might be invisible from the victim standpoint, the > malicious flow of feerate competitive transactions can be hard to > dissociate from an honest one. Thus, you can expect the > HTLC transaction issuer to only slowly increase the feerate at each block, > and those moves to be anticipated by the attacker. Even if the transaction > issuer adopts a scorched-earth approach for the latest blocks of the > deadline, the absolute value of the HTLC burnt in fees might still be less > than the transaction relay bandwidth exhaustion paid by the attacker > because the attack is batched by the attacker. > > I'm not sure if this reasoning is correct. Though if yes, the issue sounds > really similar to "flood&loot" attack affecting LN previously researched on > [0]. What worries me more with this "exhaust&loot" is that if we introduce > bounded transaction relay rate limiting, it sounds a cheaper public > ressource to buy than the mempool.. > > [0] https://arxiv.org/pdf/2006.08513.pdf > > Anyway, I would say it's worthy to investigate more transaction relay rate > limiting designs and especially carefully weigh the implications for L2s. > Those ones might have to adjust their fee-bumping and transaction > rebroadcast strategies in consequence. > > > Suhas and Matt [proposed][0] adding a policy rule allowing users to > specify > > descendant limits on their transactions. For example, some nth bit of > > nSequence with nVersion 3 means "this transaction won't have more than X > > vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It > > solves the pinning problem with package RBF where the attacker's package > > contains a very large and high-fee descendant. > > Hey, what if the pinning transaction has a parent with a junk feerate ? > > Let's say you have commitment tx for a HTLC of value 500000 sats, with top > mempool feerate of 50 sat/vbyte. The commitment tx is pinned by a malicious > tx of size 1000 vbytes, matching top mempool feerate. This malicious tx has > a second unconfirmed parent (in addition to the commitment) of size > MAX_STANDARD_TX_WEIGHT offering a 1 sat/vb. I think the pinning transaction > ancestor score would be less than 2 sat/vb and thus considered irrelevant > for block template inclusion ? At the same time, as the pinning transaction > is attached with a top mempool feerate, the honest user wouldn't be able to > replace it with a better-feerate proposal ? Unless adopting a > scorched-earth approach, although economically I don't think this > fee-bumping strategy is safe in case of batch-pinning. > > It might be fixable if we make one additional requirement "The child > transaction subject to the user-elected descendant limit must have only one > unconfirmed parent" (here the commitment > transaction) ? Though I'm not even sure of the robustness of this fix. The > commitment transaction itself could be used as a junk parent to downgrade > the pinning transaction ancestor score. E.g, using a revoked commitment > transaction with `max_accepted_htlcs` on both sides, pre-signed with a > feerate of 1 sat/vb. We might restrict the maximum number of pending HTLCs > network-wise to make the worst commitment transaction size reasonable, > though not sure if my LN colleagues are going to like the idea.. > > Is that reasoning correct and conform to our Ancestor Set Based algorithm > approach ? Maybe more details are needed. > > > Also, coming back to the idea of "we can't just use {individual, > ancestor} > > feerate," I'm interested in soliciting feedback on adding a “mining > score” > > calculator. I've implemented one [here][2] which takes the transaction in > > question, grabs all of the connected mempool transactions (including > > siblings, coparents, etc., as they wouldn’t be in the ancestor nor > > descendant sets), and builds a “block template” using our current mining > > algorithm. The mining score of a transaction is the ancestor feerate at > > which it is included. > > I don't have a strong opinion there yet, though if we make this "block > template" construction the default one, I would be really conservative to > avoid malicious child attachment on multi-party transactions downgrading > the block inclusion efficiency. > > Antoine > > Le mer. 9 mars 2022 à 10:37, Gloria Zhao via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> a écrit : > >> Hi RBF friends, >> >> Posting a summary of RBF discussions at coredev (mostly on transaction >> relay rate-limiting), user-elected descendant limit as a short term >> solution to unblock package RBF, and mining score, all open for feedback: >> >> One big concept discussed was baking DoS protection into the p2p level >> rather than policy level. TLDR: The fees are not paid to the node operator, >> but to the miner. While we can use fees to reason about the cost of an >> attack, if we're ultimately interested in preventing resource exhaustion, >> maybe we want to "stop the bleeding" when it happens and bound the amount >> of resources used in general. There were two main ideas: >> >> 1. Transaction relay rate limiting (i.e. the one you proposed above or >> some variation) with a feerate-based priority queue >> 2. Staggered broadcast of replacement transactions: within some time >> interval, maybe accept multiple replacements for the same prevout, but only >> relay the original transaction. >> >> Looking to solicit feedback on these ideas and the concept in general. Is >> it a good idea (separate from RBF) to add rate-limiting in transaction >> relay? And is it the right direction to think about RBF DoS protection this >> way? >> >> A lingering concern that I have about this idea is it would then be >> possible to impact the propagation of another person’s transaction, i.e., >> an attacker can censor somebody’s transaction from ever being announced by >> a node if they send enough transactions to fill up the rate limit. >> Obviously this would be expensive since they're spending a lot on fees, but >> I imagine it could be profitable in some situations to spend a few thousand >> dollars to prevent anyone from hearing about a transaction for a few hours. >> This might be a non-issue in practice if the rate limit is generous and >> traffic isn’t horrendous, but is this a problem? >> >> And if we don't require an increase in (i.e. addition of "new") absolute >> fees, users are essentially allowed to “recycle” fees. In the scenario >> where we prioritize relay based on feerate, users could potentially be >> placed higher in the queue, ahead of other users’ transactions, multiple >> times, without ever adding more fees to the transaction. Again, maybe this >> isn’t a huge deal in practice if we set the parameters right, but it seems… >> not great, in principle. >> >> --------- >> >> It's probably also a good idea to point out that there's been some >> discussion happening on the gist containing my original post on this thread >> (https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff). >> >> Suhas and Matt [proposed][0] adding a policy rule allowing users to >> specify descendant limits on their transactions. For example, some nth bit >> of nSequence with nVersion 3 means "this transaction won't have more than X >> vbytes of descendants" where X = max(1000, vsizeof(tx)) or something. It >> solves the pinning problem with package RBF where the attacker's package >> contains a very large and high-fee descendant. >> >> We could add this policy and deploy it with package RBF/package relay so >> that LN can use it by setting the user-elected descendant limit flag on >> commitment transactions. (Otherwise package RBF is blocked until we find a >> more comprehensive solution to the pinning attack). >> >> It's simple to [implement][1] as a mempool policy, but adds some >> complexity for wallets that use it, since it limits their use of UTXOs from >> transactions with this bit set. >> >> --------- >> >> Also, coming back to the idea of "we can't just use {individual, >> ancestor} feerate," I'm interested in soliciting feedback on adding a >> “mining score” calculator. I've implemented one [here][2] which takes the >> transaction in question, grabs all of the connected mempool transactions >> (including siblings, coparents, etc., as they wouldn’t be in the ancestor >> nor descendant sets), and builds a “block template” using our current >> mining algorithm. The mining score of a transaction is the ancestor feerate >> at which it is included. >> >> This would be helpful for something like ancestor-aware funding and >> fee-bumping in the wallet: [3], [4]. I think if we did the rate-limited >> priority queue for transaction relay, we'd want to use something like this >> as the priority value. And for RBF, we probably want to require that a >> replacement have a higher mining score than the original transactions. This >> could be computationally expensive to do all the time; it could be good to >> cache it but that could make mempool bookkeeping more complicated. Also, if >> we end up trying to switch to a candidate set-based algorithm for mining, >> we'd of course need a new calculator. >> >> [0]: >> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff?permalink_comment_id=4058140#gistcomment-4058140 >> [1]: https://github.com/glozow/bitcoin/tree/2022-02-user-desclimit >> [2] https://github.com/glozow/bitcoin/tree/2022-02-mining-score >> [3]: https://github.com/bitcoin/bitcoin/issues/9645 >> [4]: https://github.com/bitcoin/bitcoin/issues/15553 >> >> Best, >> Gloria >> >> On Tue, Feb 8, 2022 at 4:58 AM Anthony Towns <aj@erisian.com.au> wrote: >> >>> On Mon, Feb 07, 2022 at 11:16:26AM +0000, Gloria Zhao wrote: >>> > @aj: >>> > > I wonder sometimes if it could be sufficient to just have a relay >>> rate >>> > > limit and prioritise by ancestor feerate though. Maybe something >>> like: >>> > > - instead of adding txs to each peers setInventoryTxToSend >>> immediately, >>> > > set a mempool flag "relayed=false" >>> > > - on a time delay, add the top N (by fee rate) "relayed=false" txs to >>> > > each peer's setInventoryTxToSend and mark them as "relayed=true"; >>> > > calculate how much kB those txs were, and do this again after >>> > > SIZE/RATELIMIT seconds >>> >>> > > - don't include "relayed=false" txs when building blocks? >>> >>> The "?" was me not being sure that point is a good suggestion... >>> >>> Miners might reasonably decide to have no rate limit, and always relay, >>> and never exclude txs -- but the question then becomes is whether they >>> hear about the tx at all, so rate limiting behaviour could still be a >>> potential problem for whoever made the tx. >>> >>> > Wow cool! I think outbound tx relay size-based rate-limiting and >>> > prioritizing tx relay by feerate are great ideas for preventing >>> spammers >>> > from wasting bandwidth network-wide. I agree, this would slow the low >>> > feerate spam down, preventing a huge network-wide bandwidth spike. And >>> it >>> > would allow high feerate transactions to propagate as they should, >>> > regardless of how busy traffic is. Combined with inbound tx request >>> > rate-limiting, might this be sufficient to prevent DoS regardless of >>> the >>> > fee-based replacement policies? >>> >>> I think you only want to do outbound rate limits, ie, how often you send >>> INV, GETDATA and TX messages? Once you receive any of those, I think >>> you have to immediately process / ignore it, you can't really sensibly >>> defer it (beyond the existing queues we have that just build up while >>> we're busy processing other things first)? >>> >>> > One point that I'm not 100% clear on: is it ok to prioritize the >>> > transactions by ancestor feerate in this scheme? As I described in the >>> > original post, this can be quite different from the actual feerate we >>> would >>> > consider a transaction in a block for. The transaction could have a >>> high >>> > feerate sibling bumping its ancestor. >>> > For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). >>> If >>> > we just received C, it would be incorrect to give it a priority equal >>> to >>> > its ancestor feerate (3sat/vB) because if we constructed a block >>> template >>> > now, B would bump A, and C's new ancestor feerate is 5sat/vB. >>> > Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we >>> > also exclude C when building blocks, we're missing out on good fees. >>> >>> I think you're right that this would be ugly. It's something of a >>> special case: >>> >>> a) you really care about C getting into the next block; but >>> b) you're trusting B not being replaced by a higher fee tx that >>> doesn't have A as a parent; and >>> c) there's a lot of txs bidding the floor of the next block up to a >>> level in-between the ancestor fee rate of 3sat/vB and the tx fee >>> rate of 5sat/vB >>> >>> Without (a), maybe you don't care about it getting to a miner quickly. >>> If your trust in (b) was misplaced, then your tx's effective fee rate >>> will drop and (because of (c)), you'll lose anyway. And if the spam ends >>> up outside of (c)'s range, either the rate limiting won't take effect >>> (spam's too cheap) and you'll be fine, or you'll miss out on the block >>> anyway (spam's paying more than your tx rate) and you never had any hope >>> of making it in. >>> >>> Note that we already rate limit via INVENTORY_BROADCAST_MAX / >>> *_INVENTORY_BROADCAST_INTERVAL; which gets to something like 10,500 txs >>> per 10 minutes for outbound connections. This would be a weight based >>> rate limit instead-of/in-addition-to that, I guess. >>> >>> As far as a non-ugly approach goes, I think you'd have to be smarter >>> about >>> tracking the "effective fee rate" than the ancestor fee rate manages; >>> maybe that's something that could fall out of Murch and Clara's candidate >>> set blockbuilding ideas [0] ? >>> >>> Perhaps that same work would also make it possible to come up with >>> a better answer to "do I care that this replacement would invalidate >>> these descendents?" >>> >>> [0] https://github.com/Xekyo/blockbuilding >>> >>> > > - keep high-feerate evicted txs around for a while in case they get >>> > > mined by someone else to improve compact block relay, a la the >>> > > orphan pool? >>> > Replaced transactions are already added to vExtraTxnForCompact :D >>> >>> I guess I was thinking that it's just a 100 tx LRU cache, which might >>> not be good enough? >>> >>> Maybe it would be more on point to have a rate limit apply only to >>> replacement transactions? >>> >>> > For wallets, AJ's "All you need is for there to be *a* path that >>> follows >>> > the new relay rules and gets from your node/wallet to perhaps 10% of >>> > hashpower" makes sense to me (which would be the former). >>> >>> Perhaps a corollarly of that is that it's *better* to have the mempool >>> acceptance rule only consider economic incentives, and have the spam >>> prevention only be about "shall I tell my peers about this?" >>> >>> If you don't have that split; then the anti-spam rules can prevent you >>> from getting the tx in the mempool at all; whereas if you do have the >>> split, then even if the bitcoind anti-spam rules are blocking you at >>> every turn, you can still send your tx to miners by some other route, >>> and then they can add it to their mempool directly without any hassle. >>> >>> Cheers, >>> aj >>> >>> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 25984 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2022-03-17 15:59 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-02-01 2:47 [bitcoin-dev] Improving RBF Policy Prayank 2022-02-01 9:30 ` Bastien TEINTURIER 2022-02-02 10:21 ` Anthony Towns -- strict thread matches above, loose matches on Subject: below -- 2022-02-09 17:57 lisa neigut [not found] <mailman.19693.1643292568.8511.bitcoin-dev@lists.linuxfoundation.org> 2022-01-31 22:54 ` [bitcoin-dev] Improving RBF policy Bram Cohen 2022-02-01 0:08 ` Eric Voskuil 2022-02-01 8:32 ` Bram Cohen 2022-02-01 19:44 ` Eric Voskuil 2022-02-01 0:42 ` Antoine Riard 2022-01-27 13:42 [bitcoin-dev] Improving RBF Policy Gloria Zhao 2022-01-28 1:35 ` Jeremy 2022-01-30 22:53 ` Antoine Riard 2022-01-31 15:57 ` Bastien TEINTURIER 2022-02-01 1:56 ` Anthony Towns 2022-02-05 13:21 ` Michael Folkson 2022-02-07 10:22 ` Bastien TEINTURIER 2022-02-07 11:16 ` Gloria Zhao 2022-02-08 4:58 ` Anthony Towns 2022-03-09 15:09 ` Gloria Zhao 2022-03-11 16:22 ` Billy Tetrud 2022-03-12 8:18 ` Billy Tetrud 2022-03-14 10:29 ` Gloria Zhao 2022-03-15 1:43 ` Billy Tetrud 2022-03-17 2:02 ` Antoine Riard 2022-03-17 15:59 ` Billy Tetrud
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox