Re: [bitcoin-dev] RBF Pinning with Counterparties and Competing Interest

[Jeremy <jlrubin@mit.edu>]

[-- Attachment #1: Type: text/plain, Size: 4438 bytes --]

Hi everyone,

Sorry to just be getting to a response here. Hadn't noticed it till now.

*(Plug: If anyone or their organizations would like to assist in funding
the work described below for a group of developers, I've been working to
put resources together for funding the above for a few months now, and I
think it would be high leverage towards seeing this through. There are a
lot of unsexy tasks to do  that aren't coming up with a solution
(e.g.,writing a myriad of Mempool stress test scenarios) that can be a well
defined full-time job for someone to do.)*

I've been working on exactly this problem in the mempool for months now.
I'm deeply familiar with the issues here and the types of pinning possible.
I think everyone can recognize that with my work on OP_CTV I want nothing
more than the mempool to be able to accept whatever long chains we can
throw at it, but I'm pretty well steeped at this point in the obstacles to
doing that.

I don't think that we should be entertaining further carve outs at the
moment, unless it is really trivial. Every new carve out rule added to the
way that the mempool operates is removing complexity invariants we aim to
preserve in the mempool in order to keep nodes operational. Many of these
invariants are well documented, some are not. I'm happy to go off list for
a more thorough discussion with anyone qualified to have it; this isn't the
best venue for that discussion.

From my point of view the path forward here is to dedicate more development
resources towards finishing the mempool project I began. You can see the
outstanding work here: https://github.com/bitcoin/bitcoin/projects/14,
contributing review towards moving those PRs forward will greatly improve
our ability to consider a stopgap carve out measure.

The current focus of this work is primarily on:

1) Testing Construction to better test & catch regressions or
vulnerabilities introduced or extant in mempool
2) Refactoring algorithms in mempool to reduce constant factors &
asymptotics
3) Package Relay


None of these fix the exact problem at hand though, but here's part of how
they can help us:

If we finish up the algorithmic refactors I've been working on it seems
plausible to do a one-off increase of descendants limits to say, 100
descendants with no restriction. However, we could use the opportunity to
use the 75 descendant increase exclusively for a new carve out, and apply
some new stricter rules in that extra space. There are a few anti-pinning
countermeasures that you can apply in that space that you would not
generally want in the mempool. An example of one is that any new
transaction must pay more feerate and absolute fee than every child in that
space. Or that only the highest fee paying branch of the excess
transactions are mineable, no others. Another would be disabling RBF past
that watermark. In all likelihood, different subsystems interacting with
the mempool will require a different set of restrictions each with the
current architecture, I don't think there's a magic bullet.

Package relay is a promising approach for a future pinning solution as
there are opportunities to attach to packages compact proofs of improved
fee efficiency for pinned transactions. But the ground work for package
relay needs to come first. This is theoretically possible with our current
architecture of the mempool and can probably address much of the pinning
concerns by replacing pinning with more rational eviction policies.

Longer term I've been working on plans and designs to completely re-do the
mempool's architecture to make it behave for arbitrary cases. It's possible
to one day lift all preemptively enforced (e.g., before acceptance)
descendants limits, which can solve this problem for good. There is more
than one potentially good solution here, and a conjunction of them can be
used as they affect independent sub systems. But this work will probably
take years to complete to the point where restrictions can realistically be
lifted.

If developers would like to coordinate resources around completing this
work and making more regular progress on it I'm happy to help point people
to specific tasks that need to be done in order to accelerate this and help
serialize the work so that we can not get into rebase hell.

Originally I had the plug at the top as a closing note, but I figured
people might miss it.

Best,

Jeremy


--
@JeremyRubin <https://twitter.com/JeremyRubin>

[-- Attachment #2: Type: text/html, Size: 8924 bytes --]