From: Jeremy <jlrubin@mit•edu>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists•linuxfoundation.org>
Cc: lightning-dev <lightning-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] RBF Pinning with Counterparties and Competing Interest
Date: Wed, 22 Apr 2020 18:18:05 -0700 [thread overview]
Message-ID: <CAD5xwhhKirBiF8ADE-H366opYA1idteC-qWYvQ8EUeAG4HMiZA@mail.gmail.com> (raw)
In-Reply-To: <CAO3Pvs_hNhEjX_tAatnij-4vd1cgPk8DPxOdrgDVuia7h=z5UQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4438 bytes --]
Hi everyone,
Sorry to just be getting to a response here. Hadn't noticed it till now.
*(Plug: If anyone or their organizations would like to assist in funding
the work described below for a group of developers, I've been working to
put resources together for funding the above for a few months now, and I
think it would be high leverage towards seeing this through. There are a
lot of unsexy tasks to do that aren't coming up with a solution
(e.g.,writing a myriad of Mempool stress test scenarios) that can be a well
defined full-time job for someone to do.)*
I've been working on exactly this problem in the mempool for months now.
I'm deeply familiar with the issues here and the types of pinning possible.
I think everyone can recognize that with my work on OP_CTV I want nothing
more than the mempool to be able to accept whatever long chains we can
throw at it, but I'm pretty well steeped at this point in the obstacles to
doing that.
I don't think that we should be entertaining further carve outs at the
moment, unless it is really trivial. Every new carve out rule added to the
way that the mempool operates is removing complexity invariants we aim to
preserve in the mempool in order to keep nodes operational. Many of these
invariants are well documented, some are not. I'm happy to go off list for
a more thorough discussion with anyone qualified to have it; this isn't the
best venue for that discussion.
From my point of view the path forward here is to dedicate more development
resources towards finishing the mempool project I began. You can see the
outstanding work here: https://github.com/bitcoin/bitcoin/projects/14,
contributing review towards moving those PRs forward will greatly improve
our ability to consider a stopgap carve out measure.
The current focus of this work is primarily on:
1) Testing Construction to better test & catch regressions or
vulnerabilities introduced or extant in mempool
2) Refactoring algorithms in mempool to reduce constant factors &
asymptotics
3) Package Relay
None of these fix the exact problem at hand though, but here's part of how
they can help us:
If we finish up the algorithmic refactors I've been working on it seems
plausible to do a one-off increase of descendants limits to say, 100
descendants with no restriction. However, we could use the opportunity to
use the 75 descendant increase exclusively for a new carve out, and apply
some new stricter rules in that extra space. There are a few anti-pinning
countermeasures that you can apply in that space that you would not
generally want in the mempool. An example of one is that any new
transaction must pay more feerate and absolute fee than every child in that
space. Or that only the highest fee paying branch of the excess
transactions are mineable, no others. Another would be disabling RBF past
that watermark. In all likelihood, different subsystems interacting with
the mempool will require a different set of restrictions each with the
current architecture, I don't think there's a magic bullet.
Package relay is a promising approach for a future pinning solution as
there are opportunities to attach to packages compact proofs of improved
fee efficiency for pinned transactions. But the ground work for package
relay needs to come first. This is theoretically possible with our current
architecture of the mempool and can probably address much of the pinning
concerns by replacing pinning with more rational eviction policies.
Longer term I've been working on plans and designs to completely re-do the
mempool's architecture to make it behave for arbitrary cases. It's possible
to one day lift all preemptively enforced (e.g., before acceptance)
descendants limits, which can solve this problem for good. There is more
than one potentially good solution here, and a conjunction of them can be
used as they affect independent sub systems. But this work will probably
take years to complete to the point where restrictions can realistically be
lifted.
If developers would like to coordinate resources around completing this
work and making more regular progress on it I'm happy to help point people
to specific tasks that need to be done in order to accelerate this and help
serialize the work so that we can not get into rebase hell.
Originally I had the plug at the top as a closing note, but I figured
people might miss it.
Best,
Jeremy
--
@JeremyRubin <https://twitter.com/JeremyRubin>
[-- Attachment #2: Type: text/html, Size: 8924 bytes --]
next prev parent reply other threads:[~2020-04-23 1:17 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-21 2:43 Matt Corallo
2020-04-22 4:12 ` [bitcoin-dev] [Lightning-dev] " ZmnSCPxj
2020-04-22 4:18 ` Olaoluwa Osuntokun
2020-04-22 6:08 ` ZmnSCPxj
2020-04-22 8:01 ` Antoine Riard
2020-04-22 8:55 ` Bastien TEINTURIER
2020-04-22 23:05 ` Olaoluwa Osuntokun
2020-04-22 23:11 ` Olaoluwa Osuntokun
2020-04-22 16:56 ` Matt Corallo
2020-04-22 4:13 ` [bitcoin-dev] " Olaoluwa Osuntokun
2020-04-22 11:51 ` David A. Harding
2020-04-27 21:26 ` Rusty Russell
2020-04-22 16:50 ` Matt Corallo
2020-04-22 23:13 ` Olaoluwa Osuntokun
2020-04-22 23:20 ` Matt Corallo
2020-04-22 23:27 ` Olaoluwa Osuntokun
2020-04-23 1:10 ` Matt Corallo
2020-04-23 4:50 ` [bitcoin-dev] [Lightning-dev] " ZmnSCPxj
2020-04-23 6:21 ` Matt Corallo
2020-04-23 12:46 ` ZmnSCPxj
2020-04-23 22:47 ` Matt Corallo
2020-06-19 7:44 ` Bastien TEINTURIER
2020-06-19 19:58 ` David A. Harding
2020-06-19 20:52 ` David A. Harding
2020-06-20 8:54 ` Bastien TEINTURIER
2020-06-20 10:36 ` David A. Harding
2020-06-20 16:01 ` ZmnSCPxj
2020-06-21 2:10 ` ZmnSCPxj
2020-06-22 7:35 ` Bastien TEINTURIER
2020-06-22 8:15 ` ZmnSCPxj
2020-06-22 8:25 ` Bastien TEINTURIER
2020-06-24 8:32 ` Matt Corallo
2020-04-23 1:18 ` Jeremy [this message]
2020-04-22 18:24 ` [bitcoin-dev] " David A. Harding
2020-04-22 19:03 ` Antoine Riard
2020-04-22 20:28 ` David A. Harding
2020-04-22 22:53 Matt Corallo
2020-04-23 9:59 ` David A. Harding
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAD5xwhhKirBiF8ADE-H366opYA1idteC-qWYvQ8EUeAG4HMiZA@mail.gmail.com \
--to=jlrubin@mit$(echo .)edu \
--cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
--cc=lightning-dev@lists$(echo .)linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox