Re: [bitcoin-dev] On mempool policy consistency

From: Antoine Riard <antoine.riard@gmail•com>
To: Suhas Daftuar <sdaftuar@gmail•com>,
	 Bitcoin Protocol Discussion
	<bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] On mempool policy consistency
Date: Wed, 2 Nov 2022 15:50:26 -0400	[thread overview]
Message-ID: <CALZpt+HHsqN0tq7LYfR9YFRjWo_dKwDDAwB_HPLacafVj8anWw@mail.gmail.com> (raw)
In-Reply-To: <CAFp6fsFm06J2G1=3ojQvcL9gbEbQ41C4rmf3=Jkm9Qm0VBhhKw@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 27757 bytes --]

Hi Suhas,

From my understanding, the main crux of the reasoning exposed in your post
would be to solidify the transaction-relay paradigm we have been following
during the last years, e.g introducing the carve-out rule specifically for
lightning commitment transactions, or more recently version=3 transactions.
I think this paradigm could be described summarly as "to each use-case
belongs a set of transaction-relay policy rules". Some of this set of rules
could aim to non-replacement guarantees to the consumers of transactions
signaling under this regime (e.g zeroconf). Another set of rules could
provide a high-guarantee that a transaction would always get to a miner, no
matter what the state of node's mempools on the network (e.g "maximal rbf"
for contracting protocols).

First, coming out of my mind, we would have to consider isolation between
each set of policy rules, ensuring the signaling mechanism cannot be abused
by an attacker to create a new pinning vector. E.g, a hypothetical concern
could be BIP125 rules interfering with version=3 policy to block the
replacement of better ancestor feerate packages. Further, for each set of
policy rules arises the question of internal consistency, again an attacker
could abuse them to pin transactions
propagation. I think an earlier version of version=3 was suffering from
this concern of not scoping potential "junk" ancestors. None of those
issues are unsolvable, however we should be well-aware of the
non-negligeable design complexity encumbered by transaction-relay protocol
developers to achieve the correct goal. There is not only a need to ensure
careful policy rules security analysis, but further to communicate well
their usage to second-layers and wallets developers (a task far from easy
with all the confusions contained by current BIP125).

Now, in the evaluation process of a set of policy rules soundness are
proposed a few reasoning heuristics: namely that it shouldn't interfere
sensibly with a anti-DoS mempool acceptance algorithm, or shouldn't
interfere with other protocols on the network, or counter the interests of
miners or node operators. I hold the belief the latest question could be
the one raising the most concerns. Browsing in the history of Bitcoin Core,
I think one of the design goals aimed for has always been to level the
playing field between miners, e.g BIP152 improving block transfer latency
to reduce orphan rate. Following this principle, we might wonder if our
transaction-relay network should guarantee some equal access to transaction
information to all the miners.

In the present case of a non-replacement policy regime, we could see the
following situation to arise, on one side miners deploying private
transaction-relay communication channels or API to capture higher fees
income from non-standard transactions. On the other-side, transaction
issuers or consumers bypass the standard transaction-relay policy rules.
Bypass could be motivated by either a zeroconf service double-spend, or
faster confirmation of a collaborative transaction. E.g, to reuse the
example of unconfirmed transaction chaining, where the sender commit to
non-replacement by opting out from the RBF flag, this commitment could be
reevaluated in the light of changing network mempools congestion, or
liquidity preferences (e.g the quick need to open LN routing channels). The
sender could leverage such hypothetical private transaction-relay
communication channels to revoke its non-replacement commitment. Therefore
discrepancies between a set of policy rules design and miners incentives
sounds to lead to informational asymmetries, harmful for the long-term
decentralization of the mining ecosystem. Of course, miner incomes
asymmetries due to edge in transaction flows access might not be weighted
as serious today, in a world where transaction fees contribute to most of
the block reward, this is far more worrying!

Of course, one position could be to estimate that miner centralization is
beyond the scope of responsibility of the Bitcoin Core project. Or at least
as a result of lightweighted risks.

Such discrepancy between a set of policy rules design and miners incentives
could also lead to hidden security risks for second-layers. As we see more
security assumptions made on policy rules extension, e.g version=3, a
lightning channel counterparty could have a competing interest to forge a
raw package suiting better incentives, and as such nullify the security
advantage expected. This could be seen as a loose concern, however the last
time we have seen an actor deliberately providing non-standard transactions
to a miner to break second-layers was yesterday [0]!. From observing other
cryptocurrencies spaces, such "MEV-style" attacks could be more and more
concerning [1]. I don't think we should assume miners to behave as "network
gentlemen", in a world where mining can be anonymous, permissionless and
censorship-resistant (e.g Stratum V2 giving back template construction to
miners operators rather than pools).

According to me, one of the harder problem we're seeing with this fullrbf
discussion is the lack of a consistent, grounded and well-understood miner
incentive model, where not only block template construction but also
transaction collection and replacement strategies are analyzed, and against
which we could simulate the efficiency of a policy. Assuming we would have
such a model, rather than qualify a policy rule as incentive-compatible in
a binary fashion, we could evaluate them on a scale, and agree on when
they're satisfying enough in face of technical complexity, validation
resources, margin of adversarial exploitation, or whatever other relevant
criteria.

Answering a few other points raised in this post, what appears to me
obscure is the qualification that fullrbf doesn't solve the DoS issues for
contracting protocols (e..g coinjoin/dual-funded lightning). If I can
understand, it's on the ground that the imperfections of BIP125 underscore
only the direct conflict feerate. It should be remembered that allowing
replacement without considering the opting flag would be already an
improvement against the DoS attack, as the attack would have to offer a
more compelling feerate to maintain the pin. Improvements of BIP125 can
happen on the top, but solving the opt-out double-spend issue sounds to me
a prerequisite.

Beyond that, I think few questions are laid out on the conceptual soundness
of v3 transaction policy w.r.t concerns raised about fullrbf today. In my
opinion, I'm sadly with most of them, especially that miners might earn
more revenue if we allowed multiple descendant v3 transactions and the
unenforceable promise for the recipient of such package to not add more
high-value children, I've echoed those concerns earlier in the review of
nVersion=3 proposal [2]. We might have to swallow the bullet for now, and
be okay as lightning developers and operators that there is only a social
inertia of the miners and lack of reliable communication channels towards
them by an adversarial counterparty to offer security [3]. Additionally, I
think it would be acceptable to have an option to disable v3 transaction
policy, an operator could be willing to reduce the CPU/memory DoS surface
of its node from partaking to any package relay. Even if it comes at the
loss of a better view of blockspace demand and downgrades its
fee-estimation, I think we should give the maximum flexibility to operators
in choosing their risk model.

To put it in a nutshell, if we would like to pursue further in the paradigm
that "to each use-case belongs its set of policy rules" (as long as they
don't introduce any harm for the network stakeholders), I believe we would
be more grounded with a better quantitative understanding of so-called
"miners incentives". I'm still wondering if it's realistic to deploy policy
rules that are not sustainable in face of long-term mining dynamics.

Best,
Antoine

[0] https://github.com/lightningnetwork/lnd/issues/7096

[1] On risks of introducing miner harvesting attacks, especially when you
consider implications on lightning-style constructions
https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-February/002569.html
and
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-November/019615.html

[2]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020939.html

"If you're a miner and you receive a non-V3, second descendant of an
unconfirmed V3 transaction, if the offered fee is in the top mempool
backlog, I think you would have an interest to accept such a transaction.

So I'm not sure if those two rules are compatible with miners incentives..."

[3] This wonders if we should look forward in the future to lock in the
CPFP weight of a Lightning commitment with some new consensus semantic, or
leveraging any covenant magic, cf.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-March/020122.html
and
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/020991.html

Le lun. 31 oct. 2022 à 11:02, Suhas Daftuar via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> a écrit :

> AJ,
>
> Thanks for the thoughtful post. I think your observations about how we
> view mempool policy in the Bitcoin Core project, and how that seems to be
> changing in the discussions around `-mempoolfullrbf`, are on-point and
> provide a helpful baseline for considering future policy changes.
>
> For a long time I viewed fullrbf as an eventuality and I considered myself
> to be philosophically supportive of the idea.  However, after giving this
> issue some thought in the past few weeks, I am reversing my thinking on
> this.  Concretely, I will argue that we should continue to maintain a relay
> policy where replacements are rejected for transactions that don't opt-in
> to RBF (as described in BIP 125), and moreover, that we should remove the
> `-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not
> plan to release software with that flag, unless (or until) circumstances
> change on the network, which I'll discuss below.
>
> This is, of course, a nuanced topic, and among the considerations is a
> philosophy of how to think about the relay policy and configuration options
> that we make available in Bitcoin Core (a consideration that is perhaps
> unique to that project, but I think relevant for this mailing list).
>
> I'll start with some technical issues regarding the benefits of enabling
> fullrbf on the network.  In the current BIP 125 regime, every time a
> transaction is created, a choice is made whether to subject the transaction
> to BIP 125’s RBF rules or not (based on the sequence values of the
> inputs).  So given that users can already opt-in to RBF, the benefit of a
> “fullrbf” network policy would be if, somehow, RBF users were still denied
> the benefits of RBF due to the existence of other transactions that don’t
> opt-in.
>
> Along those lines, Antoine Riard brought up[1] a DoS vector that is
> available to someone who wants to interfere with multi-party funded
> transactions, and suggested that fullrbf would eliminate the problem.
> After exploring that question again in this thread (thanks to Greg Sanders
> for clarifying this to me), I understand that the issue is around ensuring
> that a multiparty (coinjoin-type) protocol is able to make eventual
> progress, by having a candidate multiparty transaction either eventually
> confirm or become conflicted with something that has been confirmed, in
> which case the double-spend information could be used to start a new
> coinjoin round with fewer participants.  The concern Antoine and Greg have
> brought up is that non-rbf transactions can persist in the mempool
> ~indefinitely (at a low feerate and not subject to replacement) and
> interfere with progress being made in a coinjoin protocol.
>
> However, it seems to me that similar problems exist for such a protocol
> even in a fullrbf world, as we understand that term today.  I mentioned the
> ability for rbf “pinning” to interfere with relay of the multiparty
> transaction (even if the conflicting transaction signals for RBF – a set of
> large but low feerate conflicting transactions can persist in the mempool
> and make it difficult for the coinjoin transaction from confirming, at
> least without attaching a very large fee); and as Greg mentioned in a
> followup, the BIP 125 rule to only permit 100 transactions to be removed
> from the mempool at a time during a replacement can also be used to pin a
> coinjoin protocol in the same way as a non-rbf transaction today.  It seems
> to me that what these multiparty protocols actually need is some sort of
> "maximal rbf" network policy: a way to guarantee that a transaction which
> should be desirable for a miner to mine would always get to a miner and
> considered for inclusion in a block, no matter what the state of node’s
> mempools on the network.
>
> While that sounds like a reasonable thing to want on its face (and worth
> working on), it's not how opt-in RBF works today, nor is it how transaction
> relay has ever conceptually worked.  We have not, thus far, been able to
> come up with a total ordering on transaction desirability.  Moreover, due
> to all the DoS issues that exist with transaction relay, there are plenty
> of seemingly legitimate ways to construct transactions that would not relay
> well on the network.  Relay has only ever been a best-efforts concept,
> where we carve out a small subset of the entire transaction universe for
> which we try to optimize propagation.  The idea behind this approach is
> that if every use case we can come up with has some way to achieve its
> goals using transactions that should (eventually) be able to relay, then
> users wouldn’t have much demand for transactions that would deviate from
> the supported policies, and we therefore shouldn’t need to worry too much
> about incentive compatibility concerns when it comes to transaction types
> that wouldn’t relay at all, even if they are high feerate.  (And when those
> situations arise where the standard transactions do not accommodate some
> needed use case, developers typically work to define a policy that is
> compatible with our anti-DoS goals to support such use cases, such as with
> the recent proposal for version=3 transactions [2].)
>
> BIP 125's RBF rules themselves were an effort to carve out just a subset
> of situations where a transaction should evict conflicting ones -- it was
> not a design that anyone thought would ensure that all replacements which
> "should" be mined would always propagate.  And I don't believe that we know
> how to design policy rules that would achieve the goals of this kind of
> multiparty protocol in a DoS resistant way, today.  Along those lines, I
> would point out that even the BIP 125 design itself is not entirely
> incentive compatible, in that it is possible to construct a replacement
> transaction that would evict transactions which would be preferable to be
> included in a block! [3]  (This has been known for years, but fixing this
> has proven difficult, and the only way to fix it that I’m aware of would be
> to make BIP 125 RBF even more restrictive than it is today. I do think this
> is something that needs to be worked on.)
>
> Given the limitations of RBF as we have it today, it appears to be
> incorrect that a fullrbf network policy would solve the problems Antoine
> raised.  And so absent any other examples, it does not seem to me that
> fullrbf solves any problems for RBF users, who are already free to choose
> to subject their transactions to BIP 125’s RBF policy.  From this
> perspective, "enabling fullrbf" is really just taking away user choice to
> opt a transaction into a non-replacement policy regime.
>
> I think we should ask, then, whether it is reasonable on its face that
> users might want to opt-in to a non-replacement policy?  Or in other words,
> is it reasonable for a user to mark a transaction as non-replaceable and
> have that indication be enforced by the network? Note that these are two
> different questions: you could imagine a world where fullrbf is a dominant
> policy, but users still use the BIP 125 signaling method to indicate, in an
> unenforced way, their intention to not replace a transaction.  This might
> give useful information to the network or the recipient for how to interact
> with such a transaction.
>
> And I think that it's entirely possible that users would continue to use
> the BIP 125 signaling to indicate that they do not intend to replace a
> transaction.  For better or worse, this might be because zeroconf services
> continue to differentiate their behavior based on such a signal (possibly
> in conjunction with other factors), or it could be because there are other
> behaviors that could be utilized more effectively if the transaction
> originator has made such a signal, such as the recipient chaining an
> unconfirmed transaction as a way to bump the fee (CPFP) [4].
>
> If it were to be the case that users continued to use BIP 125-style
> signaling to indicate that they do not plan to replace a transaction, would
> that be harmful to the network?  This is not something we can stop in our
> policy rules (short of censoring such transactions, an obviously bad
> idea).  I think network actors can always do things that we might think are
> harmful for the network, but that doesn’t mean that there are no legitimate
> use cases for the tools that such actors might be using.  Just because
> someone might use some policy to adopt a zeroconf model, doesn’t mean that
> others aren’t using the same policy to achieve benign ends (such as better
> CPFP behavior).
>
> Moreover, while users might attempt to exploit services that offer
> zeroconf or other differentiated behavior to non-replacement signaling
> transactions, they also might not -- I think predicting user behavior in
> this way (and specifically predicting the complexities of what a business
> might do and whether users might try to subvert it) is beyond the scope of
> what we can do as protocol developers.  Instead, I think we can try to
> answer a different question: if a group of users were to want the ability
> to opt-in to a non-replacement policy regime, is that a technically sound
> option for us to have on the network and enforce in software?
> Specifically, does that interfere with having a sensible anti-DoS mempool
> acceptance algorithm, or interfere with other protocols on the network, or
> necessarily run counter to the interests of miners or node operators?
>
> And I think the answer to that question, in looking at the difference
> between opt-in RBF and fullrbf, is no: offering the ability to opt-in to a
> non-replacement regime for transactions doesn't introduce any fundamental
> issues with software or network policy or other protocols.  In a world
> where we only had fullrbf, I could imagine at some point down the road
> proposing a non-replacement signal myself, because the complexities around
> transaction chains (and pinning) are more complex for the RBF case than for
> the non-RBF case (and BIP 125 is not always incentive compatible to begin
> with!).  Conceptually, this is no different to me than the version=3
> transaction policy proposal that has been advancing, if we think of it as a
> special set of restrictions on transactions designed to accommodate a
> particular use case.
>
> Philosophically, I think we should be looking to add non-interfering use
> cases to what the network supports.
>
> To those who argue for making fullrbf a default policy on the network (or
> even just offering a flag for users to enable fullrbf), I pose this
> hypothetical: suppose we deploy the v3 transaction policy proposal (which I
> hope will happen in the near future).  That policy would restrict the ways
> that outputs of a v3 transaction can be spent while the transaction is
> unconfirmed, including by limiting the number and size of descendants that
> such a transaction can have, and limiting the types of unconfirmed
> ancestors that can be included.  Suppose in a few years someone proposes
> that we add a "-disable_v3_transaction_enforcement" flag to our software,
> to let users decide to turn off those policy restrictions and treat v3
> transactions the same as v2, for all the same reasons that could be argued
> today with fullrbf: miners might earn more revenue if we allowed multiple
> descendant v3 transactions; it's illogical for the recipient of a v3
> transaction to believe what is a fundamentally unenforceable promise of a
> sender to not issue more high value children that descend from an
> unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
> policy on the network and we should honor user choice to turn off that flag
> if that’s what users want; if users are relying on v3’s policy restrictions
> for security then that is an unstable model and we should assume it will
> get broken[5].
>
> It’s obvious to me that adding a flag to disable v3 policy would be
> subversive to making the lightning use case for v3 transactions work.  And
> so my response to such a hypothetical proposal would be to argue that no,
> we should not enable users to disable this policy, because as long as that
> policy is just optional and working for those who want it, it shouldn’t
> harm anyone that we offer a tighter set of rules for a particular use
> case.  Adding a way to bypass those rules is just trying to break someone
> else’s use case, not trying to add a new one.  We should not wield
> "incentive compatibility" as a bludgeon for breaking things that appear to
> be working and not causing others harm.
>
> I think this is exactly what is happening with fullrbf.
>
> In comparing v3 transaction policy with opting out of transaction
> replacement, there is of course one significant difference that I have
> ignored thus far: I think the real difference is an opinion about whether
> non-replacement transactions that are being used today are, overall, bad
> for Bitcoin, and whether lightning’s use of v3 transactions in the future
> would be bad for Bitcoin. If you think that zeroconf is unequivocally bad,
> and that no one will be able to plausibly construct a case that lightning
> is bad, then that qualitative judgment might sway you to not worrying about
> the philosophical issues I've raised above, because these situations can be
> distinguished.
>
> However I am not personally willing to say that I think, overall,
> non-rbf-signaling transactions in use on the network today are bad for
> Bitcoin (or that fullrbf is definitely good – BIP 125’s rbf rules are
> something we’ve been trying to improve upon for years, with little
> success).  Nor am I convinced that someone couldn’t put together a cogent
> argument for lightning being bad for Bitcoin, because of its reliance on
> relay policies that are difficult to design and impossible to guarantee as
> part of its security model.  So I choose instead to merely make a judgment
> that seems more factually verifiable, which is that non-replacement is a
> policy widely in use on the network today, and we largely don't have reason
> to think (as far as I know!) that the network is seeing a lot of
> transactions that would violate that policy.
>
> If it did turn out that users were commonly signaling non-replacement, but
> then signing and trying to relay doublespends, then I think that would be a
> very good reason for Bitcoin Core to adopt fullrbf to reflect the reality
> of what is happening.  In the meantime, I think it makes more sense to say
> that because we have BIP 125, there seems to be no need for users to signal
> one way and behave another, and therefore there is no need to offer
> software that might break a policy that is working well for some users.
> Other software projects might choose differently, and it is after all a
> permissionless network, so if this is in fact an unstable equilibrium that
> will not last, then presumably someday it will be apparent it is not
> working and we’ll abandon it.  But I think the philosophy of transaction
> relay policy in Bitcoin Core should be to support disparate use cases in
> order to try to make everything work better, rather than break things
> prematurely because we guess others will break them eventually anyway.
>
> For those that have read this long email and still favor a fullrbf network
> policy (or even just the ability for users to be able to turn on fullrbf
> for themselves), I’d ask for thoughts on the following questions, which
> have guided my thinking on this:
>
> Does fullrbf offer any benefits other than breaking zeroconf business
> practices?  If so, what are they?
>
> Is it reasonable to enforce BIP 125's rbf rules on all transactions, if
> those rules themselves are not always incentive compatible?
>
> If someone were to propose a command line option that breaks v3
> transaction relay in the future, is there a logical basis for opposing that
> which is consistent with moving towards fullrbf now?
>
> Cheers,
> Suhas
>
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> [2]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
>
> [3] This is because under the BIP 125 rules, the feerate of the
> replacement transaction is not compared to the individual feerates of all
> transactions being evicted – we just compare feerates with the transactions
> that are directly in conflict (and not their descendants). So it’s possible
> for a transaction that would evict 2 or more transactions to have a higher
> feerate than the direct conflicts, and higher total fee than the set being
> evicted, but have a lower feerate (eg if it is larger) than that of some
> subset of the set of transactions being evicted.
>
> [4]  Chaining unconfirmed transactions when the sender might RBF the
> parent is far riskier than if the sender indicates they don't plan to do so
> (chaining onto an RBF transaction creates pinning issues for the sender,
> and risks having the child wiped out if the parent is replaced), so I think
> this is a concrete reason why signaling that a transaction won’t be
> replaced could be useful.
>
> [5] This is a subtle point. I don’t think v3 transactions create an
> unreasonable security assumption for the use case it is being designed for.
> However, I don’t think anyone could rule out the possibility that someone
> could adopt a usage pattern for v3 transactions that subverts the intent of
> this policy.  For example, if users started using v3 transactions for all
> their payments, then the limitations on the number of descendants could
> directly interfere with CPFP by a recipient, and someone could argue that
> we should break the policy in order to allow for this hypothetical
> behavior. I think this is a similar form of argument as saying that
> zeroconf practices + BIP 125 create an incentive to double-spend non-rbf
> signaling transactions.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 28756 bytes --]