* [bitcoin-dev] On mempool policy consistency
@ 2022-10-26 23:52 Anthony Towns
2022-10-27 12:36 ` Gloria Zhao
` (4 more replies)
0 siblings, 5 replies; 33+ messages in thread
From: Anthony Towns @ 2022-10-26 23:52 UTC (permalink / raw)
To: Bitcoin Protocol Discussion
Hi *,
TLDR: Yes, this post is too long, and there's no TLDR. If it's any
consolation, it took longer to write than it does to read?
On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev wrote:
> Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
> If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> consistent deprecation process w.r.t to policy rules heavily relied-on by
> Bitcoin users, if we do so let sets a precedent satisfying as many folks as
> we can.
One precedent that seems to be being set here, which to me seems fairly
novel for bitcoin core, is that we're about to start supporting and
encouraging nodes to have meaningfully different mempool policies. From
what I've seen, the baseline expectation has always been that while
certainly mempools can and will differ, policies will be largely the same:
Firstly, there is no "the mempool". There is no global mempool. Rather
each node maintains its own mempool and accepts and rejects transaction
to that mempool using their own internal policies. Most nodes have
the same policies, but due to different start times, relay delays,
and other factors, not every node has the same mempool, although they
may be very similar.
- https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
Up until now, the differences between node policies supported by different
nodes running core have been quite small, with essentially the following
options available:
-minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
-mempoolexpiry - how long to keep txs in the mempool
-datacarrier - reject txs creating OP_RETURN outputs
-datacarriersize - maximum size of OP_RETURN data
-permitbaremultisig - prevent relay of bare multisig
-bytespersigop - changes how SIGOP accounting works for relay and
mining prioritisation
as well as these, marked as "debug only" options (only shown with
-help-debug):
-incrementalrelayfee - make it easier/harder to spam txs by only
slightly bumping the fee; marked as a "debug only" option
-dustrelayfee - make it easier/harder to create uneconomic utxos;
marked as a "debug only" option
-limit{descendant,ancestor}{count,size} - changes how large the
transaction chains can be; marked as a "debug only" option
and in theory, but not available on mainnet:
-acceptnonstdtxn - relay/mine non standard transactions
There's also the "prioritisetransaction" rpc, which can cause you to keep
a low feerate transaction in your mempool longer than you might otherwise.
I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
ones of those options commonly set, and those only rarely result in any
differences in the txs at the top of the mempool.
There are also quite a few parameters that aren't even runtime
configurable:
- MAX_STANDARD_TX_WEIGHT
- MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
- MAX_P2SH_SIGOPS (see also #26348)
- MAX_STANDARD_TX_SIGOPS_COST
- MAX_STANDARD_P2WSH_STACK_ITEMS
- MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
- MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
- MAX_STANDARD_P2WSH_SCRIPT_SIZE
- MAX_STANDARD_SCRIPTSIG_SIZE
- EXTRA_DESCENDANT_TX_SIZE_LIMIT
- MAX_REPLACEMENT_CANDIDATES
And other plausible options aren't configurable even at compile time
-- eg, core doesn't implement BIP 125's inherited signalling rule so
there's no way to enable it; core doesn't allow opting out of BIP 125
rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
more than 1 ancestor; core doesn't allow opting out of LOW_S checks
(even via -acceptnonstdtxn); etc.
We also naturally have different mempool policies between different
releases: eg, expansions of policy, such as allowing OP_RETURN or
expanding it from 40 to 80 bytes or new soft forks where old nodes won't
relay transactions that use the new; and also occassional restrictions
in policy, such as the LOW_S requirement.
While supporting and encouraging different mempool polices might be new
for core, it's not new for knots: knots changes some of these defaults
(-permitbaremultisig defaults to false, -datacarriersize is reduced to
42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
options including -spkreuse and -mempoolreplacement (giving the latter
full rbf behaviour by default). Knots also includes a `-corepolicy`
option to make it easy to get a configuration matching core's defaults.
I think gmaxwell's take from Feb 2015 (in the context of how restrictive
policy on OP_RETURN data should be) was a reasonable description for
core's approach up until now:
There is also a matter of driving competent design rather than lazy
first thing that works. E.g. In stealth addresses the early proposals
use highly inefficient single ECDH point per output instead of simply
pooling them. Network behavior is one of the few bits of friction
driving good technical design rather than "move fast, break things, and
force everyone else onto my way of doing thing rather than discussing
the design in public". No one wants to be an outright gatekeeper,
but the network is a shared resource and it's perfectly reasonable
node behavior to be stingy about the perpetual storage impact of the
transactions they're willing to process, especially when it comes to
neutral technical criteria like the amount of network irrelevant data
stuffed in transactions.
There is also a very clear pattern we've seen in the past where
people take anything the system lets them do as strong evidence that
they have a irrevocable right to use the system in that way, and that
their only responsibility-- and if their usage harms the system it's
the responsibility of the system to not permit it. [...
...] For mitigating these risks it's optimal if transactions
seem as uniform and indistinguishable as reasonably possible.
- https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
Perhaps see also sdaftuar in Nov 2015,
To me the most important question is, is priority something that miners
want to use?
If a non-negligible amount of hashpower intends to use it in their
transaction selection, then I think it makes sense for nodes to use it
too, because it's generally helpful to have your mempool predict the
UTXO as much as possible, and for nodes to be able to have reasonable
fee and priority estimates (which won't happen unless they track the
priority transactions somehow -- I'm presuming that miners run with
much bigger mempools than regular nodes).
If the answer is no, then that's fine and I don't see a reason to push
in this direction. I sort of assumed there was enough hashpower mining
with priority, since last time I checked estimatepriority was still
giving meaningful results for low-ish blockheights, but I haven't done
any kind of real analysis.
- https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
or in June 2019,
What this PR is proposing is to get rid of a command-line option that is
(a) a footgun for users and (b) does not reflect what I believe to be
the understanding most users have, which is that [X txs] are expected
to propagate well on the network.
..
I don't think this rises to the level that Luke is concerned about,
namely a prelude to forcing a common relay policy on all nodes. In
particular I do agree it makes sense that we offer some ways of
customizing policy parameters (eg the mempool size, min relay fee,
etc). Instead, I think the justification for this change is that we
should not support behaviors we think are harmful to the ecosystem
overall and have no legitimate use-case, and we should eliminate ways
that users might inadvertently shoot themselves in the foot.
- https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
(or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
I don't mean to imply the above are saying "there's one way to do
things and it's this way", or that the old way of doing things should
necessarily be the way we keep doing things. Just that previously core
has tended towards designing a single policy that works as well as it
can for everyone and the ecosystem as a whole. (I'm also not saying that
fullrbf can't work well for everyone or the ecosystem as a whole)
By contrast, I think the most common response to pushback against the
full rbf option has been along the lines of "it's just an option, we
don't want to force people", eg:
Blaming the default false -mempoolfullrbf option for a full RBF network
would be holding Bitcoin Core developers responsible for the decisions
of individual node operators and miners. I don't think having the
option (again, default false) can directly cause a full RBF network,
and likewise, I don't think removing this option removes the "risk"
of a full RBF network.
- glozow
https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
NACK. This is a default false option.
- achow101
https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
Erecting artificial barriers to prevent or make it difficult for users
to do what they want to do, is not appropriate behaviour.
- luke-jr
https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
I'm in general against removing options.
- instagibbs
https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
I think this differs from what core has done in the past, in that
previously we've tried to ensure a new policy is good for everyone (or as
nearly as it can be), and then enabled it as soon as it's implemented.
Any options that have been added have either been to control resource
usage in ways that don't significantly effect tx propagation, to
allow people to revert to the old behaviour when the new behaviour is
controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
and to make it easier to test/debug the implementation.
Giving people a new relay behaviour they can opt-in to when we aren't
confident enough to turn on by default doesn't match the approach I've
seen core take in the past.
If this is going to be an ongoing shift in how core sees relay/mempool
policy, I think that's significant and worth paying attention to.
I don't think it's necessary to have that shift to roll out full rbf.
The other approach would be either:
* set -mempoolfullrbf=true as the default for 24.0, and just have the
command line param there in case people want to do a
"UserRejectedMempoolPolicy" campaign to get everyone to opt-out
* revert it for now because we don't think mainnet is ready for fullrbf
yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
to activate at some scheduled date in that timeframe (potentially
backporting it to previous releases to help with adoption too,
whatever). same effect as the previous option, just with a bit more
advanced notice and time to prepare
I don't think anyone's proposed the first (which I interpret as "most of
us don't think mainnet is ready for fullrbf today"), but the comments
above are all pushback by people arguing against (the first step of)
the second approach, and they seem to be winning the day.
It's also possible that this is something of a one time thing: full rbf
has been controversial for ages, but widely liked by devs, and other
attempts (eg making it available in knots) haven't actually achieved
much of a result in practice. So maybe this is just a special case and
not a precedent, and when people propose other default false options,
there will be substantially more resistance to them being merged,
despite all the talk about users having options that's going on right now.
Assuming it is the change of direction it appears to be -- and all of
the above is really just justification for that assumption -- then like
I said, I think it's worth seriously considering what it means for people
to choose their own relay/mempool policies and for you to expect to have
different mempool policies to many or most of your potential peers.
One thing maybe worth noting is that is that you can still only choose
your policy from options that people write code for -- if it wasn't
something you could get by running knots or compiling a rejected PR
yourself, it won't magically become more possible now. Presumably it
would mean that once a PR is written, it might get better review (rather
than being dismissed as not suitable for everyone), and there would be
less maintenance burden than if it had to be manually rebased every
release, though (or at least the maintenance burden would be shared
across everyone working on the codebase).
The second thing is that whatever your relay policy is, you still
need a path all the way to miners through nodes that will accept your
transaction at every step. If you're making your mempool more restrictive
(eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
you're making life more difficult for people who do create those sorts
of txs); but if you want a more permissive policy (package relay,
version-3-rbf, full-rbf), you might need to do some work.
The cutoff for that is probably something like "do 30% of listening
nodes have a compatible policy"? If they do, then you'll have about a
95% chance of having at least one of your outbound peers accept your tx,
just by random chance. If erlay allows increasing your outbound count to
12 connections instead of 8; that might reduce down to needing just 20%
of listening nodes (~93%).
But for cases where less than 30% (20%) of network supports your preferred
policy, you probably need to do something cleverer.
One approach is to set a service bit and preferentially peer with other
nodes that advertise that service bit; knots does the first half of this
for fullrbf, and both halves have been proposed for core in #25600.
Preferential peering was previously done for the segwit deployment,
though in that case it was necessary not just for tx propogation but
also for ensuring block propogation, making it effectively a consensus
critical issue.
Another approach is having a separate relay network -- eg, lightning nodes
already have a gossip network, and might want to help their own ecosystem
by ensuring unilateral channel closes and justice transactions are quickly
relayed. Using their own gossip network to relay the transaction around,
and each lightning node adding it to their local bitcoind's mempool and
allowing it to propogate (or not) from there as normal, would also be a
way of allowing transactions to propogate well. It does mean that miners
would either need to also participate in lightning gossip directly, or
that miners would need to connect to *many* peers to be confident of
seeing those transactions (eg, if only 2% of the network would see a
tx, you'd need to make 228 connections to have a 99% chance of seeing
the tx). You can't currently do something like this, because all the
relay policies are also applied when adding txs to the mempool via RPC,
and there's no convenient way to remove txs from the mempool.
A case where something like that might occur is in preventing L2
transactions from pinning attacks -- so you might have a high-fee,
low-feerate transaction that's been widely propogated, sitting in the
bottom of people's mempools, and you want to replace it with a smaller,
higher-feerate transaction, but don't want to pay a higher absolute fee,
and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
unwilling to deviate from BIP 125 rule 3 for you; because that would
make it easy for random griefers to spam their mempool with large txs
then delete them while only paying a small fee; but your L2 peers may be
able to decode your replacement transaction and be sure that you aren't
going to spam them, and thus will happily relay it.
From a technical point-of-view, that's largely fine; the downside is it
increases the centralisation pressure on mining: whether that's by having
to connect to substantially more nodes, or having to parse through more
spam, you can't just run your mining operation off a standard install
of bitcoin core anymore, but need to actively opt-in to find all the
weird unusual ways people are sending transactions around in order to
actually collect as much in fees as your competitors are.
That's probably moderately bad for privacy as well -- if lightning or
coinjoins need special relay rules that most nodes haven't opted into,
it's potentially easy to use that to find the bitcoin nodes on the
network that are participating in those protocols, and from there to
either identify the operator, or run a DoS attack to make it hard for you
to keep doing what you want. Obviously if you're setting a service bit to
get better routing, you've given up that privacy already. Likewise if the
government or random vandals are opposed to bitcoin mining, and miners
have to have special configuration on their nodes that distinguish them
from regular users, then perhaps that makes it easier to find or shut
down their operations.
There are a few efficiencies to be gained from similar mempool policies as
well: more reliable compact block reconstruction (if you're not missing
any transactions, you avoid a round-trip) and presumably more efficient
set reconstruction with erlay. You'll also waste less bandwidth sending
transactions that the other node is only going to reject. Both those
depend on how many transactions are going to rely on unusual mempool
policies in the first place though.
ariard wrote:
I know I've advocated in the past to turn RBF support by default in
the past. Though after gathering a lot of feedbacks, this approach
of offering the policy flexiblity to the interested users only and
favoring a full-rbf gradual deployment sounds better to me.
- https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
I guess all the above leads me to think that gradual deployments of
mempool policies are likely the worse approach: even when they're not
hurting anyone, it makes them hard to use during the gradual phase,
and getting around that comes with worrying compromises on privacy and
centralisation; and when they are problematic for some, the indeterminate
nature of a gradual deployment means it's hard to plan for when that
risk is going to eventuate.
Theoretically, one way to recover the good parts of core deciding on
what's good for the network might be for people outside of core to
recommend a mempool configuration; then core can just have an option
to make that easy, similar to "-std=c++17" for a C++ compiler, and much
the same as knots' "-corepolicy" option.
Presuming anyone actually wants to take on that job, and listen to the
concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
and can come up with something that keeps most of them happy, and that
70% or 90% of the network ends up just following those recommendations
because it's easy, it works, and it's recommended by all the apps they
want to use, then that could work great:
* miners don't need to do anything special, so there's no new
mining centralisation pressure
* miners and users don't reveal what they're doing with bitcoin by the way
they configure their nodes, so there's no privacy problems
* devs can be fairly confident in how they have to design their apps
in order to get their transactions to most hashpower
* devs don't have to add new p2p layers to make it happen
* at least there's someone to talk to when you're trying to figure out
how to make some new project possible when it's inhibited by current
relay policies and you don't have to try to convince everyone to
upgrade on your own
* core devs just provide options, and don't have to worry about being
seen as gatekeepers
The "downside" in that scenario is that users/dev aren't making much
actual use of all the choices core is offering by making different
options available; but the upside is that that choice is at least readily
available should whoever is coming up with these policy become out of
step with what people actually want.
One thing that might make an approach like that difficult is that core
has historically been happy to remove options that don't seem useful
anymore: eg the ability to turn of BIP 125 support (#16171), and priority
transactions (#9602). Perhaps that's fine if you're trying to actively
craft a single mempool/relay policy that's good enough for almost everyone
(after all, it makes the code simpler and more efficient, and reduces
the number of footguns); all you're doing is leaving a minority of people
who want weird things to run a fork, and that's going to happen anyway.
But if people are following policy developed outside of core, core
might well disagree with them and decide "no that's a stupid policy,
no one should do that" and remove some feature that others thing should
continue to be normal. Beyond the examples above, there's already talk of
removing the ability to disable fullrbf support in #26305, for instance.
If that happens, then the people maintaining the policy will instead
end up maintaining an entire fork of bitcoin core, and all we've done
is transition to people running software from a different repo, and a
different set of maintainers.
If we're really going to a world where core's eager to add new options,
and reluctant to remove them, at least if anyone at all finds them
interesting, that's presumably a non-issue, though.
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
@ 2022-10-27 12:36 ` Gloria Zhao
2022-10-27 15:37 ` Anthony Towns
2022-10-27 13:49 ` Greg Sanders
` (3 subsequent siblings)
4 siblings, 1 reply; 33+ messages in thread
From: Gloria Zhao @ 2022-10-27 12:36 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 25110 bytes --]
Hi AJ,
Not going to comment on what Bitcoin Core's philosophy on mempol policy is
or should be. I want to note that I think this:
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case
is true.
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance.
Yes, in most cases, whether Bitcoin Core is restricting or loosening
policy, the user in question is fine as long as they have a path from their
node to a miner that will accept it. This is the case for something like
-datacarriersize if the use case is putting stuff into OP_RETURN outputs,
or if they're LN and using CPFP carveout, v3, package relay, etc. But
replacement is not only a question of "will my transaction propagate" but
also, "will someone else's transaction propagate, invalidating mine" or, in
other words, "can I prevent someone else's transaction from propagating." A
zeroconf user relies on there *not* being a path from someone else's full
RBF node to a full RBF miner. This is why I think RBF is so controversial
in general, why -mempoolfullrbf on someone else's node is considered more
significant than another policy option, and why full RBF shouldn't be
compared with something like datacarriersize. I don't think past patterns
can be easily applied here, and I don't think this necessarily shows a
different "direction" in thinking about mempool policy in general.
Best,
Gloria
On Thu, Oct 27, 2022 at 12:52 AM Anthony Towns via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
>
> And other plausible options aren't configurable even at compile time
> -- eg, core doesn't implement BIP 125's inherited signalling rule so
> there's no way to enable it; core doesn't allow opting out of BIP 125
> rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
> more than 1 ancestor; core doesn't allow opting out of LOW_S checks
> (even via -acceptnonstdtxn); etc.
>
> We also naturally have different mempool policies between different
> releases: eg, expansions of policy, such as allowing OP_RETURN or
> expanding it from 40 to 80 bytes or new soft forks where old nodes won't
> relay transactions that use the new; and also occassional restrictions
> in policy, such as the LOW_S requirement.
>
>
> While supporting and encouraging different mempool polices might be new
> for core, it's not new for knots: knots changes some of these defaults
> (-permitbaremultisig defaults to false, -datacarriersize is reduced to
> 42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
> options including -spkreuse and -mempoolreplacement (giving the latter
> full rbf behaviour by default). Knots also includes a `-corepolicy`
> option to make it easy to get a configuration matching core's defaults.
>
>
> I think gmaxwell's take from Feb 2015 (in the context of how restrictive
> policy on OP_RETURN data should be) was a reasonable description for
> core's approach up until now:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works. E.g. In stealth addresses the early proposals
> use highly inefficient single ECDH point per output instead of simply
> pooling them. Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public". No one wants to be an outright gatekeeper,
> but the network is a shared resource and it's perfectly reasonable
> node behavior to be stingy about the perpetual storage impact of the
> transactions they're willing to process, especially when it comes to
> neutral technical criteria like the amount of network irrelevant data
> stuffed in transactions.
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it. [...
> ...] For mitigating these risks it's optimal if transactions
> seem as uniform and indistinguishable as reasonably possible.
>
> - https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
>
> Perhaps see also sdaftuar in Nov 2015,
>
> To me the most important question is, is priority something that miners
> want to use?
>
> If a non-negligible amount of hashpower intends to use it in their
> transaction selection, then I think it makes sense for nodes to use it
> too, because it's generally helpful to have your mempool predict the
> UTXO as much as possible, and for nodes to be able to have reasonable
> fee and priority estimates (which won't happen unless they track the
> priority transactions somehow -- I'm presuming that miners run with
> much bigger mempools than regular nodes).
>
> If the answer is no, then that's fine and I don't see a reason to push
> in this direction. I sort of assumed there was enough hashpower mining
> with priority, since last time I checked estimatepriority was still
> giving meaningful results for low-ish blockheights, but I haven't done
> any kind of real analysis.
>
> - https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
>
> or in June 2019,
>
> What this PR is proposing is to get rid of a command-line option that is
> (a) a footgun for users and (b) does not reflect what I believe to be
> the understanding most users have, which is that [X txs] are expected
> to propagate well on the network.
>
> ..
>
> I don't think this rises to the level that Luke is concerned about,
> namely a prelude to forcing a common relay policy on all nodes. In
> particular I do agree it makes sense that we offer some ways of
> customizing policy parameters (eg the mempool size, min relay fee,
> etc). Instead, I think the justification for this change is that we
> should not support behaviors we think are harmful to the ecosystem
> overall and have no legitimate use-case, and we should eliminate ways
> that users might inadvertently shoot themselves in the foot.
>
> - https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
>
> (or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
>
> I don't mean to imply the above are saying "there's one way to do
> things and it's this way", or that the old way of doing things should
> necessarily be the way we keep doing things. Just that previously core
> has tended towards designing a single policy that works as well as it
> can for everyone and the ecosystem as a whole. (I'm also not saying that
> fullrbf can't work well for everyone or the ecosystem as a whole)
>
>
> By contrast, I think the most common response to pushback against the
> full rbf option has been along the lines of "it's just an option, we
> don't want to force people", eg:
>
> Blaming the default false -mempoolfullrbf option for a full RBF network
> would be holding Bitcoin Core developers responsible for the decisions
> of individual node operators and miners. I don't think having the
> option (again, default false) can directly cause a full RBF network,
> and likewise, I don't think removing this option removes the "risk"
> of a full RBF network.
> - glozow
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
>
> NACK. This is a default false option.
> - achow101
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
>
> Erecting artificial barriers to prevent or make it difficult for users
> to do what they want to do, is not appropriate behaviour.
> - luke-jr
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
>
> I'm in general against removing options.
> - instagibbs
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
>
> I think this differs from what core has done in the past, in that
> previously we've tried to ensure a new policy is good for everyone (or as
> nearly as it can be), and then enabled it as soon as it's implemented.
> Any options that have been added have either been to control resource
> usage in ways that don't significantly effect tx propagation, to
> allow people to revert to the old behaviour when the new behaviour is
> controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
> and to make it easier to test/debug the implementation.
>
> Giving people a new relay behaviour they can opt-in to when we aren't
> confident enough to turn on by default doesn't match the approach I've
> seen core take in the past.
>
>
> If this is going to be an ongoing shift in how core sees relay/mempool
> policy, I think that's significant and worth paying attention to.
>
> I don't think it's necessary to have that shift to roll out full rbf.
> The other approach would be either:
>
> * set -mempoolfullrbf=true as the default for 24.0, and just have the
> command line param there in case people want to do a
> "UserRejectedMempoolPolicy" campaign to get everyone to opt-out
>
> * revert it for now because we don't think mainnet is ready for fullrbf
> yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
> to activate at some scheduled date in that timeframe (potentially
> backporting it to previous releases to help with adoption too,
> whatever). same effect as the previous option, just with a bit more
> advanced notice and time to prepare
>
> I don't think anyone's proposed the first (which I interpret as "most of
> us don't think mainnet is ready for fullrbf today"), but the comments
> above are all pushback by people arguing against (the first step of)
> the second approach, and they seem to be winning the day.
>
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case and
> not a precedent, and when people propose other default false options,
> there will be substantially more resistance to them being merged,
> despite all the talk about users having options that's going on right now.
>
>
> Assuming it is the change of direction it appears to be -- and all of
> the above is really just justification for that assumption -- then like
> I said, I think it's worth seriously considering what it means for people
> to choose their own relay/mempool policies and for you to expect to have
> different mempool policies to many or most of your potential peers.
>
>
> One thing maybe worth noting is that is that you can still only choose
> your policy from options that people write code for -- if it wasn't
> something you could get by running knots or compiling a rejected PR
> yourself, it won't magically become more possible now. Presumably it
> would mean that once a PR is written, it might get better review (rather
> than being dismissed as not suitable for everyone), and there would be
> less maintenance burden than if it had to be manually rebased every
> release, though (or at least the maintenance burden would be shared
> across everyone working on the codebase).
>
>
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
>
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance. If erlay allows increasing your outbound count to
> 12 connections instead of 8; that might reduce down to needing just 20%
> of listening nodes (~93%).
>
> But for cases where less than 30% (20%) of network supports your preferred
> policy, you probably need to do something cleverer.
>
> One approach is to set a service bit and preferentially peer with other
> nodes that advertise that service bit; knots does the first half of this
> for fullrbf, and both halves have been proposed for core in #25600.
> Preferential peering was previously done for the segwit deployment,
> though in that case it was necessary not just for tx propogation but
> also for ensuring block propogation, making it effectively a consensus
> critical issue.
>
> Another approach is having a separate relay network -- eg, lightning nodes
> already have a gossip network, and might want to help their own ecosystem
> by ensuring unilateral channel closes and justice transactions are quickly
> relayed. Using their own gossip network to relay the transaction around,
> and each lightning node adding it to their local bitcoind's mempool and
> allowing it to propogate (or not) from there as normal, would also be a
> way of allowing transactions to propogate well. It does mean that miners
> would either need to also participate in lightning gossip directly, or
> that miners would need to connect to *many* peers to be confident of
> seeing those transactions (eg, if only 2% of the network would see a
> tx, you'd need to make 228 connections to have a 99% chance of seeing
> the tx). You can't currently do something like this, because all the
> relay policies are also applied when adding txs to the mempool via RPC,
> and there's no convenient way to remove txs from the mempool.
>
> A case where something like that might occur is in preventing L2
> transactions from pinning attacks -- so you might have a high-fee,
> low-feerate transaction that's been widely propogated, sitting in the
> bottom of people's mempools, and you want to replace it with a smaller,
> higher-feerate transaction, but don't want to pay a higher absolute fee,
> and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
> unwilling to deviate from BIP 125 rule 3 for you; because that would
> make it easy for random griefers to spam their mempool with large txs
> then delete them while only paying a small fee; but your L2 peers may be
> able to decode your replacement transaction and be sure that you aren't
> going to spam them, and thus will happily relay it.
>
> From a technical point-of-view, that's largely fine; the downside is it
> increases the centralisation pressure on mining: whether that's by having
> to connect to substantially more nodes, or having to parse through more
> spam, you can't just run your mining operation off a standard install
> of bitcoin core anymore, but need to actively opt-in to find all the
> weird unusual ways people are sending transactions around in order to
> actually collect as much in fees as your competitors are.
>
> That's probably moderately bad for privacy as well -- if lightning or
> coinjoins need special relay rules that most nodes haven't opted into,
> it's potentially easy to use that to find the bitcoin nodes on the
> network that are participating in those protocols, and from there to
> either identify the operator, or run a DoS attack to make it hard for you
> to keep doing what you want. Obviously if you're setting a service bit to
> get better routing, you've given up that privacy already. Likewise if the
> government or random vandals are opposed to bitcoin mining, and miners
> have to have special configuration on their nodes that distinguish them
> from regular users, then perhaps that makes it easier to find or shut
> down their operations.
>
> There are a few efficiencies to be gained from similar mempool policies as
> well: more reliable compact block reconstruction (if you're not missing
> any transactions, you avoid a round-trip) and presumably more efficient
> set reconstruction with erlay. You'll also waste less bandwidth sending
> transactions that the other node is only going to reject. Both those
> depend on how many transactions are going to rely on unusual mempool
> policies in the first place though.
>
> ariard wrote:
>
> I know I've advocated in the past to turn RBF support by default in
> the past. Though after gathering a lot of feedbacks, this approach
> of offering the policy flexiblity to the interested users only and
> favoring a full-rbf gradual deployment sounds better to me.
>
> - https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
>
> I guess all the above leads me to think that gradual deployments of
> mempool policies are likely the worse approach: even when they're not
> hurting anyone, it makes them hard to use during the gradual phase,
> and getting around that comes with worrying compromises on privacy and
> centralisation; and when they are problematic for some, the indeterminate
> nature of a gradual deployment means it's hard to plan for when that
> risk is going to eventuate.
>
>
> Theoretically, one way to recover the good parts of core deciding on
> what's good for the network might be for people outside of core to
> recommend a mempool configuration; then core can just have an option
> to make that easy, similar to "-std=c++17" for a C++ compiler, and much
> the same as knots' "-corepolicy" option.
>
> Presuming anyone actually wants to take on that job, and listen to the
> concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
> and can come up with something that keeps most of them happy, and that
> 70% or 90% of the network ends up just following those recommendations
> because it's easy, it works, and it's recommended by all the apps they
> want to use, then that could work great:
>
> * miners don't need to do anything special, so there's no new
> mining centralisation pressure
> * miners and users don't reveal what they're doing with bitcoin by the way
> they configure their nodes, so there's no privacy problems
> * devs can be fairly confident in how they have to design their apps
> in order to get their transactions to most hashpower
> * devs don't have to add new p2p layers to make it happen
> * at least there's someone to talk to when you're trying to figure out
> how to make some new project possible when it's inhibited by current
> relay policies and you don't have to try to convince everyone to
> upgrade on your own
> * core devs just provide options, and don't have to worry about being
> seen as gatekeepers
>
> The "downside" in that scenario is that users/dev aren't making much
> actual use of all the choices core is offering by making different
> options available; but the upside is that that choice is at least readily
> available should whoever is coming up with these policy become out of
> step with what people actually want.
>
> One thing that might make an approach like that difficult is that core
> has historically been happy to remove options that don't seem useful
> anymore: eg the ability to turn of BIP 125 support (#16171), and priority
> transactions (#9602). Perhaps that's fine if you're trying to actively
> craft a single mempool/relay policy that's good enough for almost everyone
> (after all, it makes the code simpler and more efficient, and reduces
> the number of footguns); all you're doing is leaving a minority of people
> who want weird things to run a fork, and that's going to happen anyway.
>
> But if people are following policy developed outside of core, core
> might well disagree with them and decide "no that's a stupid policy,
> no one should do that" and remove some feature that others thing should
> continue to be normal. Beyond the examples above, there's already talk of
> removing the ability to disable fullrbf support in #26305, for instance.
> If that happens, then the people maintaining the policy will instead
> end up maintaining an entire fork of bitcoin core, and all we've done
> is transition to people running software from a different repo, and a
> different set of maintainers.
>
> If we're really going to a world where core's eager to add new options,
> and reluctant to remove them, at least if anyone at all finds them
> interesting, that's presumably a non-issue, though.
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 29414 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
2022-10-27 12:36 ` Gloria Zhao
@ 2022-10-27 13:49 ` Greg Sanders
2022-10-27 15:00 ` Peter Todd
2022-10-27 20:29 ` Antoine Riard
` (2 subsequent siblings)
4 siblings, 1 reply; 33+ messages in thread
From: Greg Sanders @ 2022-10-27 13:49 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 24156 bytes --]
To add a wrinkle, or possibly a confirmation of your long message, up to
readers to decipher, there historically has been at least one more RBF
related option that was included, then removed later in Core.
Introduced as "permitrbf" in b768108d9c0b83330572711aef1e569543130d5e with
default "true", later renamed to "replacebyfee", then finally as
"mempoolreplacement".
It was later removed in 8053e5cdade87550f0381d51feab81dedfec6c46 with the
message:
"
Remove -mempoolreplacement to prevent needless block prop slowness.
At this point there is no reasonable excuse to disable opt-in RBF,
and, unlike when this option was added, there are now significant
issues created when disabling it (in the form of compact block
reconstruction failures). Further, it breaks a lot of modern wallet
behavior.
"
So there is some precedence to including an option that protocol devs don't
find useful, then removing it N years later to make sure it doesn't impact
compact blocks.
Peering into the "precedence" lense, I think this does lend itself to the
theory that the transition should be as uniform as possible to avoid
degradation of fast block propagation. If not removing options(which is
deemed user hostile by a number of folks including me), then at least for a
flag day switchover.
Cheers,
Greg
On Wed, Oct 26, 2022 at 7:53 PM Anthony Towns via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
>
> And other plausible options aren't configurable even at compile time
> -- eg, core doesn't implement BIP 125's inherited signalling rule so
> there's no way to enable it; core doesn't allow opting out of BIP 125
> rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
> more than 1 ancestor; core doesn't allow opting out of LOW_S checks
> (even via -acceptnonstdtxn); etc.
>
> We also naturally have different mempool policies between different
> releases: eg, expansions of policy, such as allowing OP_RETURN or
> expanding it from 40 to 80 bytes or new soft forks where old nodes won't
> relay transactions that use the new; and also occassional restrictions
> in policy, such as the LOW_S requirement.
>
>
> While supporting and encouraging different mempool polices might be new
> for core, it's not new for knots: knots changes some of these defaults
> (-permitbaremultisig defaults to false, -datacarriersize is reduced to
> 42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
> options including -spkreuse and -mempoolreplacement (giving the latter
> full rbf behaviour by default). Knots also includes a `-corepolicy`
> option to make it easy to get a configuration matching core's defaults.
>
>
> I think gmaxwell's take from Feb 2015 (in the context of how restrictive
> policy on OP_RETURN data should be) was a reasonable description for
> core's approach up until now:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works. E.g. In stealth addresses the early proposals
> use highly inefficient single ECDH point per output instead of simply
> pooling them. Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public". No one wants to be an outright gatekeeper,
> but the network is a shared resource and it's perfectly reasonable
> node behavior to be stingy about the perpetual storage impact of the
> transactions they're willing to process, especially when it comes to
> neutral technical criteria like the amount of network irrelevant data
> stuffed in transactions.
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it. [...
> ...] For mitigating these risks it's optimal if transactions
> seem as uniform and indistinguishable as reasonably possible.
>
> - https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
>
> Perhaps see also sdaftuar in Nov 2015,
>
> To me the most important question is, is priority something that miners
> want to use?
>
> If a non-negligible amount of hashpower intends to use it in their
> transaction selection, then I think it makes sense for nodes to use it
> too, because it's generally helpful to have your mempool predict the
> UTXO as much as possible, and for nodes to be able to have reasonable
> fee and priority estimates (which won't happen unless they track the
> priority transactions somehow -- I'm presuming that miners run with
> much bigger mempools than regular nodes).
>
> If the answer is no, then that's fine and I don't see a reason to push
> in this direction. I sort of assumed there was enough hashpower mining
> with priority, since last time I checked estimatepriority was still
> giving meaningful results for low-ish blockheights, but I haven't done
> any kind of real analysis.
>
> - https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
>
> or in June 2019,
>
> What this PR is proposing is to get rid of a command-line option that is
> (a) a footgun for users and (b) does not reflect what I believe to be
> the understanding most users have, which is that [X txs] are expected
> to propagate well on the network.
>
> ..
>
> I don't think this rises to the level that Luke is concerned about,
> namely a prelude to forcing a common relay policy on all nodes. In
> particular I do agree it makes sense that we offer some ways of
> customizing policy parameters (eg the mempool size, min relay fee,
> etc). Instead, I think the justification for this change is that we
> should not support behaviors we think are harmful to the ecosystem
> overall and have no legitimate use-case, and we should eliminate ways
> that users might inadvertently shoot themselves in the foot.
>
> - https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
>
> (or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
>
> I don't mean to imply the above are saying "there's one way to do
> things and it's this way", or that the old way of doing things should
> necessarily be the way we keep doing things. Just that previously core
> has tended towards designing a single policy that works as well as it
> can for everyone and the ecosystem as a whole. (I'm also not saying that
> fullrbf can't work well for everyone or the ecosystem as a whole)
>
>
> By contrast, I think the most common response to pushback against the
> full rbf option has been along the lines of "it's just an option, we
> don't want to force people", eg:
>
> Blaming the default false -mempoolfullrbf option for a full RBF network
> would be holding Bitcoin Core developers responsible for the decisions
> of individual node operators and miners. I don't think having the
> option (again, default false) can directly cause a full RBF network,
> and likewise, I don't think removing this option removes the "risk"
> of a full RBF network.
> - glozow
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
>
> NACK. This is a default false option.
> - achow101
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
>
> Erecting artificial barriers to prevent or make it difficult for users
> to do what they want to do, is not appropriate behaviour.
> - luke-jr
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
>
> I'm in general against removing options.
> - instagibbs
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
>
> I think this differs from what core has done in the past, in that
> previously we've tried to ensure a new policy is good for everyone (or as
> nearly as it can be), and then enabled it as soon as it's implemented.
> Any options that have been added have either been to control resource
> usage in ways that don't significantly effect tx propagation, to
> allow people to revert to the old behaviour when the new behaviour is
> controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
> and to make it easier to test/debug the implementation.
>
> Giving people a new relay behaviour they can opt-in to when we aren't
> confident enough to turn on by default doesn't match the approach I've
> seen core take in the past.
>
>
> If this is going to be an ongoing shift in how core sees relay/mempool
> policy, I think that's significant and worth paying attention to.
>
> I don't think it's necessary to have that shift to roll out full rbf.
> The other approach would be either:
>
> * set -mempoolfullrbf=true as the default for 24.0, and just have the
> command line param there in case people want to do a
> "UserRejectedMempoolPolicy" campaign to get everyone to opt-out
>
> * revert it for now because we don't think mainnet is ready for fullrbf
> yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
> to activate at some scheduled date in that timeframe (potentially
> backporting it to previous releases to help with adoption too,
> whatever). same effect as the previous option, just with a bit more
> advanced notice and time to prepare
>
> I don't think anyone's proposed the first (which I interpret as "most of
> us don't think mainnet is ready for fullrbf today"), but the comments
> above are all pushback by people arguing against (the first step of)
> the second approach, and they seem to be winning the day.
>
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case and
> not a precedent, and when people propose other default false options,
> there will be substantially more resistance to them being merged,
> despite all the talk about users having options that's going on right now.
>
>
> Assuming it is the change of direction it appears to be -- and all of
> the above is really just justification for that assumption -- then like
> I said, I think it's worth seriously considering what it means for people
> to choose their own relay/mempool policies and for you to expect to have
> different mempool policies to many or most of your potential peers.
>
>
> One thing maybe worth noting is that is that you can still only choose
> your policy from options that people write code for -- if it wasn't
> something you could get by running knots or compiling a rejected PR
> yourself, it won't magically become more possible now. Presumably it
> would mean that once a PR is written, it might get better review (rather
> than being dismissed as not suitable for everyone), and there would be
> less maintenance burden than if it had to be manually rebased every
> release, though (or at least the maintenance burden would be shared
> across everyone working on the codebase).
>
>
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
>
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance. If erlay allows increasing your outbound count to
> 12 connections instead of 8; that might reduce down to needing just 20%
> of listening nodes (~93%).
>
> But for cases where less than 30% (20%) of network supports your preferred
> policy, you probably need to do something cleverer.
>
> One approach is to set a service bit and preferentially peer with other
> nodes that advertise that service bit; knots does the first half of this
> for fullrbf, and both halves have been proposed for core in #25600.
> Preferential peering was previously done for the segwit deployment,
> though in that case it was necessary not just for tx propogation but
> also for ensuring block propogation, making it effectively a consensus
> critical issue.
>
> Another approach is having a separate relay network -- eg, lightning nodes
> already have a gossip network, and might want to help their own ecosystem
> by ensuring unilateral channel closes and justice transactions are quickly
> relayed. Using their own gossip network to relay the transaction around,
> and each lightning node adding it to their local bitcoind's mempool and
> allowing it to propogate (or not) from there as normal, would also be a
> way of allowing transactions to propogate well. It does mean that miners
> would either need to also participate in lightning gossip directly, or
> that miners would need to connect to *many* peers to be confident of
> seeing those transactions (eg, if only 2% of the network would see a
> tx, you'd need to make 228 connections to have a 99% chance of seeing
> the tx). You can't currently do something like this, because all the
> relay policies are also applied when adding txs to the mempool via RPC,
> and there's no convenient way to remove txs from the mempool.
>
> A case where something like that might occur is in preventing L2
> transactions from pinning attacks -- so you might have a high-fee,
> low-feerate transaction that's been widely propogated, sitting in the
> bottom of people's mempools, and you want to replace it with a smaller,
> higher-feerate transaction, but don't want to pay a higher absolute fee,
> and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
> unwilling to deviate from BIP 125 rule 3 for you; because that would
> make it easy for random griefers to spam their mempool with large txs
> then delete them while only paying a small fee; but your L2 peers may be
> able to decode your replacement transaction and be sure that you aren't
> going to spam them, and thus will happily relay it.
>
> From a technical point-of-view, that's largely fine; the downside is it
> increases the centralisation pressure on mining: whether that's by having
> to connect to substantially more nodes, or having to parse through more
> spam, you can't just run your mining operation off a standard install
> of bitcoin core anymore, but need to actively opt-in to find all the
> weird unusual ways people are sending transactions around in order to
> actually collect as much in fees as your competitors are.
>
> That's probably moderately bad for privacy as well -- if lightning or
> coinjoins need special relay rules that most nodes haven't opted into,
> it's potentially easy to use that to find the bitcoin nodes on the
> network that are participating in those protocols, and from there to
> either identify the operator, or run a DoS attack to make it hard for you
> to keep doing what you want. Obviously if you're setting a service bit to
> get better routing, you've given up that privacy already. Likewise if the
> government or random vandals are opposed to bitcoin mining, and miners
> have to have special configuration on their nodes that distinguish them
> from regular users, then perhaps that makes it easier to find or shut
> down their operations.
>
> There are a few efficiencies to be gained from similar mempool policies as
> well: more reliable compact block reconstruction (if you're not missing
> any transactions, you avoid a round-trip) and presumably more efficient
> set reconstruction with erlay. You'll also waste less bandwidth sending
> transactions that the other node is only going to reject. Both those
> depend on how many transactions are going to rely on unusual mempool
> policies in the first place though.
>
> ariard wrote:
>
> I know I've advocated in the past to turn RBF support by default in
> the past. Though after gathering a lot of feedbacks, this approach
> of offering the policy flexiblity to the interested users only and
> favoring a full-rbf gradual deployment sounds better to me.
>
> - https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
>
> I guess all the above leads me to think that gradual deployments of
> mempool policies are likely the worse approach: even when they're not
> hurting anyone, it makes them hard to use during the gradual phase,
> and getting around that comes with worrying compromises on privacy and
> centralisation; and when they are problematic for some, the indeterminate
> nature of a gradual deployment means it's hard to plan for when that
> risk is going to eventuate.
>
>
> Theoretically, one way to recover the good parts of core deciding on
> what's good for the network might be for people outside of core to
> recommend a mempool configuration; then core can just have an option
> to make that easy, similar to "-std=c++17" for a C++ compiler, and much
> the same as knots' "-corepolicy" option.
>
> Presuming anyone actually wants to take on that job, and listen to the
> concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
> and can come up with something that keeps most of them happy, and that
> 70% or 90% of the network ends up just following those recommendations
> because it's easy, it works, and it's recommended by all the apps they
> want to use, then that could work great:
>
> * miners don't need to do anything special, so there's no new
> mining centralisation pressure
> * miners and users don't reveal what they're doing with bitcoin by the way
> they configure their nodes, so there's no privacy problems
> * devs can be fairly confident in how they have to design their apps
> in order to get their transactions to most hashpower
> * devs don't have to add new p2p layers to make it happen
> * at least there's someone to talk to when you're trying to figure out
> how to make some new project possible when it's inhibited by current
> relay policies and you don't have to try to convince everyone to
> upgrade on your own
> * core devs just provide options, and don't have to worry about being
> seen as gatekeepers
>
> The "downside" in that scenario is that users/dev aren't making much
> actual use of all the choices core is offering by making different
> options available; but the upside is that that choice is at least readily
> available should whoever is coming up with these policy become out of
> step with what people actually want.
>
> One thing that might make an approach like that difficult is that core
> has historically been happy to remove options that don't seem useful
> anymore: eg the ability to turn of BIP 125 support (#16171), and priority
> transactions (#9602). Perhaps that's fine if you're trying to actively
> craft a single mempool/relay policy that's good enough for almost everyone
> (after all, it makes the code simpler and more efficient, and reduces
> the number of footguns); all you're doing is leaving a minority of people
> who want weird things to run a fork, and that's going to happen anyway.
>
> But if people are following policy developed outside of core, core
> might well disagree with them and decide "no that's a stupid policy,
> no one should do that" and remove some feature that others thing should
> continue to be normal. Beyond the examples above, there's already talk of
> removing the ability to disable fullrbf support in #26305, for instance.
> If that happens, then the people maintaining the policy will instead
> end up maintaining an entire fork of bitcoin core, and all we've done
> is transition to people running software from a different repo, and a
> different set of maintainers.
>
> If we're really going to a world where core's eager to add new options,
> and reluctant to remove them, at least if anyone at all finds them
> interesting, that's presumably a non-issue, though.
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 28286 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 13:49 ` Greg Sanders
@ 2022-10-27 15:00 ` Peter Todd
0 siblings, 0 replies; 33+ messages in thread
From: Peter Todd @ 2022-10-27 15:00 UTC (permalink / raw)
To: Greg Sanders, Bitcoin Protocol Discussion; +Cc: Anthony Towns
[-- Attachment #1: Type: text/plain, Size: 1843 bytes --]
On Thu, Oct 27, 2022 at 09:49:48AM -0400, Greg Sanders via bitcoin-dev wrote:
> So there is some precedence to including an option that protocol devs don't
> find useful, then removing it N years later to make sure it doesn't impact
> compact blocks.
I think the lesson there is we're willing to remove options that are
ridiculous. Replacements are widely used, and downright essential in high-fee
situations.
> Peering into the "precedence" lense, I think this does lend itself to the
> theory that the transition should be as uniform as possible to avoid
> degradation of fast block propagation. If not removing options(which is
> deemed user hostile by a number of folks including me), then at least for a
> flag day switchover.
Re: compact blocks, note that RBF is a special case: for the sake of
reconstruction, it'd make sense to temporarily cache transactions have have
been replaced rather than discarding them entirely, in case a prior version
gets mined. Irregardless of policy this will happen occasionally simple due to
propagation delays. Equally, if we cached transactions that we rejected due to
policy, that'd help with reconstruction success in the event that policy is
changing.
Anyway, since the compact blocks implementation efficiently deals with the case
where miners have policy that differs from most nodes, by immediately
forwarding missing transactions, I don't think the occasional full-rbf
replacement is going to have much impact. The nodes that had full-rbf disabled
will forward the tx to their peers directly, and then the subset of full-rbf
disabled peers will do the same again. So long as the network has a mix of both
types, and they're interconnected rather than in clusters, the latency impact
should be minimal.
--
https://petertodd.org 'peter'[:-1]@petertodd.org
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 12:36 ` Gloria Zhao
@ 2022-10-27 15:37 ` Anthony Towns
2022-10-27 18:17 ` Luke Dashjr
0 siblings, 1 reply; 33+ messages in thread
From: Anthony Towns @ 2022-10-27 15:37 UTC (permalink / raw)
To: Gloria Zhao; +Cc: Bitcoin Protocol Discussion
On Thu, Oct 27, 2022 at 01:36:47PM +0100, Gloria Zhao wrote:
> > The cutoff for that is probably something like "do 30% of listening
> > nodes have a compatible policy"? If they do, then you'll have about a
> > 95% chance of having at least one of your outbound peers accept your tx,
> > just by random chance.
> Yes, in most cases, whether Bitcoin Core is restricting or loosening
> policy, the user in question is fine as long as they have a path from their
> node to a miner that will accept it. This is the case for something like
> -datacarriersize if the use case is putting stuff into OP_RETURN outputs,
> or if they're LN and using CPFP carveout, v3, package relay, etc.
> But
> replacement is not only a question of "will my transaction propagate" but
> also, "will someone else's transaction propagate, invalidating mine" or, in
> other words, "can I prevent someone else's transaction from propagating."
"Can I prevent someone else's transaction from propagating" is almost
the entirety of the question with -datacarrier, -datacarriersize and
-permitbaremultisig though: "we" don't want people to spam the utxo
set or the blockchain with lots of data (cf BSV's gigabytes worth of
dog pictures [0]), so for the people who are going to find some way
of putting data in we'd like to encourage them to make it small, and
do it in a way that's prunable and doesn't bloat the utxo set, whose
size matters even more than the overall blockchain's size does. As I
understand it, people were doing that by creating bare multisig utxos,
ie a bare (non-p2sh) scriptPubKey that perhaps looks like:
1 my_key data1 data2 data3 data4 data5 5 CHECKMULTISIG
which is "bad" in two ways: you're only committing to the data, so why
not save 128 bytes by doing hash(data1 data2 data3 data4 data5) instead;
and even more so, that data is only interesting to you, not everyone else,
so why not do it in a way that doesn't bloat the utxo set, which we want
to keep small so that it's easier to efficiently look up potential spends.
Hence the -datacarriersize limitation that limits you to about 2.5
"dataN" entries per tx ("we'll prevent your tx from propagating if you
do much more than publish a hash") and hence at least the potential for
doing the same for baremultisig in general.
[0] https://twitter.com/bsvdata/status/1427866510035324936
> A
> zeroconf user relies on there *not* being a path from someone else's full
> RBF node to a full RBF miner. This is why I think RBF is so controversial
> in general,
Yes; but I think it's also true to say that this is why *zeroconf* is as
controversial as it is.
Likewise OP_RETURN has had its own "controversies" to some extent, too:
https://blog.bitmex.com/dapps-or-only-bitcoin-transactions-the-2014-debate/
https://github.com/bitcoin/bitcoin/pull/3715
https://github.com/bitcoin/bitcoin/pull/3939
> why -mempoolfullrbf on someone else's node is considered more
> significant than another policy option, and why full RBF shouldn't be
> compared with something like datacarriersize.
It's definitely a different scenario: unexpected RBF can cause you to
have less money than you expected; whereas more OP_RETURN data just
bloats the blockchain, and losing money that you thought was yours is
definitely more painful than more spam.
But while the level of pain is different; I don't think the mechanism is:
whether you're trying to preserve zeroconf or prevent utxo set spam,
you're still relying on a vast majority of nodes working together to
prevent even a small minority of hashpower from doing "bad" things, with
no cryptographic assurances that will continue to work well or at all.
> I don't think past patterns can be easily applied here,
I mean, technically they trivially could? We *could* roll out support for
full RBF in exactly the same way we rolled out support for opt-in RBF:
making it the default for all nodes, but supplying an option that node
operators can use to disable the feature for seven releases / ~4 years:
https://bitcoin.org/en/release/v0.12.0#opt-in-replace-by-fee-transactions
https://bitcoin.org/en/release/v0.19.0.1#deprecated-or-removed-configuration-options
If we don't want to do that immediately, but also want to make a definite
move forward, then we could:
* just say that, and then keep our word about it
* keep the feature in master, but remove it in 24.x
* put a time delay on the feature so that it doesn't happen immediately
but is locked in in the code for whenever we are ready to do it
> and I don't think this necessarily shows a
> different "direction" in thinking about mempool policy in general.
If we're not applying past patterns, then this is a different direction
in how we're thinking about things than what we did in the past.
That's not necessarily a bad thing -- maybe we should be thinking
differently; but I don't see how you can honestly dispute it: those are
just two ways of saying the exact same thing.
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 15:37 ` Anthony Towns
@ 2022-10-27 18:17 ` Luke Dashjr
0 siblings, 0 replies; 33+ messages in thread
From: Luke Dashjr @ 2022-10-27 18:17 UTC (permalink / raw)
To: bitcoin-dev, Anthony Towns
More generally, some of the arguments against full RBF seem like debatable
reasons (though not fully convincing) to possibly leave it off, and/or
disabled by default, but definitely NOT reasons to remove the option and
prevent users from deciding for themselves.
On Thursday 27 October 2022 15:37:27 Anthony Towns via bitcoin-dev wrote:
> "Can I prevent someone else's transaction from propagating" is almost
> the entirety of the question with -datacarrier, -datacarriersize and
> -permitbaremultisig though:
Not necessarily the entirety, no. Even if others would propagate it, you also
don't want to waste _your_ bandwidth doing so. This also reveals a difference
between the two policies: with RBF, you have _already_ spent resources
propagating the first transaction (what this implies is not immediately
obvious).
Luke
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
2022-10-27 12:36 ` Gloria Zhao
2022-10-27 13:49 ` Greg Sanders
@ 2022-10-27 20:29 ` Antoine Riard
2022-10-30 2:24 ` Anthony Towns
2022-10-29 7:45 ` David A. Harding
2022-10-31 13:02 ` Suhas Daftuar
4 siblings, 1 reply; 33+ messages in thread
From: Antoine Riard @ 2022-10-27 20:29 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 30205 bytes --]
Hi AJ,
Let's take the contra.
I would say the current post describes the state of Bitcoin Core and
beyond policy
rules with a high-degree of exhaustivity and completeness, though itt what
is, mostly a description. While I think it ends with a set of
recommendations on what could be the relations between miners, devs and
node operators w.r.t adopting policy rules, to me it doesn't explore enough
the hard part of the subject. What should be actually the design goals and
principles of Core's transaction-relay propagation rules
of which mempool accepts ones is a subset ? By such design goals, I'm
thinking either, a qualitative framework, like attacks game for a concrete
application ("Can
we prevent pinning against multi-party Coinjoin ?"). Or a quantitative
approach, e.g how fast should be transaction-relay throughput for a given
topology, or how much accurate is the block template against usual blocks
[0]
Assuming we would have technical principles and goals guiding our
development process, of which the discussion and revision should be an
inherent part of the process itself, I believe we would come up with a
second-order observation. That we might not be able to satisfy every
use-case with the standard set of policy rules. E.g, a contracting protocol
could look for package size beyond the upper bound anti-Dos limit. Or even the
global ressources offered by the network of full-nodes are not high enough
to handle some application event. E.g a Lightning Service Provider doing a
liquidity maintenance round of all its counterparties, and as such
force-closing and broadcasting more transactions than can be handled at the
transaction-relay layer due to default MAX_PEER_TX_ANNOUNCEMENTS value.
My personal take on those subjects, we might have to realize we're facing
an heterogeneity of Bitcoin applications and use-cases [1]. And this sounds
like a long-term upward trend, akin to the history of the Internet: mail
clients, web browser, streaming applications, etc, all with different
service-level requirements in terms of latency, jitters and bandwidth. We might
observe that Bitcoin actors might adopt individual strategies for the
consumption of it. E.g a multi-party contracting protocol with asymmetric
states, a counterparty could "front-run" the others at the
transaction-relay to avoid encumbering the RBF penalty in case of
concurrent broadcast. Or they could deploy an additive layer of
communication channels, like private transaction-relay to miners, e.g a LN
node broadcasting a HTLC-preimage and restraining the odds of mapping to an
off-chain payment path. Those additive layers of communication can be also
seen as mitigations against components failure risks, e.g a
transaction-relay
censorship vector in the p2p stack. To put it simply, some advanced Bitcoin
applications might have to outgrow the "mempool policy rules" game, as
we're currently defining it to satisfy to a higher degree their security
and operational models. I think this has been historically the case with
some miners deciding to join FIBER, to improve their view of mined blocks.
While I still think we can come up with clear policy rules for some ideal
or reasonable use-case (let's say the Bitcoin hobbyist aiming to strong
censorship-resistance of its
payment). And I hold the belief we can improve on that front, as we've done
during the past years by formalizing a subset of policy rules in Bitcoin
Core documentation. In
parallel, we might acknowledge the heterogeneity of use-cases, and that we
cannot offer the same level of censorship-resistance, privacy or
cost-effectiveness (And that might be okay if we remember that e.g
censorship-resistance between a simple Bitcoin payment and a Lightning HTLC
isn't defined the same, due to the usage of timelocks by the latter).
What I'm expressing is a long-term perspective, and we might be too early
in the evolutionary process that is Bitcoin Core development to abandon yet
the "one-size-fits-all"
policy rules conception that I understand from your post. Though to me, as
we progress on modularity and flexibility of the Core codebase, we might
have to envision a Bitcoin ecosystem where configuring a client is on the
same bar as the kernel Kconfig (hopefully not because no drivers/ but
still..). Of course, you can have "blessed" config to avoid the complexity
turning away node operators and users, though at the price of increasing
the trust in your vendors.
More concretely, about the issue of today, full-rbf, I think the lack of
design heuristic in the sense of what should be the reasonable or average
use-case
supported by the Bitcoin base-layer transaction-relay rules. Should we
favor the historical ones (0confs) or the new incumbent (contracting
protocols) ? There is always the "First, do not harm" principle in lack of
clear guidelines in any area of Bitcoin protocol development. Though when
we're facing risk arbitrage like the current one, I don't think as protocol
devs we have any strong reasoning framework. Deferring the decision to the
node operators as the approach is with #25353 + #25000, did acknowledge the
uncharted territory we're in with policy rules deprecation or novation.
Qualifying the gradual approach of "worst" sounds to me that every policy
rule will ever have to design or deploy will suffer from the same trade-off of
full-rbf. A gradual approach could be a really realistic way to move forward
with some experimental package formats or Dandelion-like transaction
dissemination strategy. Such gradual approach of full-rbf deployment was
also aiming to set the precedent that in a situation of risk arbitrage,
with irreconcilable views between class of use-cases operators, the
decision shouldn't be in the scope of Bitcoin Core.
Hopefully, with a renewed effort of communication in the case of full-rbf,
we can accommodate the maximum of use-cases risks model and move towards a
coordinated approach, offering a visible and predictable timeline. Though
if we fail to reach "consensus" there, we might have to fallback on the
"gradual" approach, stating the full-rbf case is beyond the responsibility of
scope of Bitcoin Core.
Overall, many thanks for this thoughtful post. Good for me to uplift the
level of the discussion when we're hitting some bottleneck in our
development process.
Best,
Antoine
[0] https://github.com/bitcoin/bitcoin/issues/19820
[1] To be clear, this the opposite view I did express in :
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-July/018063.html
After exposure and exploration of more Bitcoin use-cases and applications,
and even among the different requirement among types of use-cases nodes
(e.g LN mobile vs LSP), I believe more heterogeneity in the policy rules
usage makes more sense
Le jeu. 27 oct. 2022 à 00:52, Anthony Towns via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> a écrit :
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
>
> And other plausible options aren't configurable even at compile time
> -- eg, core doesn't implement BIP 125's inherited signalling rule so
> there's no way to enable it; core doesn't allow opting out of BIP 125
> rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
> more than 1 ancestor; core doesn't allow opting out of LOW_S checks
> (even via -acceptnonstdtxn); etc.
>
> We also naturally have different mempool policies between different
> releases: eg, expansions of policy, such as allowing OP_RETURN or
> expanding it from 40 to 80 bytes or new soft forks where old nodes won't
> relay transactions that use the new; and also occassional restrictions
> in policy, such as the LOW_S requirement.
>
>
> While supporting and encouraging different mempool polices might be new
> for core, it's not new for knots: knots changes some of these defaults
> (-permitbaremultisig defaults to false, -datacarriersize is reduced to
> 42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
> options including -spkreuse and -mempoolreplacement (giving the latter
> full rbf behaviour by default). Knots also includes a `-corepolicy`
> option to make it easy to get a configuration matching core's defaults.
>
>
> I think gmaxwell's take from Feb 2015 (in the context of how restrictive
> policy on OP_RETURN data should be) was a reasonable description for
> core's approach up until now:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works. E.g. In stealth addresses the early proposals
> use highly inefficient single ECDH point per output instead of simply
> pooling them. Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public". No one wants to be an outright gatekeeper,
> but the network is a shared resource and it's perfectly reasonable
> node behavior to be stingy about the perpetual storage impact of the
> transactions they're willing to process, especially when it comes to
> neutral technical criteria like the amount of network irrelevant data
> stuffed in transactions.
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it. [...
> ...] For mitigating these risks it's optimal if transactions
> seem as uniform and indistinguishable as reasonably possible.
>
> - https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
>
> Perhaps see also sdaftuar in Nov 2015,
>
> To me the most important question is, is priority something that miners
> want to use?
>
> If a non-negligible amount of hashpower intends to use it in their
> transaction selection, then I think it makes sense for nodes to use it
> too, because it's generally helpful to have your mempool predict the
> UTXO as much as possible, and for nodes to be able to have reasonable
> fee and priority estimates (which won't happen unless they track the
> priority transactions somehow -- I'm presuming that miners run with
> much bigger mempools than regular nodes).
>
> If the answer is no, then that's fine and I don't see a reason to push
> in this direction. I sort of assumed there was enough hashpower mining
> with priority, since last time I checked estimatepriority was still
> giving meaningful results for low-ish blockheights, but I haven't done
> any kind of real analysis.
>
> - https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
>
> or in June 2019,
>
> What this PR is proposing is to get rid of a command-line option that is
> (a) a footgun for users and (b) does not reflect what I believe to be
> the understanding most users have, which is that [X txs] are expected
> to propagate well on the network.
>
> ..
>
> I don't think this rises to the level that Luke is concerned about,
> namely a prelude to forcing a common relay policy on all nodes. In
> particular I do agree it makes sense that we offer some ways of
> customizing policy parameters (eg the mempool size, min relay fee,
> etc). Instead, I think the justification for this change is that we
> should not support behaviors we think are harmful to the ecosystem
> overall and have no legitimate use-case, and we should eliminate ways
> that users might inadvertently shoot themselves in the foot.
>
> - https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
>
> (or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
>
> I don't mean to imply the above are saying "there's one way to do
> things and it's this way", or that the old way of doing things should
> necessarily be the way we keep doing things. Just that previously core
> has tended towards designing a single policy that works as well as it
> can for everyone and the ecosystem as a whole. (I'm also not saying that
> fullrbf can't work well for everyone or the ecosystem as a whole)
>
>
> By contrast, I think the most common response to pushback against the
> full rbf option has been along the lines of "it's just an option, we
> don't want to force people", eg:
>
> Blaming the default false -mempoolfullrbf option for a full RBF network
> would be holding Bitcoin Core developers responsible for the decisions
> of individual node operators and miners. I don't think having the
> option (again, default false) can directly cause a full RBF network,
> and likewise, I don't think removing this option removes the "risk"
> of a full RBF network.
> - glozow
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
>
> NACK. This is a default false option.
> - achow101
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
>
> Erecting artificial barriers to prevent or make it difficult for users
> to do what they want to do, is not appropriate behaviour.
> - luke-jr
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
>
> I'm in general against removing options.
> - instagibbs
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
>
> I think this differs from what core has done in the past, in that
> previously we've tried to ensure a new policy is good for everyone (or as
> nearly as it can be), and then enabled it as soon as it's implemented.
> Any options that have been added have either been to control resource
> usage in ways that don't significantly effect tx propagation, to
> allow people to revert to the old behaviour when the new behaviour is
> controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
> and to make it easier to test/debug the implementation.
>
> Giving people a new relay behaviour they can opt-in to when we aren't
> confident enough to turn on by default doesn't match the approach I've
> seen core take in the past.
>
>
> If this is going to be an ongoing shift in how core sees relay/mempool
> policy, I think that's significant and worth paying attention to.
>
> I don't think it's necessary to have that shift to roll out full rbf.
> The other approach would be either:
>
> * set -mempoolfullrbf=true as the default for 24.0, and just have the
> command line param there in case people want to do a
> "UserRejectedMempoolPolicy" campaign to get everyone to opt-out
>
> * revert it for now because we don't think mainnet is ready for fullrbf
> yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
> to activate at some scheduled date in that timeframe (potentially
> backporting it to previous releases to help with adoption too,
> whatever). same effect as the previous option, just with a bit more
> advanced notice and time to prepare
>
> I don't think anyone's proposed the first (which I interpret as "most of
> us don't think mainnet is ready for fullrbf today"), but the comments
> above are all pushback by people arguing against (the first step of)
> the second approach, and they seem to be winning the day.
>
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case and
> not a precedent, and when people propose other default false options,
> there will be substantially more resistance to them being merged,
> despite all the talk about users having options that's going on right now.
>
>
> Assuming it is the change of direction it appears to be -- and all of
> the above is really just justification for that assumption -- then like
> I said, I think it's worth seriously considering what it means for people
> to choose their own relay/mempool policies and for you to expect to have
> different mempool policies to many or most of your potential peers.
>
>
> One thing maybe worth noting is that is that you can still only choose
> your policy from options that people write code for -- if it wasn't
> something you could get by running knots or compiling a rejected PR
> yourself, it won't magically become more possible now. Presumably it
> would mean that once a PR is written, it might get better review (rather
> than being dismissed as not suitable for everyone), and there would be
> less maintenance burden than if it had to be manually rebased every
> release, though (or at least the maintenance burden would be shared
> across everyone working on the codebase).
>
>
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
>
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance. If erlay allows increasing your outbound count to
> 12 connections instead of 8; that might reduce down to needing just 20%
> of listening nodes (~93%).
>
> But for cases where less than 30% (20%) of network supports your preferred
> policy, you probably need to do something cleverer.
>
> One approach is to set a service bit and preferentially peer with other
> nodes that advertise that service bit; knots does the first half of this
> for fullrbf, and both halves have been proposed for core in #25600.
> Preferential peering was previously done for the segwit deployment,
> though in that case it was necessary not just for tx propogation but
> also for ensuring block propogation, making it effectively a consensus
> critical issue.
>
> Another approach is having a separate relay network -- eg, lightning nodes
> already have a gossip network, and might want to help their own ecosystem
> by ensuring unilateral channel closes and justice transactions are quickly
> relayed. Using their own gossip network to relay the transaction around,
> and each lightning node adding it to their local bitcoind's mempool and
> allowing it to propogate (or not) from there as normal, would also be a
> way of allowing transactions to propogate well. It does mean that miners
> would either need to also participate in lightning gossip directly, or
> that miners would need to connect to *many* peers to be confident of
> seeing those transactions (eg, if only 2% of the network would see a
> tx, you'd need to make 228 connections to have a 99% chance of seeing
> the tx). You can't currently do something like this, because all the
> relay policies are also applied when adding txs to the mempool via RPC,
> and there's no convenient way to remove txs from the mempool.
>
> A case where something like that might occur is in preventing L2
> transactions from pinning attacks -- so you might have a high-fee,
> low-feerate transaction that's been widely propogated, sitting in the
> bottom of people's mempools, and you want to replace it with a smaller,
> higher-feerate transaction, but don't want to pay a higher absolute fee,
> and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
> unwilling to deviate from BIP 125 rule 3 for you; because that would
> make it easy for random griefers to spam their mempool with large txs
> then delete them while only paying a small fee; but your L2 peers may be
> able to decode your replacement transaction and be sure that you aren't
> going to spam them, and thus will happily relay it.
>
> From a technical point-of-view, that's largely fine; the downside is it
> increases the centralisation pressure on mining: whether that's by having
> to connect to substantially more nodes, or having to parse through more
> spam, you can't just run your mining operation off a standard install
> of bitcoin core anymore, but need to actively opt-in to find all the
> weird unusual ways people are sending transactions around in order to
> actually collect as much in fees as your competitors are.
>
> That's probably moderately bad for privacy as well -- if lightning or
> coinjoins need special relay rules that most nodes haven't opted into,
> it's potentially easy to use that to find the bitcoin nodes on the
> network that are participating in those protocols, and from there to
> either identify the operator, or run a DoS attack to make it hard for you
> to keep doing what you want. Obviously if you're setting a service bit to
> get better routing, you've given up that privacy already. Likewise if the
> government or random vandals are opposed to bitcoin mining, and miners
> have to have special configuration on their nodes that distinguish them
> from regular users, then perhaps that makes it easier to find or shut
> down their operations.
>
> There are a few efficiencies to be gained from similar mempool policies as
> well: more reliable compact block reconstruction (if you're not missing
> any transactions, you avoid a round-trip) and presumably more efficient
> set reconstruction with erlay. You'll also waste less bandwidth sending
> transactions that the other node is only going to reject. Both those
> depend on how many transactions are going to rely on unusual mempool
> policies in the first place though.
>
> ariard wrote:
>
> I know I've advocated in the past to turn RBF support by default in
> the past. Though after gathering a lot of feedbacks, this approach
> of offering the policy flexiblity to the interested users only and
> favoring a full-rbf gradual deployment sounds better to me.
>
> - https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
>
> I guess all the above leads me to think that gradual deployments of
> mempool policies are likely the worse approach: even when they're not
> hurting anyone, it makes them hard to use during the gradual phase,
> and getting around that comes with worrying compromises on privacy and
> centralisation; and when they are problematic for some, the indeterminate
> nature of a gradual deployment means it's hard to plan for when that
> risk is going to eventuate.
>
>
> Theoretically, one way to recover the good parts of core deciding on
> what's good for the network might be for people outside of core to
> recommend a mempool configuration; then core can just have an option
> to make that easy, similar to "-std=c++17" for a C++ compiler, and much
> the same as knots' "-corepolicy" option.
>
> Presuming anyone actually wants to take on that job, and listen to the
> concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
> and can come up with something that keeps most of them happy, and that
> 70% or 90% of the network ends up just following those recommendations
> because it's easy, it works, and it's recommended by all the apps they
> want to use, then that could work great:
>
> * miners don't need to do anything special, so there's no new
> mining centralisation pressure
> * miners and users don't reveal what they're doing with bitcoin by the way
> they configure their nodes, so there's no privacy problems
> * devs can be fairly confident in how they have to design their apps
> in order to get their transactions to most hashpower
> * devs don't have to add new p2p layers to make it happen
> * at least there's someone to talk to when you're trying to figure out
> how to make some new project possible when it's inhibited by current
> relay policies and you don't have to try to convince everyone to
> upgrade on your own
> * core devs just provide options, and don't have to worry about being
> seen as gatekeepers
>
> The "downside" in that scenario is that users/dev aren't making much
> actual use of all the choices core is offering by making different
> options available; but the upside is that that choice is at least readily
> available should whoever is coming up with these policy become out of
> step with what people actually want.
>
> One thing that might make an approach like that difficult is that core
> has historically been happy to remove options that don't seem useful
> anymore: eg the ability to turn of BIP 125 support (#16171), and priority
> transactions (#9602). Perhaps that's fine if you're trying to actively
> craft a single mempool/relay policy that's good enough for almost everyone
> (after all, it makes the code simpler and more efficient, and reduces
> the number of footguns); all you're doing is leaving a minority of people
> who want weird things to run a fork, and that's going to happen anyway.
>
> But if people are following policy developed outside of core, core
> might well disagree with them and decide "no that's a stupid policy,
> no one should do that" and remove some feature that others thing should
> continue to be normal. Beyond the examples above, there's already talk of
> removing the ability to disable fullrbf support in #26305, for instance.
> If that happens, then the people maintaining the policy will instead
> end up maintaining an entire fork of bitcoin core, and all we've done
> is transition to people running software from a different repo, and a
> different set of maintainers.
>
> If we're really going to a world where core's eager to add new options,
> and reluctant to remove them, at least if anyone at all finds them
> interesting, that's presumably a non-issue, though.
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 43884 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
` (2 preceding siblings ...)
2022-10-27 20:29 ` Antoine Riard
@ 2022-10-29 7:45 ` David A. Harding
2022-10-30 1:02 ` Anthony Towns
2022-10-31 13:02 ` Suhas Daftuar
4 siblings, 1 reply; 33+ messages in thread
From: David A. Harding @ 2022-10-29 7:45 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
On 2022-10-26 13:52, Anthony Towns via bitcoin-dev wrote:
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your
> tx,
> just by random chance.
I think this might be understating the problem. A 95% chance of having
an outbound peer accept your tx conversely implies 1 in 20 payments will
fail to
propagate on their initial broadcast. That seems to me like an
unacceptably high failure rate both for the UX of regular payments and
for the safety of time-sensitive transactions like onchain HTLC
resolutions.
Additionally, the less reliable propagation is, the more reliably spy
nodes can assume the first IP address they received a transaction from
is the creator of that transaction.
I think those two problems combine in an especially unfortunate way for
lightweight clients. Lightweight clients wanting to find a peer who
supports a more permissive policy than most of the network and whose
client authors want to provide a good UX (or safety in the case of time
sensitive contract protocols like LN) will need to open large numbers of
connections, increasing their chance of connecting to a spy node which
will associate their IP address with their transaction, especially since
lightweight clients can't pretend to be relaying transactions for other
users. Some napkin math: there are about 250,000 transactions a day; if
we round that up to 100 million a year and assume we only want one
transaction per year to fail to initially propagate on a network where
30% of nodes have adopted a more permissive policy, lightweight clients
will need to connect to over 50 randomly selected nodes.[1] For a more
permissive policy only adopted by 10% of nodes, the lightweight client
needs to connect to almost 150 nodes.
This also implies that nodes adopting a more restrictive policy degrades
UX, safety, and privacy for users of transactions violating that policy.
For example, if 30% of nodes used Knots's -spkreuse configuration option
and about 50% of transactions reuse scriptPubKeys, then about 9
transactions a day wouldn't initially propagate (assuming 8 randomly
selected peers[2]) and lightweight clients who wanted 1-in-100-million
safety would need to connect to about 15 random nodes.
Towns's post to which I'm replying describes several alternative
approaches which mitigate the above problems, but he also documents that
they're not without tradeoffs.
-Dave
[1] (1-0.3)**50 * 100_000_000 =~ 1.8
[2] That assumes every transaction is sent to a different
randomly-selected set of peers, which isn't really the case. However,
one day $GIANT_EXCHANGE could suddenly be unable to broadcast hundreds
or
thousands of withdrawal transactions because all of its peers implement
a restrictive policy.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-29 7:45 ` David A. Harding
@ 2022-10-30 1:02 ` Anthony Towns
2022-10-30 2:40 ` Anthony Towns
2022-10-30 11:06 ` email
0 siblings, 2 replies; 33+ messages in thread
From: Anthony Towns @ 2022-10-30 1:02 UTC (permalink / raw)
To: David A. Harding, Bitcoin Protocol Discussion
On Fri, Oct 28, 2022 at 09:45:09PM -1000, David A. Harding via bitcoin-dev wrote:
> I think this might be understating the problem. A 95% chance of having
> an outbound peer accept your tx conversely implies 1 in 20 payments will
> fail to propagate on their initial broadcast.
Whether that's terrible or not depends on how easy it is to retry (and how
likely the retry is to succeed) after a failure -- if a TCP packet fails,
it just gets automatically resent, and if that succeeds, there's a little
lag, but your connection is still usable. I think it's *conceivable* that
a 5% failure rate could be detectable and automatically rectified. Not
that I have a good idea how you'd actually do that, in a way that's
efficient/private/decentralised...
> Some napkin math: there are about 250,000 transactions a day; if
> we round that up to 100 million a year and assume we only want one
> transaction per year to fail to initially propagate on a network where
> 30% of nodes have adopted a more permissive policy, lightweight clients
> will need to connect to over 50 randomly selected nodes.[1]
A target failure probability of 1-in-1e8 means:
* with 8 connections, you need 90% of the network to support your txs
* with 12 connections, you need ~79%
* with 24 connections (eg everyone running a long-lived node is
listening, so long lived nodes make 12 outbound and receive about
~12 inbound; shortlived nodes just do 24 outbound), you need ~54%
So with that success target, and no preferential peering, you need
a majority of listening nodes to support your tx's features in most
reasonable scenarios, I think.
> For a more
> permissive policy only adopted by 10% of nodes, the lightweight client
> needs to connect to almost 150 nodes.
I get 175 connections needed for that scenario; or 153 with a target
failure rate of 1-in-10-million.
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 20:29 ` Antoine Riard
@ 2022-10-30 2:24 ` Anthony Towns
0 siblings, 0 replies; 33+ messages in thread
From: Anthony Towns @ 2022-10-30 2:24 UTC (permalink / raw)
To: Antoine Riard, Bitcoin Protocol Discussion
On Thu, Oct 27, 2022 at 09:29:47PM +0100, Antoine Riard via bitcoin-dev wrote:
> Let's take the contra.
(I don't think I know that phrase? Is it like "play devil's advocate"?)
> I would say the current post describes the state of Bitcoin Core and
> beyond policy
> rules with a high-degree of exhaustivity and completeness, though itt what
> is, mostly a description. While I think it ends with a set of
> recommendations
It was only intended as a description, not a recommendation for anything.
At this point, the only thing I think I could honestly recommend
that doesn't seem like it comes with massive downsides, is for core to
recommend and implement a particular mempool policy, and only have options
that either make it feasible to scale that policy to different hardware
limitations, and provide options that users can activate en-masse if it
turns out people are doing crazy things in the mempool (eg, a new policy
turns out to be ill-conceived, and it's better to revert to a previous
policy; or a potential spam vector gets exploited at scale).
> What should be actually the design goals and
> principles of Core's transaction-relay propagation rules
> of which mempool accepts ones is a subset?
I think the goals of mempool/relay policy are _really_ simple; namely:
* relay transactions from users to all potential miners, so that
non-mining nodes don't have to directly contact miners to announce
their tx, both for efficiency (your tx can appear in the next block
anyone mines, rather than just the next block you mine) and privacy
(so that miners don't know who a transaction belongs to, so that
users don't have to know who miners are, and so there's no/minimal
correlation between who proposed a tx and who mined the block it
appears in)
* having most of the data that makes up the next block pre-validated
and pre-distributed throughout the network, so that block validation
and relay is much more efficient
> By such design goals, I'm
> thinking either, a qualitative framework, like attacks game for a concrete
> application ("Can we prevent pinning against multi-party Coinjoin ?").
I don't think that even makes sense as a question at that level: you can
only ask questions like that if you already have known mempool policies
across the majority of nodes and miners. If you don't, you have to allow
for the possibility that 99% of hashrate is receiving private blacklists
from OFAC and that one of your coinjoin counterparties is on that list,
eg, and at that point, I don't think pinning is even conceivably solvable.
> I believe we would come up with a
> second-order observation. That we might not be able to satisfy every
> use-case with the standard set of policy rules. E.g, a contracting protocol
> could look for package size beyond the upper bound anti-Dos limit.
One reason that limit is in place is that it the larger the tx is
compared to the block limit, the more likely you are to hit corner cases
where greedily filling a block with the highest fee ratex txs first
is significantly suboptimal. That might mean, eg, that there's 410kvB
of higher fee rate txs than your 600kvB large package, and that your
stuff gets delayed, because the miner isn't clever enough to realise
dropping the 10kvB is worthwhile. Or it might mean that your tx gets
delayed because the complicated analysis takes a minute to run and a
block was mined using the simpler algorithm first. Or it might mean that
some mining startup with clever proprietary software that can calculate
this stuff quickly make substantially more profit than everyone else,
so they start dominating block generation, despite the fact that they're
censoring transactions due to OFAC rules.
> Or even the
> global ressources offered by the network of full-nodes are not high enough
> to handle some application event.
Blocks are limited on average to 4MB-per-10-minutes (6.7kB/second),
and applications certainly shouldn't be being designed to only work if
they can monopolise the entirety of the next few blocks. I don't think
it makes any sense to imagine application events in Bitcoin that exceed
the capacity of a random full node. And equally, even if you're talking
about some other blockchain with higher capacity; I don't really think
it makes sense to call it a "full" node if it can't actually cope with
the demands placed on it by any application that works on that network.
> E.g a Lightning Service Provider doing a
> liquidity maintenance round of all its counterparties, and as such
> force-closing and broadcasting more transactions than can be handled at the
> transaction-relay layer due to default MAX_PEER_TX_ANNOUNCEMENTS value.
MAX_PEER_TX_ANNOUNCEMENTS is 5000 txs, and it's per-peer. If you're an
LSP that's doing that much work, it seems likely that you'd at least
be running a long-lived listening node, so likely have 100+ peers, and
could conceivably simultaneously announce 500k txs distributed across
them, which at 130vB each (1-taproot-in, 2-p2wpkh out, which I think is
pretty minimal) adds up to 65 blocks worth of transactions. And then,
you could run more than one node, as well.
Your real limitation is likely that most nodes on the network
will only relay your txs onwards at an average rate of ~7/second
(INVENTORY_BROADCAST_PER_SECOND), so even 5000 txs will likely take over
700s to propagate anyway.
> My personal take on those subjects, we might have to realize we're facing
> an heterogeneity of Bitcoin applications and use-cases [1].
Sure; but that's why you make your policy generic, rather than having
to introduce a new, different policy targeted at each new use case.
> And this sounds
> like a long-term upward trend, akin to the history of the Internet: mail
> clients, web browser, streaming applications, etc, all with different
> service-level requirements in terms of latency, jitters and bandwidth.
Back in the mid/late 90s, people argued that real-time communication,
(like audio chat, let alone streaming video) wasn't suitable for IP,
but would require a different network like ATM where dedicated circuits
were established between the sender and recipient to avoid latency,
jitter and bandwidth competition. Turns out that separate networks
weren't optimal for that.
> To put it simply, some advanced Bitcoin
> applications might have to outgrow the "mempool policy rules" game,
I think if you stick to the fundamentals -- that relay/mempool is about
getting transactions to miners and widely preseeding the contents of
whatever the next block will be -- then it's pretty unlikely that any
Bitcoin application will outgrow the mempool policy game.
> I think this has been historically the case with
> some miners deciding to join FIBER, to improve their view of mined blocks.
FIBRE (it doesn't use the US spelling) is a speedup on top of compact
block relay -- it still gets you exactly the same data if you don't use,
it's just everything is slightly faster if you do. Even better, if you
get a block via FIBRE, then you relay it on to your peers over regular
p2p, helping them get it faster too.
Doing something similar with mempool txs -- having some high bandwidth
overlay network where the edges then feed txs back into the main p2p
network at a slower rate that filters out spam or whatever -- would
probably likewise be a fine addition to bitcoin, provided it had the
same policy rules as regular bitcoin nodes employ for tx relay. If it
had different ones, it would become a signficant centralisation risk: app
developers who make use of the different rules would need to comply with
the overlay networks ToS to avoid getting banned, and miners would need
to subscribe to the feed to avoid missing out on txs and thus fee income.
> What I'm expressing is a long-term perspective, and we might be too early
> in the evolutionary process that is Bitcoin Core development to abandon yet
> the "one-size-fits-all" policy rules conception that I understand from
> your post.
I don't think "one-size-fits-all" is a principle at all; I think
decentralisation/censorship-resistance, privacy, and efficiency are the
fundamental principles. As far as I can see, a one-size-fits-all approach
(or, more precisely, an approach where >90% of the network converges to
the same set of rules) is far better at achieving those principles than
a heterogenous policy approach.
> After exposure and exploration of more Bitcoin use-cases and applications,
> and even among the different requirement among types of use-cases nodes
> (e.g LN mobile vs LSP), I believe more heterogeneity in the policy rules
> usage makes more sense
I think when you say "more heterogeneity" what you're actually assuming
is that miners will implement a superset of all those policies, so that
if *any* node on the network accepts a tx X, *every* miner will also
accept a tx X, with the only exception being if there's some conflicting
tx Y that allows the miner to collect more fees.
But that's not really a heterogenous policy: in that case all miners
are employing exactly the same policy.
In that scenario, I don't think you'll end up with nodes running
heteregenous policies either: part of the point of having mempool
policies is to predict the next block, so if all miners really do have
a common policy, it makes sense for nodes to have the same policy. The
only potential difference is miners might be willing to dedicate more
resources, so might set some knobs related to memory/bandwidth/cpu
consumption differently.
I think what you're actually assuming is that this scenario will mean
that miners will quickly expand their shared policy to accept *any*
set of txs that are accepted by a small minority of relay nodes: after
all, if there are some txs out there that pay fees, why wouldn't miners
want to include them? That's what incentive compatible means, right? And
that's great from a protocol reasearch point-of-view: it allows you to
handwave away people complaining that your idea is bad -- by assumption,
all you need to do is deploy it, and it immediately starts working,
without anyone else needing to adopt it.
I don't think that's actually a realistic assumption though: first,
updating miners' policy rules requires updated software to be tested
and deployed, so isn't trivial enough that it should be handwaved away,
second, as in the "big packages" example above, constructing an efficient
block becomes harder the more mempool rules you throw away, so even if
there are txs violating those rules that are offering extra fees, they
may not actually cover the extra costs to generate a block when you're
no longer able to rely on those rules to reduce the complexity of the
problem space.
Note also that "relay nodes will want to use the same policy as mining
nodes" goes both ways -- if that doesn't happen, and compact block
relay requires an extra round trip to reconstruct the block, miners'
blocks won't relay as quickly, and they'll have an increased orphan rate.
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-30 1:02 ` Anthony Towns
@ 2022-10-30 2:40 ` Anthony Towns
2022-10-30 11:06 ` email
1 sibling, 0 replies; 33+ messages in thread
From: Anthony Towns @ 2022-10-30 2:40 UTC (permalink / raw)
To: Bitcoin Protocol Discussion
On Sun, Oct 30, 2022 at 11:02:43AM +1000, Anthony Towns via bitcoin-dev wrote:
> > Some napkin math: there are about 250,000 transactions a day; if
> > we round that up to 100 million a year and assume we only want one
> > transaction per year to fail to initially propagate on a network where
> > 30% of nodes have adopted a more permissive policy, lightweight clients
> > will need to connect to over 50 randomly selected nodes.[1]
> A target failure probability of 1-in-1e8 means:
Oh, based on the "receive version message" log entries of a node that
only does outbound connections, over the last ~3 weeks I see about 3000
outbound connections (mostly feelers/block-relay-only ones), of which
a bunch identify as non-taproot supporting:
10 /Satoshi:0.16.0/:
13 /Satoshi:0.17.0/:
13 /Satoshi:0.17.0.1/:
28 /Satoshi:0.16.3/:
29 /Satoshi:0.19.0.1/:
36 /Satoshi:0.18.1/:
37 /Satoshi:0.19.1/:
39 /Satoshi:0.17.1/:
50 /Satoshi:0.20.0/:
94 /Satoshi:0.21.0/:
95 /Satoshi:0.18.0/:
244 /Satoshi:0.20.1/:
Those add up to 688+ of 3065 total; if that's representative, it
presumably means a random node connecting to 8 random listening peers has
a 6.44-in-1-million chance of only connecting to peers that don't support
taproot, ie failing your suggested threshold by a factor of about 644.
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-30 1:02 ` Anthony Towns
2022-10-30 2:40 ` Anthony Towns
@ 2022-10-30 11:06 ` email
1 sibling, 0 replies; 33+ messages in thread
From: email @ 2022-10-30 11:06 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 3264 bytes --]
> Whether that's terrible or not depends on how easy it is to retry (and
> how
> likely the retry is to succeed) after a failure -- if a TCP packet
> fails,
> it just gets automatically resent, and if that succeeds, there's a
> little
> lag, but your connection is still usable
I'm not sure if that analogy fits here. A TCP packet will be retried
(as opposed to UDP), although usually the retry strategy is because of
an underlying connection issue, not a protocol mismatch or in this case
"policy" mismatch, right?
If network propagation and node discovery becomes an issue, and as
Antoine mentions, there becomes a need for competing services as I
understand it, could that give rise to indexes and trackers who spider
the network to create world view? Perhaps it's a bad idea since "third
party" trackers are security holes, however, to my knowledge, we already
have "trusted" nodes during the initial bootstrap process. Even so, I
don't think we could preclude such solutions from occurring organically
if the need is arises.
Cheers,
-Yancy
On 2022-10-30 02:02, Anthony Towns via bitcoin-dev wrote:
> On Fri, Oct 28, 2022 at 09:45:09PM -1000, David A. Harding via
> bitcoin-dev wrote:
>
>> I think this might be understating the problem. A 95% chance of
>> having
>> an outbound peer accept your tx conversely implies 1 in 20 payments
>> will
>> fail to propagate on their initial broadcast.
>
> Whether that's terrible or not depends on how easy it is to retry (and
> how
> likely the retry is to succeed) after a failure -- if a TCP packet
> fails,
> it just gets automatically resent, and if that succeeds, there's a
> little
> lag, but your connection is still usable. I think it's *conceivable*
> that
> a 5% failure rate could be detectable and automatically rectified. Not
> that I have a good idea how you'd actually do that, in a way that's
> efficient/private/decentralised...
>
>> Some napkin math: there are about 250,000 transactions a day; if
>> we round that up to 100 million a year and assume we only want one
>> transaction per year to fail to initially propagate on a network where
>> 30% of nodes have adopted a more permissive policy, lightweight
>> clients
>> will need to connect to over 50 randomly selected nodes.[1]
>
> A target failure probability of 1-in-1e8 means:
>
> * with 8 connections, you need 90% of the network to support your txs
> * with 12 connections, you need ~79%
> * with 24 connections (eg everyone running a long-lived node is
> listening, so long lived nodes make 12 outbound and receive about
> ~12 inbound; shortlived nodes just do 24 outbound), you need ~54%
>
> So with that success target, and no preferential peering, you need
> a majority of listening nodes to support your tx's features in most
> reasonable scenarios, I think.
>
>> For a more
>> permissive policy only adopted by 10% of nodes, the lightweight client
>> needs to connect to almost 150 nodes.
>
> I get 175 connections needed for that scenario; or 153 with a target
> failure rate of 1-in-10-million.
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[-- Attachment #2: Type: text/html, Size: 5213 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
` (3 preceding siblings ...)
2022-10-29 7:45 ` David A. Harding
@ 2022-10-31 13:02 ` Suhas Daftuar
2022-10-31 16:25 ` Greg Sanders
` (2 more replies)
4 siblings, 3 replies; 33+ messages in thread
From: Suhas Daftuar @ 2022-10-31 13:02 UTC (permalink / raw)
To: Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 17731 bytes --]
AJ,
Thanks for the thoughtful post. I think your observations about how we view
mempool policy in the Bitcoin Core project, and how that seems to be
changing in the discussions around `-mempoolfullrbf`, are on-point and
provide a helpful baseline for considering future policy changes.
For a long time I viewed fullrbf as an eventuality and I considered myself
to be philosophically supportive of the idea. However, after giving this
issue some thought in the past few weeks, I am reversing my thinking on
this. Concretely, I will argue that we should continue to maintain a relay
policy where replacements are rejected for transactions that don't opt-in
to RBF (as described in BIP 125), and moreover, that we should remove the
`-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not
plan to release software with that flag, unless (or until) circumstances
change on the network, which I'll discuss below.
This is, of course, a nuanced topic, and among the considerations is a
philosophy of how to think about the relay policy and configuration options
that we make available in Bitcoin Core (a consideration that is perhaps
unique to that project, but I think relevant for this mailing list).
I'll start with some technical issues regarding the benefits of enabling
fullrbf on the network. In the current BIP 125 regime, every time a
transaction is created, a choice is made whether to subject the transaction
to BIP 125’s RBF rules or not (based on the sequence values of the
inputs). So given that users can already opt-in to RBF, the benefit of a
“fullrbf” network policy would be if, somehow, RBF users were still denied
the benefits of RBF due to the existence of other transactions that don’t
opt-in.
Along those lines, Antoine Riard brought up[1] a DoS vector that is
available to someone who wants to interfere with multi-party funded
transactions, and suggested that fullrbf would eliminate the problem.
After exploring that question again in this thread (thanks to Greg Sanders
for clarifying this to me), I understand that the issue is around ensuring
that a multiparty (coinjoin-type) protocol is able to make eventual
progress, by having a candidate multiparty transaction either eventually
confirm or become conflicted with something that has been confirmed, in
which case the double-spend information could be used to start a new
coinjoin round with fewer participants. The concern Antoine and Greg have
brought up is that non-rbf transactions can persist in the mempool
~indefinitely (at a low feerate and not subject to replacement) and
interfere with progress being made in a coinjoin protocol.
However, it seems to me that similar problems exist for such a protocol
even in a fullrbf world, as we understand that term today. I mentioned the
ability for rbf “pinning” to interfere with relay of the multiparty
transaction (even if the conflicting transaction signals for RBF – a set of
large but low feerate conflicting transactions can persist in the mempool
and make it difficult for the coinjoin transaction from confirming, at
least without attaching a very large fee); and as Greg mentioned in a
followup, the BIP 125 rule to only permit 100 transactions to be removed
from the mempool at a time during a replacement can also be used to pin a
coinjoin protocol in the same way as a non-rbf transaction today. It seems
to me that what these multiparty protocols actually need is some sort of
"maximal rbf" network policy: a way to guarantee that a transaction which
should be desirable for a miner to mine would always get to a miner and
considered for inclusion in a block, no matter what the state of node’s
mempools on the network.
While that sounds like a reasonable thing to want on its face (and worth
working on), it's not how opt-in RBF works today, nor is it how transaction
relay has ever conceptually worked. We have not, thus far, been able to
come up with a total ordering on transaction desirability. Moreover, due
to all the DoS issues that exist with transaction relay, there are plenty
of seemingly legitimate ways to construct transactions that would not relay
well on the network. Relay has only ever been a best-efforts concept,
where we carve out a small subset of the entire transaction universe for
which we try to optimize propagation. The idea behind this approach is
that if every use case we can come up with has some way to achieve its
goals using transactions that should (eventually) be able to relay, then
users wouldn’t have much demand for transactions that would deviate from
the supported policies, and we therefore shouldn’t need to worry too much
about incentive compatibility concerns when it comes to transaction types
that wouldn’t relay at all, even if they are high feerate. (And when those
situations arise where the standard transactions do not accommodate some
needed use case, developers typically work to define a policy that is
compatible with our anti-DoS goals to support such use cases, such as with
the recent proposal for version=3 transactions [2].)
BIP 125's RBF rules themselves were an effort to carve out just a subset of
situations where a transaction should evict conflicting ones -- it was not
a design that anyone thought would ensure that all replacements which
"should" be mined would always propagate. And I don't believe that we know
how to design policy rules that would achieve the goals of this kind of
multiparty protocol in a DoS resistant way, today. Along those lines, I
would point out that even the BIP 125 design itself is not entirely
incentive compatible, in that it is possible to construct a replacement
transaction that would evict transactions which would be preferable to be
included in a block! [3] (This has been known for years, but fixing this
has proven difficult, and the only way to fix it that I’m aware of would be
to make BIP 125 RBF even more restrictive than it is today. I do think this
is something that needs to be worked on.)
Given the limitations of RBF as we have it today, it appears to be
incorrect that a fullrbf network policy would solve the problems Antoine
raised. And so absent any other examples, it does not seem to me that
fullrbf solves any problems for RBF users, who are already free to choose
to subject their transactions to BIP 125’s RBF policy. From this
perspective, "enabling fullrbf" is really just taking away user choice to
opt a transaction into a non-replacement policy regime.
I think we should ask, then, whether it is reasonable on its face that
users might want to opt-in to a non-replacement policy? Or in other words,
is it reasonable for a user to mark a transaction as non-replaceable and
have that indication be enforced by the network? Note that these are two
different questions: you could imagine a world where fullrbf is a dominant
policy, but users still use the BIP 125 signaling method to indicate, in an
unenforced way, their intention to not replace a transaction. This might
give useful information to the network or the recipient for how to interact
with such a transaction.
And I think that it's entirely possible that users would continue to use
the BIP 125 signaling to indicate that they do not intend to replace a
transaction. For better or worse, this might be because zeroconf services
continue to differentiate their behavior based on such a signal (possibly
in conjunction with other factors), or it could be because there are other
behaviors that could be utilized more effectively if the transaction
originator has made such a signal, such as the recipient chaining an
unconfirmed transaction as a way to bump the fee (CPFP) [4].
If it were to be the case that users continued to use BIP 125-style
signaling to indicate that they do not plan to replace a transaction, would
that be harmful to the network? This is not something we can stop in our
policy rules (short of censoring such transactions, an obviously bad
idea). I think network actors can always do things that we might think are
harmful for the network, but that doesn’t mean that there are no legitimate
use cases for the tools that such actors might be using. Just because
someone might use some policy to adopt a zeroconf model, doesn’t mean that
others aren’t using the same policy to achieve benign ends (such as better
CPFP behavior).
Moreover, while users might attempt to exploit services that offer zeroconf
or other differentiated behavior to non-replacement signaling transactions,
they also might not -- I think predicting user behavior in this way (and
specifically predicting the complexities of what a business might do and
whether users might try to subvert it) is beyond the scope of what we can
do as protocol developers. Instead, I think we can try to answer a
different question: if a group of users were to want the ability to opt-in
to a non-replacement policy regime, is that a technically sound option for
us to have on the network and enforce in software? Specifically, does that
interfere with having a sensible anti-DoS mempool acceptance algorithm, or
interfere with other protocols on the network, or necessarily run counter
to the interests of miners or node operators?
And I think the answer to that question, in looking at the difference
between opt-in RBF and fullrbf, is no: offering the ability to opt-in to a
non-replacement regime for transactions doesn't introduce any fundamental
issues with software or network policy or other protocols. In a world
where we only had fullrbf, I could imagine at some point down the road
proposing a non-replacement signal myself, because the complexities around
transaction chains (and pinning) are more complex for the RBF case than for
the non-RBF case (and BIP 125 is not always incentive compatible to begin
with!). Conceptually, this is no different to me than the version=3
transaction policy proposal that has been advancing, if we think of it as a
special set of restrictions on transactions designed to accommodate a
particular use case.
Philosophically, I think we should be looking to add non-interfering use
cases to what the network supports.
To those who argue for making fullrbf a default policy on the network (or
even just offering a flag for users to enable fullrbf), I pose this
hypothetical: suppose we deploy the v3 transaction policy proposal (which I
hope will happen in the near future). That policy would restrict the ways
that outputs of a v3 transaction can be spent while the transaction is
unconfirmed, including by limiting the number and size of descendants that
such a transaction can have, and limiting the types of unconfirmed
ancestors that can be included. Suppose in a few years someone proposes
that we add a "-disable_v3_transaction_enforcement" flag to our software,
to let users decide to turn off those policy restrictions and treat v3
transactions the same as v2, for all the same reasons that could be argued
today with fullrbf: miners might earn more revenue if we allowed multiple
descendant v3 transactions; it's illogical for the recipient of a v3
transaction to believe what is a fundamentally unenforceable promise of a
sender to not issue more high value children that descend from an
unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
policy on the network and we should honor user choice to turn off that flag
if that’s what users want; if users are relying on v3’s policy restrictions
for security then that is an unstable model and we should assume it will
get broken[5].
It’s obvious to me that adding a flag to disable v3 policy would be
subversive to making the lightning use case for v3 transactions work. And
so my response to such a hypothetical proposal would be to argue that no,
we should not enable users to disable this policy, because as long as that
policy is just optional and working for those who want it, it shouldn’t
harm anyone that we offer a tighter set of rules for a particular use
case. Adding a way to bypass those rules is just trying to break someone
else’s use case, not trying to add a new one. We should not wield
"incentive compatibility" as a bludgeon for breaking things that appear to
be working and not causing others harm.
I think this is exactly what is happening with fullrbf.
In comparing v3 transaction policy with opting out of transaction
replacement, there is of course one significant difference that I have
ignored thus far: I think the real difference is an opinion about whether
non-replacement transactions that are being used today are, overall, bad
for Bitcoin, and whether lightning’s use of v3 transactions in the future
would be bad for Bitcoin. If you think that zeroconf is unequivocally bad,
and that no one will be able to plausibly construct a case that lightning
is bad, then that qualitative judgment might sway you to not worrying about
the philosophical issues I've raised above, because these situations can be
distinguished.
However I am not personally willing to say that I think, overall,
non-rbf-signaling transactions in use on the network today are bad for
Bitcoin (or that fullrbf is definitely good – BIP 125’s rbf rules are
something we’ve been trying to improve upon for years, with little
success). Nor am I convinced that someone couldn’t put together a cogent
argument for lightning being bad for Bitcoin, because of its reliance on
relay policies that are difficult to design and impossible to guarantee as
part of its security model. So I choose instead to merely make a judgment
that seems more factually verifiable, which is that non-replacement is a
policy widely in use on the network today, and we largely don't have reason
to think (as far as I know!) that the network is seeing a lot of
transactions that would violate that policy.
If it did turn out that users were commonly signaling non-replacement, but
then signing and trying to relay doublespends, then I think that would be a
very good reason for Bitcoin Core to adopt fullrbf to reflect the reality
of what is happening. In the meantime, I think it makes more sense to say
that because we have BIP 125, there seems to be no need for users to signal
one way and behave another, and therefore there is no need to offer
software that might break a policy that is working well for some users.
Other software projects might choose differently, and it is after all a
permissionless network, so if this is in fact an unstable equilibrium that
will not last, then presumably someday it will be apparent it is not
working and we’ll abandon it. But I think the philosophy of transaction
relay policy in Bitcoin Core should be to support disparate use cases in
order to try to make everything work better, rather than break things
prematurely because we guess others will break them eventually anyway.
For those that have read this long email and still favor a fullrbf network
policy (or even just the ability for users to be able to turn on fullrbf
for themselves), I’d ask for thoughts on the following questions, which
have guided my thinking on this:
Does fullrbf offer any benefits other than breaking zeroconf business
practices? If so, what are they?
Is it reasonable to enforce BIP 125's rbf rules on all transactions, if
those rules themselves are not always incentive compatible?
If someone were to propose a command line option that breaks v3 transaction
relay in the future, is there a logical basis for opposing that which is
consistent with moving towards fullrbf now?
Cheers,
Suhas
[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
[2]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
[3] This is because under the BIP 125 rules, the feerate of the replacement
transaction is not compared to the individual feerates of all transactions
being evicted – we just compare feerates with the transactions that are
directly in conflict (and not their descendants). So it’s possible for a
transaction that would evict 2 or more transactions to have a higher
feerate than the direct conflicts, and higher total fee than the set being
evicted, but have a lower feerate (eg if it is larger) than that of some
subset of the set of transactions being evicted.
[4] Chaining unconfirmed transactions when the sender might RBF the parent
is far riskier than if the sender indicates they don't plan to do so
(chaining onto an RBF transaction creates pinning issues for the sender,
and risks having the child wiped out if the parent is replaced), so I think
this is a concrete reason why signaling that a transaction won’t be
replaced could be useful.
[5] This is a subtle point. I don’t think v3 transactions create an
unreasonable security assumption for the use case it is being designed for.
However, I don’t think anyone could rule out the possibility that someone
could adopt a usage pattern for v3 transactions that subverts the intent of
this policy. For example, if users started using v3 transactions for all
their payments, then the limitations on the number of descendants could
directly interfere with CPFP by a recipient, and someone could argue that
we should break the policy in order to allow for this hypothetical
behavior. I think this is a similar form of argument as saying that
zeroconf practices + BIP 125 create an incentive to double-spend non-rbf
signaling transactions.
[-- Attachment #2: Type: text/html, Size: 18139 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 13:02 ` Suhas Daftuar
@ 2022-10-31 16:25 ` Greg Sanders
2022-10-31 17:21 ` email
2022-11-02 3:07 ` Anthony Towns
2022-11-02 19:50 ` Antoine Riard
2022-11-05 2:35 ` Peter Todd
2 siblings, 2 replies; 33+ messages in thread
From: Greg Sanders @ 2022-10-31 16:25 UTC (permalink / raw)
To: Suhas Daftuar, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 20400 bytes --]
Thanks for your full thoughts Suhas,
The idea of V3 is that we're currently leaving fees on the table by
allowing use-cases to be pinned, not that we like Lightning and we think
miners should stop being profit maximizing somehow to enable safer/better
layer 2 systems.
If someone wants to bump fees for V3 transactions(or replace them!),
there's a much simpler "API" to do so than in legacy policy land. The fact
that it disallows more idiotic ways to add more total fees means wallets
"shouldn't do that". If it ends up that V3 is disallowing too many
"natural" ways to fee bump, that's a strike against the V3 idea and should
be discussed. For 0-conf services we have potential thieves who are willing
to *out-bid themselves* to have funds come back to themselves. It's not a
"legitimate" use-case, but a rational one.
I have mostly come around to not pushing for fullrbf due to the issues you
mentioned, except taking away the option. Removing a
quite-likely-incentive-compatible option from the software just encourages
miners to adopt an additional patch if they ever deem it necessary to
increase their revenue, even if that revenue is from hurting 0-conf
businesses.
Maybe putting/leaving in a default-false flag for disabling these "carve
outs" is the least bad option. V3 usage patterns shouldn't crumble if a
large majority of miners opt out, but 0-conf use cases crumble after a
small percentage of adoption.
To recap my thoughts:
1) I have put away my fullrbf hats, I will not advocate anyone running it
as I think it doesn't really do anything useful for users who aren't trying
to double-spend merchants.
2) Forcing miners to honor fees left on the table with respect to 0-conf,
or forcing them to run a custom patchset to go around it, is a step
backwards.
Greg
On Mon, Oct 31, 2022 at 11:03 AM Suhas Daftuar via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> AJ,
>
> Thanks for the thoughtful post. I think your observations about how we
> view mempool policy in the Bitcoin Core project, and how that seems to be
> changing in the discussions around `-mempoolfullrbf`, are on-point and
> provide a helpful baseline for considering future policy changes.
>
> For a long time I viewed fullrbf as an eventuality and I considered myself
> to be philosophically supportive of the idea. However, after giving this
> issue some thought in the past few weeks, I am reversing my thinking on
> this. Concretely, I will argue that we should continue to maintain a relay
> policy where replacements are rejected for transactions that don't opt-in
> to RBF (as described in BIP 125), and moreover, that we should remove the
> `-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not
> plan to release software with that flag, unless (or until) circumstances
> change on the network, which I'll discuss below.
>
> This is, of course, a nuanced topic, and among the considerations is a
> philosophy of how to think about the relay policy and configuration options
> that we make available in Bitcoin Core (a consideration that is perhaps
> unique to that project, but I think relevant for this mailing list).
>
> I'll start with some technical issues regarding the benefits of enabling
> fullrbf on the network. In the current BIP 125 regime, every time a
> transaction is created, a choice is made whether to subject the transaction
> to BIP 125’s RBF rules or not (based on the sequence values of the
> inputs). So given that users can already opt-in to RBF, the benefit of a
> “fullrbf” network policy would be if, somehow, RBF users were still denied
> the benefits of RBF due to the existence of other transactions that don’t
> opt-in.
>
> Along those lines, Antoine Riard brought up[1] a DoS vector that is
> available to someone who wants to interfere with multi-party funded
> transactions, and suggested that fullrbf would eliminate the problem.
> After exploring that question again in this thread (thanks to Greg Sanders
> for clarifying this to me), I understand that the issue is around ensuring
> that a multiparty (coinjoin-type) protocol is able to make eventual
> progress, by having a candidate multiparty transaction either eventually
> confirm or become conflicted with something that has been confirmed, in
> which case the double-spend information could be used to start a new
> coinjoin round with fewer participants. The concern Antoine and Greg have
> brought up is that non-rbf transactions can persist in the mempool
> ~indefinitely (at a low feerate and not subject to replacement) and
> interfere with progress being made in a coinjoin protocol.
>
> However, it seems to me that similar problems exist for such a protocol
> even in a fullrbf world, as we understand that term today. I mentioned the
> ability for rbf “pinning” to interfere with relay of the multiparty
> transaction (even if the conflicting transaction signals for RBF – a set of
> large but low feerate conflicting transactions can persist in the mempool
> and make it difficult for the coinjoin transaction from confirming, at
> least without attaching a very large fee); and as Greg mentioned in a
> followup, the BIP 125 rule to only permit 100 transactions to be removed
> from the mempool at a time during a replacement can also be used to pin a
> coinjoin protocol in the same way as a non-rbf transaction today. It seems
> to me that what these multiparty protocols actually need is some sort of
> "maximal rbf" network policy: a way to guarantee that a transaction which
> should be desirable for a miner to mine would always get to a miner and
> considered for inclusion in a block, no matter what the state of node’s
> mempools on the network.
>
> While that sounds like a reasonable thing to want on its face (and worth
> working on), it's not how opt-in RBF works today, nor is it how transaction
> relay has ever conceptually worked. We have not, thus far, been able to
> come up with a total ordering on transaction desirability. Moreover, due
> to all the DoS issues that exist with transaction relay, there are plenty
> of seemingly legitimate ways to construct transactions that would not relay
> well on the network. Relay has only ever been a best-efforts concept,
> where we carve out a small subset of the entire transaction universe for
> which we try to optimize propagation. The idea behind this approach is
> that if every use case we can come up with has some way to achieve its
> goals using transactions that should (eventually) be able to relay, then
> users wouldn’t have much demand for transactions that would deviate from
> the supported policies, and we therefore shouldn’t need to worry too much
> about incentive compatibility concerns when it comes to transaction types
> that wouldn’t relay at all, even if they are high feerate. (And when those
> situations arise where the standard transactions do not accommodate some
> needed use case, developers typically work to define a policy that is
> compatible with our anti-DoS goals to support such use cases, such as with
> the recent proposal for version=3 transactions [2].)
>
> BIP 125's RBF rules themselves were an effort to carve out just a subset
> of situations where a transaction should evict conflicting ones -- it was
> not a design that anyone thought would ensure that all replacements which
> "should" be mined would always propagate. And I don't believe that we know
> how to design policy rules that would achieve the goals of this kind of
> multiparty protocol in a DoS resistant way, today. Along those lines, I
> would point out that even the BIP 125 design itself is not entirely
> incentive compatible, in that it is possible to construct a replacement
> transaction that would evict transactions which would be preferable to be
> included in a block! [3] (This has been known for years, but fixing this
> has proven difficult, and the only way to fix it that I’m aware of would be
> to make BIP 125 RBF even more restrictive than it is today. I do think this
> is something that needs to be worked on.)
>
> Given the limitations of RBF as we have it today, it appears to be
> incorrect that a fullrbf network policy would solve the problems Antoine
> raised. And so absent any other examples, it does not seem to me that
> fullrbf solves any problems for RBF users, who are already free to choose
> to subject their transactions to BIP 125’s RBF policy. From this
> perspective, "enabling fullrbf" is really just taking away user choice to
> opt a transaction into a non-replacement policy regime.
>
> I think we should ask, then, whether it is reasonable on its face that
> users might want to opt-in to a non-replacement policy? Or in other words,
> is it reasonable for a user to mark a transaction as non-replaceable and
> have that indication be enforced by the network? Note that these are two
> different questions: you could imagine a world where fullrbf is a dominant
> policy, but users still use the BIP 125 signaling method to indicate, in an
> unenforced way, their intention to not replace a transaction. This might
> give useful information to the network or the recipient for how to interact
> with such a transaction.
>
> And I think that it's entirely possible that users would continue to use
> the BIP 125 signaling to indicate that they do not intend to replace a
> transaction. For better or worse, this might be because zeroconf services
> continue to differentiate their behavior based on such a signal (possibly
> in conjunction with other factors), or it could be because there are other
> behaviors that could be utilized more effectively if the transaction
> originator has made such a signal, such as the recipient chaining an
> unconfirmed transaction as a way to bump the fee (CPFP) [4].
>
> If it were to be the case that users continued to use BIP 125-style
> signaling to indicate that they do not plan to replace a transaction, would
> that be harmful to the network? This is not something we can stop in our
> policy rules (short of censoring such transactions, an obviously bad
> idea). I think network actors can always do things that we might think are
> harmful for the network, but that doesn’t mean that there are no legitimate
> use cases for the tools that such actors might be using. Just because
> someone might use some policy to adopt a zeroconf model, doesn’t mean that
> others aren’t using the same policy to achieve benign ends (such as better
> CPFP behavior).
>
> Moreover, while users might attempt to exploit services that offer
> zeroconf or other differentiated behavior to non-replacement signaling
> transactions, they also might not -- I think predicting user behavior in
> this way (and specifically predicting the complexities of what a business
> might do and whether users might try to subvert it) is beyond the scope of
> what we can do as protocol developers. Instead, I think we can try to
> answer a different question: if a group of users were to want the ability
> to opt-in to a non-replacement policy regime, is that a technically sound
> option for us to have on the network and enforce in software?
> Specifically, does that interfere with having a sensible anti-DoS mempool
> acceptance algorithm, or interfere with other protocols on the network, or
> necessarily run counter to the interests of miners or node operators?
>
> And I think the answer to that question, in looking at the difference
> between opt-in RBF and fullrbf, is no: offering the ability to opt-in to a
> non-replacement regime for transactions doesn't introduce any fundamental
> issues with software or network policy or other protocols. In a world
> where we only had fullrbf, I could imagine at some point down the road
> proposing a non-replacement signal myself, because the complexities around
> transaction chains (and pinning) are more complex for the RBF case than for
> the non-RBF case (and BIP 125 is not always incentive compatible to begin
> with!). Conceptually, this is no different to me than the version=3
> transaction policy proposal that has been advancing, if we think of it as a
> special set of restrictions on transactions designed to accommodate a
> particular use case.
>
> Philosophically, I think we should be looking to add non-interfering use
> cases to what the network supports.
>
> To those who argue for making fullrbf a default policy on the network (or
> even just offering a flag for users to enable fullrbf), I pose this
> hypothetical: suppose we deploy the v3 transaction policy proposal (which I
> hope will happen in the near future). That policy would restrict the ways
> that outputs of a v3 transaction can be spent while the transaction is
> unconfirmed, including by limiting the number and size of descendants that
> such a transaction can have, and limiting the types of unconfirmed
> ancestors that can be included. Suppose in a few years someone proposes
> that we add a "-disable_v3_transaction_enforcement" flag to our software,
> to let users decide to turn off those policy restrictions and treat v3
> transactions the same as v2, for all the same reasons that could be argued
> today with fullrbf: miners might earn more revenue if we allowed multiple
> descendant v3 transactions; it's illogical for the recipient of a v3
> transaction to believe what is a fundamentally unenforceable promise of a
> sender to not issue more high value children that descend from an
> unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
> policy on the network and we should honor user choice to turn off that flag
> if that’s what users want; if users are relying on v3’s policy restrictions
> for security then that is an unstable model and we should assume it will
> get broken[5].
>
> It’s obvious to me that adding a flag to disable v3 policy would be
> subversive to making the lightning use case for v3 transactions work. And
> so my response to such a hypothetical proposal would be to argue that no,
> we should not enable users to disable this policy, because as long as that
> policy is just optional and working for those who want it, it shouldn’t
> harm anyone that we offer a tighter set of rules for a particular use
> case. Adding a way to bypass those rules is just trying to break someone
> else’s use case, not trying to add a new one. We should not wield
> "incentive compatibility" as a bludgeon for breaking things that appear to
> be working and not causing others harm.
>
> I think this is exactly what is happening with fullrbf.
>
> In comparing v3 transaction policy with opting out of transaction
> replacement, there is of course one significant difference that I have
> ignored thus far: I think the real difference is an opinion about whether
> non-replacement transactions that are being used today are, overall, bad
> for Bitcoin, and whether lightning’s use of v3 transactions in the future
> would be bad for Bitcoin. If you think that zeroconf is unequivocally bad,
> and that no one will be able to plausibly construct a case that lightning
> is bad, then that qualitative judgment might sway you to not worrying about
> the philosophical issues I've raised above, because these situations can be
> distinguished.
>
> However I am not personally willing to say that I think, overall,
> non-rbf-signaling transactions in use on the network today are bad for
> Bitcoin (or that fullrbf is definitely good – BIP 125’s rbf rules are
> something we’ve been trying to improve upon for years, with little
> success). Nor am I convinced that someone couldn’t put together a cogent
> argument for lightning being bad for Bitcoin, because of its reliance on
> relay policies that are difficult to design and impossible to guarantee as
> part of its security model. So I choose instead to merely make a judgment
> that seems more factually verifiable, which is that non-replacement is a
> policy widely in use on the network today, and we largely don't have reason
> to think (as far as I know!) that the network is seeing a lot of
> transactions that would violate that policy.
>
> If it did turn out that users were commonly signaling non-replacement, but
> then signing and trying to relay doublespends, then I think that would be a
> very good reason for Bitcoin Core to adopt fullrbf to reflect the reality
> of what is happening. In the meantime, I think it makes more sense to say
> that because we have BIP 125, there seems to be no need for users to signal
> one way and behave another, and therefore there is no need to offer
> software that might break a policy that is working well for some users.
> Other software projects might choose differently, and it is after all a
> permissionless network, so if this is in fact an unstable equilibrium that
> will not last, then presumably someday it will be apparent it is not
> working and we’ll abandon it. But I think the philosophy of transaction
> relay policy in Bitcoin Core should be to support disparate use cases in
> order to try to make everything work better, rather than break things
> prematurely because we guess others will break them eventually anyway.
>
> For those that have read this long email and still favor a fullrbf network
> policy (or even just the ability for users to be able to turn on fullrbf
> for themselves), I’d ask for thoughts on the following questions, which
> have guided my thinking on this:
>
> Does fullrbf offer any benefits other than breaking zeroconf business
> practices? If so, what are they?
>
> Is it reasonable to enforce BIP 125's rbf rules on all transactions, if
> those rules themselves are not always incentive compatible?
>
> If someone were to propose a command line option that breaks v3
> transaction relay in the future, is there a logical basis for opposing that
> which is consistent with moving towards fullrbf now?
>
> Cheers,
> Suhas
>
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> [2]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
>
> [3] This is because under the BIP 125 rules, the feerate of the
> replacement transaction is not compared to the individual feerates of all
> transactions being evicted – we just compare feerates with the transactions
> that are directly in conflict (and not their descendants). So it’s possible
> for a transaction that would evict 2 or more transactions to have a higher
> feerate than the direct conflicts, and higher total fee than the set being
> evicted, but have a lower feerate (eg if it is larger) than that of some
> subset of the set of transactions being evicted.
>
> [4] Chaining unconfirmed transactions when the sender might RBF the
> parent is far riskier than if the sender indicates they don't plan to do so
> (chaining onto an RBF transaction creates pinning issues for the sender,
> and risks having the child wiped out if the parent is replaced), so I think
> this is a concrete reason why signaling that a transaction won’t be
> replaced could be useful.
>
> [5] This is a subtle point. I don’t think v3 transactions create an
> unreasonable security assumption for the use case it is being designed for.
> However, I don’t think anyone could rule out the possibility that someone
> could adopt a usage pattern for v3 transactions that subverts the intent of
> this policy. For example, if users started using v3 transactions for all
> their payments, then the limitations on the number of descendants could
> directly interfere with CPFP by a recipient, and someone could argue that
> we should break the policy in order to allow for this hypothetical
> behavior. I think this is a similar form of argument as saying that
> zeroconf practices + BIP 125 create an incentive to double-spend non-rbf
> signaling transactions.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 21009 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 16:25 ` Greg Sanders
@ 2022-10-31 17:21 ` email
2022-10-31 17:51 ` Peter Todd
2022-11-02 3:07 ` Anthony Towns
1 sibling, 1 reply; 33+ messages in thread
From: email @ 2022-10-31 17:21 UTC (permalink / raw)
To: Greg Sanders, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 21618 bytes --]
Protocol Devs,
After reading through this email thread and BIP125, I'm curious if
non-rbf nodes will relay full-rbf transactions and vice versa. That is
to say, if only one non-rbf node exists on the network, however, every
other node implements full-rbf, will the transaction still be
propagated? IE can we always guarantee a path through the network for
either transaction type no matter what the combination of network
policies are?
> Does fullrbf offer any benefits other than breaking zeroconf
> business practices? If so, what are they?
I think AJ mentioned this earlier, but adding more configuration options
always increases code complexity, and with that, there is likely more
unforeseen bugs. However, there is a section of network participants
that rely on both types of transaction policy, so from my limited
view-point, it seems worth accommodating if possible.
Cheers,
-Yancy
On 2022-10-31 17:25, Greg Sanders via bitcoin-dev wrote:
> Thanks for your full thoughts Suhas,
>
> The idea of V3 is that we're currently leaving fees on the table by
> allowing use-cases to be pinned, not that we like Lightning and we
> think miners should stop being profit maximizing somehow to enable
> safer/better layer 2 systems.
>
> If someone wants to bump fees for V3 transactions(or replace them!),
> there's a much simpler "API" to do so than in legacy policy land. The
> fact that it disallows more idiotic ways to add more total fees means
> wallets "shouldn't do that". If it ends up that V3 is disallowing too
> many "natural" ways to fee bump, that's a strike against the V3 idea
> and should be discussed. For 0-conf services we have potential thieves
> who are willing to *out-bid themselves* to have funds come back to
> themselves. It's not a "legitimate" use-case, but a rational one.
>
> I have mostly come around to not pushing for fullrbf due to the issues
> you mentioned, except taking away the option. Removing a
> quite-likely-incentive-compatible option from the software just
> encourages miners to adopt an additional patch if they ever deem it
> necessary to increase their revenue, even if that revenue is from
> hurting 0-conf businesses.
>
> Maybe putting/leaving in a default-false flag for disabling these
> "carve outs" is the least bad option. V3 usage patterns shouldn't
> crumble if a large majority of miners opt out, but 0-conf use cases
> crumble after a small percentage of adoption.
>
> To recap my thoughts:
>
> 1) I have put away my fullrbf hats, I will not advocate anyone running
> it as I think it doesn't really do anything useful for users who
> aren't trying to double-spend merchants.
>
> 2) Forcing miners to honor fees left on the table with respect to
> 0-conf, or forcing them to run a custom patchset to go around it, is a
> step backwards.
>
> Greg
>
> On Mon, Oct 31, 2022 at 11:03 AM Suhas Daftuar via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> AJ,
>>
>> Thanks for the thoughtful post. I think your observations about how
>> we view mempool policy in the Bitcoin Core project, and how that
>> seems to be changing in the discussions around `-mempoolfullrbf`,
>> are on-point and provide a helpful baseline for considering future
>> policy changes.
>>
>> For a long time I viewed fullrbf as an eventuality and I considered
>> myself to be philosophically supportive of the idea. However, after
>> giving this issue some thought in the past few weeks, I am reversing
>> my thinking on this. Concretely, I will argue that we should
>> continue to maintain a relay policy where replacements are rejected
>> for transactions that don't opt-in to RBF (as described in BIP 125),
>> and moreover, that we should remove the `-mempoolfullrbf` flag from
>> Bitcoin Core's latest release candidate and not plan to release
>> software with that flag, unless (or until) circumstances change on
>> the network, which I'll discuss below.
>>
>> This is, of course, a nuanced topic, and among the considerations is
>> a philosophy of how to think about the relay policy and
>> configuration options that we make available in Bitcoin Core (a
>> consideration that is perhaps unique to that project, but I think
>> relevant for this mailing list).
>>
>> I'll start with some technical issues regarding the benefits of
>> enabling fullrbf on the network. In the current BIP 125 regime,
>> every time a transaction is created, a choice is made whether to
>> subject the transaction to BIP 125's RBF rules or not (based on
>> the sequence values of the inputs). So given that users can already
>> opt-in to RBF, the benefit of a "fullrbf" network policy would
>> be if, somehow, RBF users were still denied the benefits of RBF due
>> to the existence of other transactions that don't opt-in.
>>
>> Along those lines, Antoine Riard brought up[1] a DoS vector that is
>> available to someone who wants to interfere with multi-party funded
>> transactions, and suggested that fullrbf would eliminate the
>> problem. After exploring that question again in this thread (thanks
>> to Greg Sanders for clarifying this to me), I understand that the
>> issue is around ensuring that a multiparty (coinjoin-type) protocol
>> is able to make eventual progress, by having a candidate multiparty
>> transaction either eventually confirm or become conflicted with
>> something that has been confirmed, in which case the double-spend
>> information could be used to start a new coinjoin round with fewer
>> participants. The concern Antoine and Greg have brought up is that
>> non-rbf transactions can persist in the mempool ~indefinitely (at a
>> low feerate and not subject to replacement) and interfere with
>> progress being made in a coinjoin protocol.
>>
>> However, it seems to me that similar problems exist for such a
>> protocol even in a fullrbf world, as we understand that term today.
>> I mentioned the ability for rbf "pinning" to interfere with
>> relay of the multiparty transaction (even if the conflicting
>> transaction signals for RBF - a set of large but low feerate
>> conflicting transactions can persist in the mempool and make it
>> difficult for the coinjoin transaction from confirming, at least
>> without attaching a very large fee); and as Greg mentioned in a
>> followup, the BIP 125 rule to only permit 100 transactions to be
>> removed from the mempool at a time during a replacement can also be
>> used to pin a coinjoin protocol in the same way as a non-rbf
>> transaction today. It seems to me that what these multiparty
>> protocols actually need is some sort of "maximal rbf" network
>> policy: a way to guarantee that a transaction which should be
>> desirable for a miner to mine would always get to a miner and
>> considered for inclusion in a block, no matter what the state of
>> node's mempools on the network.
>>
>> While that sounds like a reasonable thing to want on its face (and
>> worth working on), it's not how opt-in RBF works today, nor is it
>> how transaction relay has ever conceptually worked. We have not,
>> thus far, been able to come up with a total ordering on transaction
>> desirability. Moreover, due to all the DoS issues that exist with
>> transaction relay, there are plenty of seemingly legitimate ways to
>> construct transactions that would not relay well on the network.
>> Relay has only ever been a best-efforts concept, where we carve out
>> a small subset of the entire transaction universe for which we try
>> to optimize propagation. The idea behind this approach is that if
>> every use case we can come up with has some way to achieve its goals
>> using transactions that should (eventually) be able to relay, then
>> users wouldn't have much demand for transactions that would
>> deviate from the supported policies, and we therefore shouldn't
>> need to worry too much about incentive compatibility concerns when
>> it comes to transaction types that wouldn't relay at all, even if
>> they are high feerate. (And when those situations arise where the
>> standard transactions do not accommodate some needed use case,
>> developers typically work to define a policy that is compatible with
>> our anti-DoS goals to support such use cases, such as with the
>> recent proposal for version=3 transactions [2].)
>>
>> BIP 125's RBF rules themselves were an effort to carve out just a
>> subset of situations where a transaction should evict conflicting
>> ones -- it was not a design that anyone thought would ensure that
>> all replacements which "should" be mined would always propagate.
>> And I don't believe that we know how to design policy rules that
>> would achieve the goals of this kind of multiparty protocol in a DoS
>> resistant way, today. Along those lines, I would point out that
>> even the BIP 125 design itself is not entirely incentive compatible,
>> in that it is possible to construct a replacement transaction that
>> would evict transactions which would be preferable to be included in
>> a block! [3] (This has been known for years, but fixing this has
>> proven difficult, and the only way to fix it that I'm aware of
>> would be to make BIP 125 RBF even more restrictive than it is today.
>> I do think this is something that needs to be worked on.)
>>
>> Given the limitations of RBF as we have it today, it appears to be
>> incorrect that a fullrbf network policy would solve the problems
>> Antoine raised. And so absent any other examples, it does not seem
>> to me that fullrbf solves any problems for RBF users, who are
>> already free to choose to subject their transactions to BIP 125's
>> RBF policy. From this perspective, "enabling fullrbf" is really
>> just taking away user choice to opt a transaction into a
>> non-replacement policy regime.
>>
>> I think we should ask, then, whether it is reasonable on its face
>> that users might want to opt-in to a non-replacement policy? Or in
>> other words, is it reasonable for a user to mark a transaction as
>> non-replaceable and have that indication be enforced by the network?
>> Note that these are two different questions: you could imagine a
>> world where fullrbf is a dominant policy, but users still use the
>> BIP 125 signaling method to indicate, in an unenforced way, their
>> intention to not replace a transaction. This might give useful
>> information to the network or the recipient for how to interact with
>> such a transaction.
>>
>> And I think that it's entirely possible that users would continue to
>> use the BIP 125 signaling to indicate that they do not intend to
>> replace a transaction. For better or worse, this might be because
>> zeroconf services continue to differentiate their behavior based on
>> such a signal (possibly in conjunction with other factors), or it
>> could be because there are other behaviors that could be utilized
>> more effectively if the transaction originator has made such a
>> signal, such as the recipient chaining an unconfirmed transaction as
>> a way to bump the fee (CPFP) [4].
>>
>> If it were to be the case that users continued to use BIP 125-style
>> signaling to indicate that they do not plan to replace a
>> transaction, would that be harmful to the network? This is not
>> something we can stop in our policy rules (short of censoring such
>> transactions, an obviously bad idea). I think network actors can
>> always do things that we might think are harmful for the network,
>> but that doesn't mean that there are no legitimate use cases for
>> the tools that such actors might be using. Just because someone
>> might use some policy to adopt a zeroconf model, doesn't mean that
>> others aren't using the same policy to achieve benign ends (such
>> as better CPFP behavior).
>>
>> Moreover, while users might attempt to exploit services that offer
>> zeroconf or other differentiated behavior to non-replacement
>> signaling transactions, they also might not -- I think predicting
>> user behavior in this way (and specifically predicting the
>> complexities of what a business might do and whether users might try
>> to subvert it) is beyond the scope of what we can do as protocol
>> developers. Instead, I think we can try to answer a different
>> question: if a group of users were to want the ability to opt-in to
>> a non-replacement policy regime, is that a technically sound option
>> for us to have on the network and enforce in software?
>> Specifically, does that interfere with having a sensible anti-DoS
>> mempool acceptance algorithm, or interfere with other protocols on
>> the network, or necessarily run counter to the interests of miners
>> or node operators?
>>
>> And I think the answer to that question, in looking at the
>> difference between opt-in RBF and fullrbf, is no: offering the
>> ability to opt-in to a non-replacement regime for transactions
>> doesn't introduce any fundamental issues with software or network
>> policy or other protocols. In a world where we only had fullrbf, I
>> could imagine at some point down the road proposing a
>> non-replacement signal myself, because the complexities around
>> transaction chains (and pinning) are more complex for the RBF case
>> than for the non-RBF case (and BIP 125 is not always incentive
>> compatible to begin with!). Conceptually, this is no different to
>> me than the version=3 transaction policy proposal that has been
>> advancing, if we think of it as a special set of restrictions on
>> transactions designed to accommodate a particular use case.
>>
>> Philosophically, I think we should be looking to add non-interfering
>> use cases to what the network supports.
>>
>> To those who argue for making fullrbf a default policy on the
>> network (or even just offering a flag for users to enable fullrbf),
>> I pose this hypothetical: suppose we deploy the v3 transaction
>> policy proposal (which I hope will happen in the near future). That
>> policy would restrict the ways that outputs of a v3 transaction can
>> be spent while the transaction is unconfirmed, including by limiting
>> the number and size of descendants that such a transaction can have,
>> and limiting the types of unconfirmed ancestors that can be
>> included. Suppose in a few years someone proposes that we add a
>> "-disable_v3_transaction_enforcement" flag to our software, to let
>> users decide to turn off those policy restrictions and treat v3
>> transactions the same as v2, for all the same reasons that could be
>> argued today with fullrbf: miners might earn more revenue if we
>> allowed multiple descendant v3 transactions; it's illogical for the
>> recipient of a v3 transaction to believe what is a fundamentally
>> unenforceable promise of a sender to not issue more high value
>> children that descend from an unconfirmed transaction; it's
>> inappropriate for Bitcoin Core to dictate policy on the network and
>> we should honor user choice to turn off that flag if that's what
>> users want; if users are relying on v3's policy restrictions for
>> security then that is an unstable model and we should assume it will
>> get broken[5].
>>
>> It's obvious to me that adding a flag to disable v3 policy would
>> be subversive to making the lightning use case for v3 transactions
>> work. And so my response to such a hypothetical proposal would be
>> to argue that no, we should not enable users to disable this policy,
>> because as long as that policy is just optional and working for
>> those who want it, it shouldn't harm anyone that we offer a
>> tighter set of rules for a particular use case. Adding a way to
>> bypass those rules is just trying to break someone else's use
>> case, not trying to add a new one. We should not wield "incentive
>> compatibility" as a bludgeon for breaking things that appear to be
>> working and not causing others harm.
>>
>> I think this is exactly what is happening with fullrbf.
>>
>> In comparing v3 transaction policy with opting out of transaction
>> replacement, there is of course one significant difference that I
>> have ignored thus far: I think the real difference is an opinion
>> about whether non-replacement transactions that are being used today
>> are, overall, bad for Bitcoin, and whether lightning's use of v3
>> transactions in the future would be bad for Bitcoin. If you think
>> that zeroconf is unequivocally bad, and that no one will be able to
>> plausibly construct a case that lightning is bad, then that
>> qualitative judgment might sway you to not worrying about the
>> philosophical issues I've raised above, because these situations can
>> be distinguished.
>>
>> However I am not personally willing to say that I think, overall,
>> non-rbf-signaling transactions in use on the network today are bad
>> for Bitcoin (or that fullrbf is definitely good - BIP 125's rbf
>> rules are something we've been trying to improve upon for years,
>> with little success). Nor am I convinced that someone couldn't
>> put together a cogent argument for lightning being bad for Bitcoin,
>> because of its reliance on relay policies that are difficult to
>> design and impossible to guarantee as part of its security model.
>> So I choose instead to merely make a judgment that seems more
>> factually verifiable, which is that non-replacement is a policy
>> widely in use on the network today, and we largely don't have reason
>> to think (as far as I know!) that the network is seeing a lot of
>> transactions that would violate that policy.
>>
>> If it did turn out that users were commonly signaling
>> non-replacement, but then signing and trying to relay doublespends,
>> then I think that would be a very good reason for Bitcoin Core to
>> adopt fullrbf to reflect the reality of what is happening. In the
>> meantime, I think it makes more sense to say that because we have
>> BIP 125, there seems to be no need for users to signal one way and
>> behave another, and therefore there is no need to offer software
>> that might break a policy that is working well for some users.
>> Other software projects might choose differently, and it is after
>> all a permissionless network, so if this is in fact an unstable
>> equilibrium that will not last, then presumably someday it will be
>> apparent it is not working and we'll abandon it. But I think the
>> philosophy of transaction relay policy in Bitcoin Core should be to
>> support disparate use cases in order to try to make everything work
>> better, rather than break things prematurely because we guess others
>> will break them eventually anyway.
>>
>> For those that have read this long email and still favor a fullrbf
>> network policy (or even just the ability for users to be able to
>> turn on fullrbf for themselves), I'd ask for thoughts on the
>> following questions, which have guided my thinking on this:
>>
>> Does fullrbf offer any benefits other than breaking zeroconf
>> business practices? If so, what are they?
>>
>> Is it reasonable to enforce BIP 125's rbf rules on all transactions,
>> if those rules themselves are not always incentive compatible?
>>
>> If someone were to propose a command line option that breaks v3
>> transaction relay in the future, is there a logical basis for
>> opposing that which is consistent with moving towards fullrbf now?
>>
>> Cheers,
>> Suhas
>>
>> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
>> [2]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
>
>> [3] This is because under the BIP 125 rules, the feerate of the
>> replacement transaction is not compared to the individual feerates
>> of all transactions being evicted - we just compare feerates with
>> the transactions that are directly in conflict (and not their
>> descendants). So it's possible for a transaction that would evict
>> 2 or more transactions to have a higher feerate than the direct
>> conflicts, and higher total fee than the set being evicted, but have
>> a lower feerate (eg if it is larger) than that of some subset of the
>> set of transactions being evicted.
>>
>> [4] Chaining unconfirmed transactions when the sender might RBF the
>> parent is far riskier than if the sender indicates they don't plan
>> to do so (chaining onto an RBF transaction creates pinning issues
>> for the sender, and risks having the child wiped out if the parent
>> is replaced), so I think this is a concrete reason why signaling
>> that a transaction won't be replaced could be useful.
>>
>> [5] This is a subtle point. I don't think v3 transactions create
>> an unreasonable security assumption for the use case it is being
>> designed for. However, I don't think anyone could rule out the
>> possibility that someone could adopt a usage pattern for v3
>> transactions that subverts the intent of this policy. For example,
>> if users started using v3 transactions for all their payments, then
>> the limitations on the number of descendants could directly
>> interfere with CPFP by a recipient, and someone could argue that we
>> should break the policy in order to allow for this hypothetical
>> behavior. I think this is a similar form of argument as saying that
>> zeroconf practices + BIP 125 create an incentive to double-spend
>> non-rbf signaling transactions.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[-- Attachment #2: Type: text/html, Size: 25152 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 17:21 ` email
@ 2022-10-31 17:51 ` Peter Todd
2022-11-04 10:28 ` email
0 siblings, 1 reply; 33+ messages in thread
From: Peter Todd @ 2022-10-31 17:51 UTC (permalink / raw)
To: email, Bitcoin Protocol Discussion; +Cc: Greg Sanders
[-- Attachment #1: Type: text/plain, Size: 2414 bytes --]
On Mon, Oct 31, 2022 at 06:21:08PM +0100, yancy via bitcoin-dev wrote:
>
> Protocol Devs,
>
> After reading through this email thread and BIP125, I'm curious if non-rbf
> nodes will relay full-rbf transactions and vice versa. That is to say, if
> only one non-rbf node exists on the network, however, every other node
> implements full-rbf, will the transaction still be propagated? IE can we
> always guarantee a path through the network for either transaction type no
> matter what the combination of network policies are?
1) There are nodes that signal full-rbf, and preferentially peer to each other,
thus ensuring good transaction propagation. The most recent patch to implement
this is: https://github.com/bitcoin/bitcoin/pull/25600
There's enough peers running full-rbf that the last time I started up a new
node on a fresh IP address, it happened to have a peer relaying full-rbf
replacements to it. And of course, if people want full-rbf to work more
reliably, it's very easy to just run some nodes with a large number of outgoing
peers. Changing the hard-coded 8 outgoing peers to, say, 800, isn't very hard.
2) There's nothing special about a "full-rbf transaction" other than the fact
that it's replacing a previously broadcast transaction that didn't signal
replacement. There is not consensus over the mempool, so in certain cases
non-full-rbf nodes will in fact broadcast replacements when they didn't happen
to receive the "first" transaction first.
The latter makes testing full-rbf a bit problematic, as if you don't take
special measures to ensure good propagation a small % of the time the
"replacement" transaction will in fact be the one that gets gets mined.
> > Does fullrbf offer any benefits other than breaking zeroconf
> > business practices? If so, what are they?
>
> I think AJ mentioned this earlier, but adding more configuration options
> always increases code complexity, and with that, there is likely more
> unforeseen bugs. However, there is a section of network participants that
> rely on both types of transaction policy, so from my limited view-point, it
> seems worth accommodating if possible.
Since all the machinery to do replacemnt already exists, adding a full-rbf
config flag is particularly trivial. It requires just a single line in the
mempool code.
--
https://petertodd.org 'peter'[:-1]@petertodd.org
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 16:25 ` Greg Sanders
2022-10-31 17:21 ` email
@ 2022-11-02 3:07 ` Anthony Towns
2022-11-02 13:32 ` Greg Sanders
1 sibling, 1 reply; 33+ messages in thread
From: Anthony Towns @ 2022-11-02 3:07 UTC (permalink / raw)
To: Greg Sanders, Bitcoin Protocol Discussion
On Mon, Oct 31, 2022 at 12:25:46PM -0400, Greg Sanders via bitcoin-dev wrote:
> For 0-conf services we have potential thieves who are willing
> to *out-bid themselves* to have funds come back to themselves. It's not a
> "legitimate" use-case, but a rational one.
I think that's a huge oversimplification of "rational" -- otherwise
you might as well say that deliberately pinning txs is also rational,
because it allows the person doing the pinning to steal funds from their
counterparty by forcing a timeout to expire.
There's no need for us as developers, or us as node operators, to support
every use case that some individual might find rational at some point in
time. After all, it might be individually rational for someone to want the
subsidy to stop decreasing, or to include 8MB of transactions per block.
Note that it's also straightforwardly rational and incentive compatible
for miners to not want this patch to be available, under the following
scenario:
- a significant number of on-chain txs are for zeroconf services
- fee income would be reduced if zeroconf services went away
(both directly due to the absence of zeroconf payments, and by
reducing mempool pressure, reducing fee income from the on-chain txs
that remain)
- miners adopting fullrbf would cause zeroconf services to go away,
(and won't enable a comparable volume of new services that generates
comparable transaction volume)
- including the option in core would make other miners adopting
fullrbf more likely
I think the first three of those are fairly straightforward and objective,
at least at this point in time. The last is just a risk; but without
any counterbalancing benefit, why take it?
Gaining a few thousand sats due to high feerate replacement txs from
people exploiting zeroconf services for a few months before all those
services shutdown doesn't make up for the lost fee income over the months
or years it might have otherwise taken people to naturally switch to
some better alternative.
Even if fullrbf worked for preventing pinning that likely doesn't directly
result in much additional fee income: once you know that pinning doesn't
work, you just don't try it, which means there's no opportunity for
miners to profit from a bidding war from the pinners counterparties
repeatedly RBFing their preferred tx to get it mined.
That also excludes second order risks: if you can't do zeroconf with BTC
anymore, do you switch to ERC20 tokens, and then trade your BTC savings
for ETH or USDT, and do enough people do that to lower the price of BTC?
If investors see BTC being less used for payments, does that lower their
confidence in bitcoin's future, and cause them to sell?
> Removing a
> quite-likely-incentive-compatible option from the software just encourages
> miners to adopt an additional patch
Why shouldn't miners adopt an additional patch if they want some unusual
functionality?
Don't we want/expect miners to have the ability to change the code in
meaningful ways, at a minimum to be able to cope with the scenario where
core somehow gets coopted and releases bad code, or to be able to deal
with the case where an emergency patch is needed?
Is there any evidence miners even want this option? Peter suggested
that some non-signalling replacements were being mined already [0], but
as far as I can see [1] all of those are simply due to the transaction
they replaced not having propagated in the first place (or having been
evicted somehow? hard to tell without any data on the original tx).
[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021012.html
[1] https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292692367
> 2) Forcing miners to honor fees left on the table with respect to 0-conf,
> or forcing them to run a custom patchset to go around it, is a step
> backwards.
As you already acknowledged, any miner that wants this behaviour can just
pick up the patch (or could run Knots, which already has the feature
enabled by default). It's simply false to say miners are being forced
to do anything, no matter what we do here.
If the direction you're facing is one where you're moving towards making
life easier for people to commit fraud, and driving away businesses
that aren't doing anyone harm, without achieving anything much else;
then taking a step backwards seems like a sensible thing to do to me.
(I remain optimistic about coming up with better RBF policy, and willing
to be gung ho about everyone switching over to it even if it does kill
off zeroconf, provided it actually does some good and we give people 6
months or more notice that it's definitely happening and what exactly
the new rules will be, though)
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-02 3:07 ` Anthony Towns
@ 2022-11-02 13:32 ` Greg Sanders
0 siblings, 0 replies; 33+ messages in thread
From: Greg Sanders @ 2022-11-02 13:32 UTC (permalink / raw)
To: Anthony Towns; +Cc: Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 5781 bytes --]
> I think that's a huge oversimplification of "rational" -- otherwise
you might as well say that deliberately pinning txs is also rational,
because it allows the person doing the pinning to steal funds from their
counterparty by forcing a timeout to expire.
To be clear, a pinner is attempting to *not* pay
the most fees, by definition. If we're somehow sure something is a pin,
we should not allow it, because miners rationally do not want it vs
an "honest" bid for fees. V3 design is one attempt to carve out a safe
space for fee bidding. Carving out a safe space for *non-bidding* is not the
same thing.
I think this mostly boils down having knobs or not. I'm fine with knobs
with paternalistic defaults, especially when a non-zero percentage of users
disagree with a value in either direction.
Greg
On Tue, Nov 1, 2022 at 11:07 PM Anthony Towns <aj@erisian•com.au> wrote:
> On Mon, Oct 31, 2022 at 12:25:46PM -0400, Greg Sanders via bitcoin-dev
> wrote:
> > For 0-conf services we have potential thieves who are willing
> > to *out-bid themselves* to have funds come back to themselves. It's not a
> > "legitimate" use-case, but a rational one.
>
> I think that's a huge oversimplification of "rational" -- otherwise
> you might as well say that deliberately pinning txs is also rational,
> because it allows the person doing the pinning to steal funds from their
> counterparty by forcing a timeout to expire.
>
> There's no need for us as developers, or us as node operators, to support
> every use case that some individual might find rational at some point in
> time. After all, it might be individually rational for someone to want the
> subsidy to stop decreasing, or to include 8MB of transactions per block.
>
> Note that it's also straightforwardly rational and incentive compatible
> for miners to not want this patch to be available, under the following
> scenario:
>
> - a significant number of on-chain txs are for zeroconf services
> - fee income would be reduced if zeroconf services went away
> (both directly due to the absence of zeroconf payments, and by
> reducing mempool pressure, reducing fee income from the on-chain txs
> that remain)
> - miners adopting fullrbf would cause zeroconf services to go away,
> (and won't enable a comparable volume of new services that generates
> comparable transaction volume)
> - including the option in core would make other miners adopting
> fullrbf more likely
>
> I think the first three of those are fairly straightforward and objective,
> at least at this point in time. The last is just a risk; but without
> any counterbalancing benefit, why take it?
>
> Gaining a few thousand sats due to high feerate replacement txs from
> people exploiting zeroconf services for a few months before all those
> services shutdown doesn't make up for the lost fee income over the months
> or years it might have otherwise taken people to naturally switch to
> some better alternative.
>
> Even if fullrbf worked for preventing pinning that likely doesn't directly
> result in much additional fee income: once you know that pinning doesn't
> work, you just don't try it, which means there's no opportunity for
> miners to profit from a bidding war from the pinners counterparties
> repeatedly RBFing their preferred tx to get it mined.
>
> That also excludes second order risks: if you can't do zeroconf with BTC
> anymore, do you switch to ERC20 tokens, and then trade your BTC savings
> for ETH or USDT, and do enough people do that to lower the price of BTC?
> If investors see BTC being less used for payments, does that lower their
> confidence in bitcoin's future, and cause them to sell?
>
> > Removing a
> > quite-likely-incentive-compatible option from the software just
> encourages
> > miners to adopt an additional patch
>
> Why shouldn't miners adopt an additional patch if they want some unusual
> functionality?
>
> Don't we want/expect miners to have the ability to change the code in
> meaningful ways, at a minimum to be able to cope with the scenario where
> core somehow gets coopted and releases bad code, or to be able to deal
> with the case where an emergency patch is needed?
>
> Is there any evidence miners even want this option? Peter suggested
> that some non-signalling replacements were being mined already [0], but
> as far as I can see [1] all of those are simply due to the transaction
> they replaced not having propagated in the first place (or having been
> evicted somehow? hard to tell without any data on the original tx).
>
> [0]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021012.html
> [1] https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292692367
>
> > 2) Forcing miners to honor fees left on the table with respect to 0-conf,
> > or forcing them to run a custom patchset to go around it, is a step
> > backwards.
>
> As you already acknowledged, any miner that wants this behaviour can just
> pick up the patch (or could run Knots, which already has the feature
> enabled by default). It's simply false to say miners are being forced
> to do anything, no matter what we do here.
>
> If the direction you're facing is one where you're moving towards making
> life easier for people to commit fraud, and driving away businesses
> that aren't doing anyone harm, without achieving anything much else;
> then taking a step backwards seems like a sensible thing to do to me.
>
> (I remain optimistic about coming up with better RBF policy, and willing
> to be gung ho about everyone switching over to it even if it does kill
> off zeroconf, provided it actually does some good and we give people 6
> months or more notice that it's definitely happening and what exactly
> the new rules will be, though)
>
> Cheers,
> aj
>
[-- Attachment #2: Type: text/html, Size: 6915 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 13:02 ` Suhas Daftuar
2022-10-31 16:25 ` Greg Sanders
@ 2022-11-02 19:50 ` Antoine Riard
2022-11-05 2:35 ` Peter Todd
2 siblings, 0 replies; 33+ messages in thread
From: Antoine Riard @ 2022-11-02 19:50 UTC (permalink / raw)
To: Suhas Daftuar, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 27757 bytes --]
Hi Suhas,
From my understanding, the main crux of the reasoning exposed in your post
would be to solidify the transaction-relay paradigm we have been following
during the last years, e.g introducing the carve-out rule specifically for
lightning commitment transactions, or more recently version=3 transactions.
I think this paradigm could be described summarly as "to each use-case
belongs a set of transaction-relay policy rules". Some of this set of rules
could aim to non-replacement guarantees to the consumers of transactions
signaling under this regime (e.g zeroconf). Another set of rules could
provide a high-guarantee that a transaction would always get to a miner, no
matter what the state of node's mempools on the network (e.g "maximal rbf"
for contracting protocols).
First, coming out of my mind, we would have to consider isolation between
each set of policy rules, ensuring the signaling mechanism cannot be abused
by an attacker to create a new pinning vector. E.g, a hypothetical concern
could be BIP125 rules interfering with version=3 policy to block the
replacement of better ancestor feerate packages. Further, for each set of
policy rules arises the question of internal consistency, again an attacker
could abuse them to pin transactions
propagation. I think an earlier version of version=3 was suffering from
this concern of not scoping potential "junk" ancestors. None of those
issues are unsolvable, however we should be well-aware of the
non-negligeable design complexity encumbered by transaction-relay protocol
developers to achieve the correct goal. There is not only a need to ensure
careful policy rules security analysis, but further to communicate well
their usage to second-layers and wallets developers (a task far from easy
with all the confusions contained by current BIP125).
Now, in the evaluation process of a set of policy rules soundness are
proposed a few reasoning heuristics: namely that it shouldn't interfere
sensibly with a anti-DoS mempool acceptance algorithm, or shouldn't
interfere with other protocols on the network, or counter the interests of
miners or node operators. I hold the belief the latest question could be
the one raising the most concerns. Browsing in the history of Bitcoin Core,
I think one of the design goals aimed for has always been to level the
playing field between miners, e.g BIP152 improving block transfer latency
to reduce orphan rate. Following this principle, we might wonder if our
transaction-relay network should guarantee some equal access to transaction
information to all the miners.
In the present case of a non-replacement policy regime, we could see the
following situation to arise, on one side miners deploying private
transaction-relay communication channels or API to capture higher fees
income from non-standard transactions. On the other-side, transaction
issuers or consumers bypass the standard transaction-relay policy rules.
Bypass could be motivated by either a zeroconf service double-spend, or
faster confirmation of a collaborative transaction. E.g, to reuse the
example of unconfirmed transaction chaining, where the sender commit to
non-replacement by opting out from the RBF flag, this commitment could be
reevaluated in the light of changing network mempools congestion, or
liquidity preferences (e.g the quick need to open LN routing channels). The
sender could leverage such hypothetical private transaction-relay
communication channels to revoke its non-replacement commitment. Therefore
discrepancies between a set of policy rules design and miners incentives
sounds to lead to informational asymmetries, harmful for the long-term
decentralization of the mining ecosystem. Of course, miner incomes
asymmetries due to edge in transaction flows access might not be weighted
as serious today, in a world where transaction fees contribute to most of
the block reward, this is far more worrying!
Of course, one position could be to estimate that miner centralization is
beyond the scope of responsibility of the Bitcoin Core project. Or at least
as a result of lightweighted risks.
Such discrepancy between a set of policy rules design and miners incentives
could also lead to hidden security risks for second-layers. As we see more
security assumptions made on policy rules extension, e.g version=3, a
lightning channel counterparty could have a competing interest to forge a
raw package suiting better incentives, and as such nullify the security
advantage expected. This could be seen as a loose concern, however the last
time we have seen an actor deliberately providing non-standard transactions
to a miner to break second-layers was yesterday [0]!. From observing other
cryptocurrencies spaces, such "MEV-style" attacks could be more and more
concerning [1]. I don't think we should assume miners to behave as "network
gentlemen", in a world where mining can be anonymous, permissionless and
censorship-resistant (e.g Stratum V2 giving back template construction to
miners operators rather than pools).
According to me, one of the harder problem we're seeing with this fullrbf
discussion is the lack of a consistent, grounded and well-understood miner
incentive model, where not only block template construction but also
transaction collection and replacement strategies are analyzed, and against
which we could simulate the efficiency of a policy. Assuming we would have
such a model, rather than qualify a policy rule as incentive-compatible in
a binary fashion, we could evaluate them on a scale, and agree on when
they're satisfying enough in face of technical complexity, validation
resources, margin of adversarial exploitation, or whatever other relevant
criteria.
Answering a few other points raised in this post, what appears to me
obscure is the qualification that fullrbf doesn't solve the DoS issues for
contracting protocols (e..g coinjoin/dual-funded lightning). If I can
understand, it's on the ground that the imperfections of BIP125 underscore
only the direct conflict feerate. It should be remembered that allowing
replacement without considering the opting flag would be already an
improvement against the DoS attack, as the attack would have to offer a
more compelling feerate to maintain the pin. Improvements of BIP125 can
happen on the top, but solving the opt-out double-spend issue sounds to me
a prerequisite.
Beyond that, I think few questions are laid out on the conceptual soundness
of v3 transaction policy w.r.t concerns raised about fullrbf today. In my
opinion, I'm sadly with most of them, especially that miners might earn
more revenue if we allowed multiple descendant v3 transactions and the
unenforceable promise for the recipient of such package to not add more
high-value children, I've echoed those concerns earlier in the review of
nVersion=3 proposal [2]. We might have to swallow the bullet for now, and
be okay as lightning developers and operators that there is only a social
inertia of the miners and lack of reliable communication channels towards
them by an adversarial counterparty to offer security [3]. Additionally, I
think it would be acceptable to have an option to disable v3 transaction
policy, an operator could be willing to reduce the CPU/memory DoS surface
of its node from partaking to any package relay. Even if it comes at the
loss of a better view of blockspace demand and downgrades its
fee-estimation, I think we should give the maximum flexibility to operators
in choosing their risk model.
To put it in a nutshell, if we would like to pursue further in the paradigm
that "to each use-case belongs its set of policy rules" (as long as they
don't introduce any harm for the network stakeholders), I believe we would
be more grounded with a better quantitative understanding of so-called
"miners incentives". I'm still wondering if it's realistic to deploy policy
rules that are not sustainable in face of long-term mining dynamics.
Best,
Antoine
[0] https://github.com/lightningnetwork/lnd/issues/7096
[1] On risks of introducing miner harvesting attacks, especially when you
consider implications on lightning-style constructions
https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-February/002569.html
and
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-November/019615.html
[2]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020939.html
"If you're a miner and you receive a non-V3, second descendant of an
unconfirmed V3 transaction, if the offered fee is in the top mempool
backlog, I think you would have an interest to accept such a transaction.
So I'm not sure if those two rules are compatible with miners incentives..."
[3] This wonders if we should look forward in the future to lock in the
CPFP weight of a Lightning commitment with some new consensus semantic, or
leveraging any covenant magic, cf.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-March/020122.html
and
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/020991.html
Le lun. 31 oct. 2022 à 11:02, Suhas Daftuar via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> a écrit :
> AJ,
>
> Thanks for the thoughtful post. I think your observations about how we
> view mempool policy in the Bitcoin Core project, and how that seems to be
> changing in the discussions around `-mempoolfullrbf`, are on-point and
> provide a helpful baseline for considering future policy changes.
>
> For a long time I viewed fullrbf as an eventuality and I considered myself
> to be philosophically supportive of the idea. However, after giving this
> issue some thought in the past few weeks, I am reversing my thinking on
> this. Concretely, I will argue that we should continue to maintain a relay
> policy where replacements are rejected for transactions that don't opt-in
> to RBF (as described in BIP 125), and moreover, that we should remove the
> `-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not
> plan to release software with that flag, unless (or until) circumstances
> change on the network, which I'll discuss below.
>
> This is, of course, a nuanced topic, and among the considerations is a
> philosophy of how to think about the relay policy and configuration options
> that we make available in Bitcoin Core (a consideration that is perhaps
> unique to that project, but I think relevant for this mailing list).
>
> I'll start with some technical issues regarding the benefits of enabling
> fullrbf on the network. In the current BIP 125 regime, every time a
> transaction is created, a choice is made whether to subject the transaction
> to BIP 125’s RBF rules or not (based on the sequence values of the
> inputs). So given that users can already opt-in to RBF, the benefit of a
> “fullrbf” network policy would be if, somehow, RBF users were still denied
> the benefits of RBF due to the existence of other transactions that don’t
> opt-in.
>
> Along those lines, Antoine Riard brought up[1] a DoS vector that is
> available to someone who wants to interfere with multi-party funded
> transactions, and suggested that fullrbf would eliminate the problem.
> After exploring that question again in this thread (thanks to Greg Sanders
> for clarifying this to me), I understand that the issue is around ensuring
> that a multiparty (coinjoin-type) protocol is able to make eventual
> progress, by having a candidate multiparty transaction either eventually
> confirm or become conflicted with something that has been confirmed, in
> which case the double-spend information could be used to start a new
> coinjoin round with fewer participants. The concern Antoine and Greg have
> brought up is that non-rbf transactions can persist in the mempool
> ~indefinitely (at a low feerate and not subject to replacement) and
> interfere with progress being made in a coinjoin protocol.
>
> However, it seems to me that similar problems exist for such a protocol
> even in a fullrbf world, as we understand that term today. I mentioned the
> ability for rbf “pinning” to interfere with relay of the multiparty
> transaction (even if the conflicting transaction signals for RBF – a set of
> large but low feerate conflicting transactions can persist in the mempool
> and make it difficult for the coinjoin transaction from confirming, at
> least without attaching a very large fee); and as Greg mentioned in a
> followup, the BIP 125 rule to only permit 100 transactions to be removed
> from the mempool at a time during a replacement can also be used to pin a
> coinjoin protocol in the same way as a non-rbf transaction today. It seems
> to me that what these multiparty protocols actually need is some sort of
> "maximal rbf" network policy: a way to guarantee that a transaction which
> should be desirable for a miner to mine would always get to a miner and
> considered for inclusion in a block, no matter what the state of node’s
> mempools on the network.
>
> While that sounds like a reasonable thing to want on its face (and worth
> working on), it's not how opt-in RBF works today, nor is it how transaction
> relay has ever conceptually worked. We have not, thus far, been able to
> come up with a total ordering on transaction desirability. Moreover, due
> to all the DoS issues that exist with transaction relay, there are plenty
> of seemingly legitimate ways to construct transactions that would not relay
> well on the network. Relay has only ever been a best-efforts concept,
> where we carve out a small subset of the entire transaction universe for
> which we try to optimize propagation. The idea behind this approach is
> that if every use case we can come up with has some way to achieve its
> goals using transactions that should (eventually) be able to relay, then
> users wouldn’t have much demand for transactions that would deviate from
> the supported policies, and we therefore shouldn’t need to worry too much
> about incentive compatibility concerns when it comes to transaction types
> that wouldn’t relay at all, even if they are high feerate. (And when those
> situations arise where the standard transactions do not accommodate some
> needed use case, developers typically work to define a policy that is
> compatible with our anti-DoS goals to support such use cases, such as with
> the recent proposal for version=3 transactions [2].)
>
> BIP 125's RBF rules themselves were an effort to carve out just a subset
> of situations where a transaction should evict conflicting ones -- it was
> not a design that anyone thought would ensure that all replacements which
> "should" be mined would always propagate. And I don't believe that we know
> how to design policy rules that would achieve the goals of this kind of
> multiparty protocol in a DoS resistant way, today. Along those lines, I
> would point out that even the BIP 125 design itself is not entirely
> incentive compatible, in that it is possible to construct a replacement
> transaction that would evict transactions which would be preferable to be
> included in a block! [3] (This has been known for years, but fixing this
> has proven difficult, and the only way to fix it that I’m aware of would be
> to make BIP 125 RBF even more restrictive than it is today. I do think this
> is something that needs to be worked on.)
>
> Given the limitations of RBF as we have it today, it appears to be
> incorrect that a fullrbf network policy would solve the problems Antoine
> raised. And so absent any other examples, it does not seem to me that
> fullrbf solves any problems for RBF users, who are already free to choose
> to subject their transactions to BIP 125’s RBF policy. From this
> perspective, "enabling fullrbf" is really just taking away user choice to
> opt a transaction into a non-replacement policy regime.
>
> I think we should ask, then, whether it is reasonable on its face that
> users might want to opt-in to a non-replacement policy? Or in other words,
> is it reasonable for a user to mark a transaction as non-replaceable and
> have that indication be enforced by the network? Note that these are two
> different questions: you could imagine a world where fullrbf is a dominant
> policy, but users still use the BIP 125 signaling method to indicate, in an
> unenforced way, their intention to not replace a transaction. This might
> give useful information to the network or the recipient for how to interact
> with such a transaction.
>
> And I think that it's entirely possible that users would continue to use
> the BIP 125 signaling to indicate that they do not intend to replace a
> transaction. For better or worse, this might be because zeroconf services
> continue to differentiate their behavior based on such a signal (possibly
> in conjunction with other factors), or it could be because there are other
> behaviors that could be utilized more effectively if the transaction
> originator has made such a signal, such as the recipient chaining an
> unconfirmed transaction as a way to bump the fee (CPFP) [4].
>
> If it were to be the case that users continued to use BIP 125-style
> signaling to indicate that they do not plan to replace a transaction, would
> that be harmful to the network? This is not something we can stop in our
> policy rules (short of censoring such transactions, an obviously bad
> idea). I think network actors can always do things that we might think are
> harmful for the network, but that doesn’t mean that there are no legitimate
> use cases for the tools that such actors might be using. Just because
> someone might use some policy to adopt a zeroconf model, doesn’t mean that
> others aren’t using the same policy to achieve benign ends (such as better
> CPFP behavior).
>
> Moreover, while users might attempt to exploit services that offer
> zeroconf or other differentiated behavior to non-replacement signaling
> transactions, they also might not -- I think predicting user behavior in
> this way (and specifically predicting the complexities of what a business
> might do and whether users might try to subvert it) is beyond the scope of
> what we can do as protocol developers. Instead, I think we can try to
> answer a different question: if a group of users were to want the ability
> to opt-in to a non-replacement policy regime, is that a technically sound
> option for us to have on the network and enforce in software?
> Specifically, does that interfere with having a sensible anti-DoS mempool
> acceptance algorithm, or interfere with other protocols on the network, or
> necessarily run counter to the interests of miners or node operators?
>
> And I think the answer to that question, in looking at the difference
> between opt-in RBF and fullrbf, is no: offering the ability to opt-in to a
> non-replacement regime for transactions doesn't introduce any fundamental
> issues with software or network policy or other protocols. In a world
> where we only had fullrbf, I could imagine at some point down the road
> proposing a non-replacement signal myself, because the complexities around
> transaction chains (and pinning) are more complex for the RBF case than for
> the non-RBF case (and BIP 125 is not always incentive compatible to begin
> with!). Conceptually, this is no different to me than the version=3
> transaction policy proposal that has been advancing, if we think of it as a
> special set of restrictions on transactions designed to accommodate a
> particular use case.
>
> Philosophically, I think we should be looking to add non-interfering use
> cases to what the network supports.
>
> To those who argue for making fullrbf a default policy on the network (or
> even just offering a flag for users to enable fullrbf), I pose this
> hypothetical: suppose we deploy the v3 transaction policy proposal (which I
> hope will happen in the near future). That policy would restrict the ways
> that outputs of a v3 transaction can be spent while the transaction is
> unconfirmed, including by limiting the number and size of descendants that
> such a transaction can have, and limiting the types of unconfirmed
> ancestors that can be included. Suppose in a few years someone proposes
> that we add a "-disable_v3_transaction_enforcement" flag to our software,
> to let users decide to turn off those policy restrictions and treat v3
> transactions the same as v2, for all the same reasons that could be argued
> today with fullrbf: miners might earn more revenue if we allowed multiple
> descendant v3 transactions; it's illogical for the recipient of a v3
> transaction to believe what is a fundamentally unenforceable promise of a
> sender to not issue more high value children that descend from an
> unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
> policy on the network and we should honor user choice to turn off that flag
> if that’s what users want; if users are relying on v3’s policy restrictions
> for security then that is an unstable model and we should assume it will
> get broken[5].
>
> It’s obvious to me that adding a flag to disable v3 policy would be
> subversive to making the lightning use case for v3 transactions work. And
> so my response to such a hypothetical proposal would be to argue that no,
> we should not enable users to disable this policy, because as long as that
> policy is just optional and working for those who want it, it shouldn’t
> harm anyone that we offer a tighter set of rules for a particular use
> case. Adding a way to bypass those rules is just trying to break someone
> else’s use case, not trying to add a new one. We should not wield
> "incentive compatibility" as a bludgeon for breaking things that appear to
> be working and not causing others harm.
>
> I think this is exactly what is happening with fullrbf.
>
> In comparing v3 transaction policy with opting out of transaction
> replacement, there is of course one significant difference that I have
> ignored thus far: I think the real difference is an opinion about whether
> non-replacement transactions that are being used today are, overall, bad
> for Bitcoin, and whether lightning’s use of v3 transactions in the future
> would be bad for Bitcoin. If you think that zeroconf is unequivocally bad,
> and that no one will be able to plausibly construct a case that lightning
> is bad, then that qualitative judgment might sway you to not worrying about
> the philosophical issues I've raised above, because these situations can be
> distinguished.
>
> However I am not personally willing to say that I think, overall,
> non-rbf-signaling transactions in use on the network today are bad for
> Bitcoin (or that fullrbf is definitely good – BIP 125’s rbf rules are
> something we’ve been trying to improve upon for years, with little
> success). Nor am I convinced that someone couldn’t put together a cogent
> argument for lightning being bad for Bitcoin, because of its reliance on
> relay policies that are difficult to design and impossible to guarantee as
> part of its security model. So I choose instead to merely make a judgment
> that seems more factually verifiable, which is that non-replacement is a
> policy widely in use on the network today, and we largely don't have reason
> to think (as far as I know!) that the network is seeing a lot of
> transactions that would violate that policy.
>
> If it did turn out that users were commonly signaling non-replacement, but
> then signing and trying to relay doublespends, then I think that would be a
> very good reason for Bitcoin Core to adopt fullrbf to reflect the reality
> of what is happening. In the meantime, I think it makes more sense to say
> that because we have BIP 125, there seems to be no need for users to signal
> one way and behave another, and therefore there is no need to offer
> software that might break a policy that is working well for some users.
> Other software projects might choose differently, and it is after all a
> permissionless network, so if this is in fact an unstable equilibrium that
> will not last, then presumably someday it will be apparent it is not
> working and we’ll abandon it. But I think the philosophy of transaction
> relay policy in Bitcoin Core should be to support disparate use cases in
> order to try to make everything work better, rather than break things
> prematurely because we guess others will break them eventually anyway.
>
> For those that have read this long email and still favor a fullrbf network
> policy (or even just the ability for users to be able to turn on fullrbf
> for themselves), I’d ask for thoughts on the following questions, which
> have guided my thinking on this:
>
> Does fullrbf offer any benefits other than breaking zeroconf business
> practices? If so, what are they?
>
> Is it reasonable to enforce BIP 125's rbf rules on all transactions, if
> those rules themselves are not always incentive compatible?
>
> If someone were to propose a command line option that breaks v3
> transaction relay in the future, is there a logical basis for opposing that
> which is consistent with moving towards fullrbf now?
>
> Cheers,
> Suhas
>
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> [2]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
>
> [3] This is because under the BIP 125 rules, the feerate of the
> replacement transaction is not compared to the individual feerates of all
> transactions being evicted – we just compare feerates with the transactions
> that are directly in conflict (and not their descendants). So it’s possible
> for a transaction that would evict 2 or more transactions to have a higher
> feerate than the direct conflicts, and higher total fee than the set being
> evicted, but have a lower feerate (eg if it is larger) than that of some
> subset of the set of transactions being evicted.
>
> [4] Chaining unconfirmed transactions when the sender might RBF the
> parent is far riskier than if the sender indicates they don't plan to do so
> (chaining onto an RBF transaction creates pinning issues for the sender,
> and risks having the child wiped out if the parent is replaced), so I think
> this is a concrete reason why signaling that a transaction won’t be
> replaced could be useful.
>
> [5] This is a subtle point. I don’t think v3 transactions create an
> unreasonable security assumption for the use case it is being designed for.
> However, I don’t think anyone could rule out the possibility that someone
> could adopt a usage pattern for v3 transactions that subverts the intent of
> this policy. For example, if users started using v3 transactions for all
> their payments, then the limitations on the number of descendants could
> directly interfere with CPFP by a recipient, and someone could argue that
> we should break the policy in order to allow for this hypothetical
> behavior. I think this is a similar form of argument as saying that
> zeroconf practices + BIP 125 create an incentive to double-spend non-rbf
> signaling transactions.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 28756 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 17:51 ` Peter Todd
@ 2022-11-04 10:28 ` email
0 siblings, 0 replies; 33+ messages in thread
From: email @ 2022-11-04 10:28 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Protocol Discussion, Greg Sanders
[-- Attachment #1: Type: text/plain, Size: 3253 bytes --]
Peter,
> There's nothing special about a "full-rbf transaction" other than the
> fact
> that it's replacing a previously broadcast transaction that didn't
> signal
> replacement.
Thanks, this is a piece I haven't seen. It sounds like "full-rbf"
policy is fundamentally different from BIP125, where in BIP125 a
transaction must signal that it can be replaced. If I'm reading what
you said correctly, then "full-rbf" policy will allow the replacement of
any transaction, whether it's signaled or not..
> Since all the machinery to do replacemnt already exists, adding a
> full-rbf
> config flag is particularly trivial. It requires just a single line in
> the
> mempool code.
Agree the flag is trivial. The interplay between mempool policies may
not be trivial.
Cheers,
-Yancy
On 2022-10-31 18:51, Peter Todd wrote:
> On Mon, Oct 31, 2022 at 06:21:08PM +0100, yancy via bitcoin-dev wrote:
>
>> Protocol Devs,
>>
>> After reading through this email thread and BIP125, I'm curious if
>> non-rbf
>> nodes will relay full-rbf transactions and vice versa. That is to
>> say, if
>> only one non-rbf node exists on the network, however, every other node
>> implements full-rbf, will the transaction still be propagated? IE can
>> we
>> always guarantee a path through the network for either transaction
>> type no
>> matter what the combination of network policies are?
>
> 1) There are nodes that signal full-rbf, and preferentially peer to
> each other,
> thus ensuring good transaction propagation. The most recent patch to
> implement
> this is: https://github.com/bitcoin/bitcoin/pull/25600
>
> There's enough peers running full-rbf that the last time I started up a
> new
> node on a fresh IP address, it happened to have a peer relaying
> full-rbf
> replacements to it. And of course, if people want full-rbf to work more
> reliably, it's very easy to just run some nodes with a large number of
> outgoing
> peers. Changing the hard-coded 8 outgoing peers to, say, 800, isn't
> very hard.
>
> 2) There's nothing special about a "full-rbf transaction" other than
> the fact
> that it's replacing a previously broadcast transaction that didn't
> signal
> replacement. There is not consensus over the mempool, so in certain
> cases
> non-full-rbf nodes will in fact broadcast replacements when they didn't
> happen
> to receive the "first" transaction first.
>
> The latter makes testing full-rbf a bit problematic, as if you don't
> take
> special measures to ensure good propagation a small % of the time the
> "replacement" transaction will in fact be the one that gets gets mined.
>
> Does fullrbf offer any benefits other than breaking zeroconf
> business practices? If so, what are they?
> I think AJ mentioned this earlier, but adding more configuration
> options
> always increases code complexity, and with that, there is likely more
> unforeseen bugs. However, there is a section of network participants
> that
> rely on both types of transaction policy, so from my limited
> view-point, it
> seems worth accommodating if possible.
Since all the machinery to do replacemnt already exists, adding a
full-rbf
config flag is particularly trivial. It requires just a single line in
the
mempool code.
[-- Attachment #2: Type: text/html, Size: 5477 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-31 13:02 ` Suhas Daftuar
2022-10-31 16:25 ` Greg Sanders
2022-11-02 19:50 ` Antoine Riard
@ 2022-11-05 2:35 ` Peter Todd
2 siblings, 0 replies; 33+ messages in thread
From: Peter Todd @ 2022-11-05 2:35 UTC (permalink / raw)
To: Suhas Daftuar, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 3036 bytes --]
On Mon, Oct 31, 2022 at 09:02:02AM -0400, Suhas Daftuar via bitcoin-dev wrote:
Sending this email for the sake of repeating a point I made on GitHub for the
mailing list audience:
> AJ,
>
> Thanks for the thoughtful post. I think your observations about how we view
> mempool policy in the Bitcoin Core project, and how that seems to be
> changing in the discussions around `-mempoolfullrbf`, are on-point and
> provide a helpful baseline for considering future policy changes.
<snip>
> To those who argue for making fullrbf a default policy on the network (or
> even just offering a flag for users to enable fullrbf), I pose this
> hypothetical: suppose we deploy the v3 transaction policy proposal (which I
> hope will happen in the near future). That policy would restrict the ways
> that outputs of a v3 transaction can be spent while the transaction is
> unconfirmed, including by limiting the number and size of descendants that
> such a transaction can have, and limiting the types of unconfirmed
> ancestors that can be included. Suppose in a few years someone proposes
> that we add a "-disable_v3_transaction_enforcement" flag to our software,
> to let users decide to turn off those policy restrictions and treat v3
> transactions the same as v2, for all the same reasons that could be argued
> today with fullrbf: miners might earn more revenue if we allowed multiple
> descendant v3 transactions; it's illogical for the recipient of a v3
> transaction to believe what is a fundamentally unenforceable promise of a
> sender to not issue more high value children that descend from an
> unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
> policy on the network and we should honor user choice to turn off that flag
> if that’s what users want; if users are relying on v3’s policy restrictions
> for security then that is an unstable model and we should assume it will
> get broken[5].
Let's frame this question differently: given that there are potential
incentives around a hypothetical `-disable_v3_transaction_enforcement` flag,
why are we trying to prevent miners and others from experimenting with
incentives around `fullrbf`?
Yanking the `mempoolfullrbf` flag from Bitcoin Core v24.0 simply puts a
temporary roadblock in the face of full-rbf. Without that roadblock, we might
find that some miners do in fact choose to enable it. The sooner we find that
out, the sooner we can learn about the incentives involved in that decision.
Meanwhile, if we insted put up those roadblocks, we'll be designing mechanisms
like v3 blind, without the benefit of seeing how incentives play out fully.
This experimentation can't happen on testnet: incentives don't work properly
when there isn't money at stake. And the proposed reversion pull-reqs don't
even leave the option for testnet anyway. So we're left with one choice:
release a full-rbf option, and see what happens on mainnet.
--
https://petertodd.org 'peter'[:-1]@petertodd.org
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-08 9:28 ` AdamISZ
@ 2022-11-10 14:38 ` email
0 siblings, 0 replies; 33+ messages in thread
From: email @ 2022-11-10 14:38 UTC (permalink / raw)
To: AdamISZ, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 4479 bytes --]
> I read Antoine's original post on this and got the general gist, and
> here also, it makes sense, but I'd like to ask: is it necessary that
> (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs:
> A1, outputs: A, fees: low; call it TX_2)? I get that they cannot
> replace it
Is it actually true that they cannot replace it? If miners and node
operators collude and have the incentive to run a patched version of
core, is it still technically impossible to replace?
> the idea that they suffer financial loss from
> "ignorant" fee bumping is the part that seems weird to me.
Even if they waste resources trying to fee-bump, I agree that this does
not appear to be a catastrophe.There doesn't seem to be any technical
reason why improvements can't be made to allow B and C to have a better
view.
Cheers,
-Yancy
On 2022-11-08 10:28, AdamISZ via bitcoin-dev wrote:
> Hi aj and list,
> (questions inline)
>
> ------- Original Message -------
> On Thursday, October 27th, 2022 at 18:21, Anthony Towns via
> bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> Is that true? Antoine claims [1 [1]] that opt-in RBF isn't enough to
>> avoid
>> a DoS issue when utxos are jointly funded by untrusting partners, and,
>> aiui, that's the main motivation for addressing this now.
>>
>> [1]
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>>
>> The scenario he describes is: A, B, C create a tx:
>>
>> inputs: A1, B1, C1 [opts in to RBF]
>> fees: normal
>> outputs:
>> [lightning channel, DLC, etc, who knows]
>>
>> they all analyse the tx, and agree it looks great; however just before
>> publishing it, A spams the network with an alternative tx, double
>> spending her input:
>>
>> inputs: A1 [does not opt in to RBF]
>> fees: low
>> outputs: A
>>
>> If A gets the timing right, that's bad for B and C because they've
>> populated their mempool with the 1st transaction, while everyone else
>> sees the 2nd one instead; and neither tx will replace the other. B and
>> C can't know that they should just cancel their transaction, eg:
>>
>> inputs: B1, C1 [opts in to RBF]
>> fees: 50% above normal
>> outputs:
>> [smaller channel, refund, whatever]
>>
>> and might instead waste time trying to fee bump the tx to get it
>> mined,
>> or similar.
>>
>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
>> solve that problem if they have only opt-in RBF available?
> <snip>
> I read Antoine's original post on this and got the general gist, and
> here also, it makes sense, but I'd like to ask: is it necessary that
> (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs:
> A1, outputs: A, fees: low; call it TX_2)? I get that they cannot
> replace it, but the idea that they suffer financial loss from
> "ignorant" fee bumping is the part that seems weird to me. Clearly
> TX_2 gets gossiped to other mempools; and understood that it does not
> replace the TX_1 (the 3-input) in B's mempool, say, but why should
> they not even hear about it? Is it just a matter of engineering, or is
> there some deeper problem with that.
>
> About this general flavour of attack, it's never been a *big* concern
> in Joinmarket imo (though, we did until recently have a bug that made
> this happen *by accident*, i.e. people double spending an input out of
> a negotiated join, albeit it was rare; but it's ofc definitely
> *possible* to 'grief' like this, given the ordering of events; maker
> sends signature, maker broadcasts double spend - 95% of the time they
> will be first). Interactive protocols are yucky and I think there'll
> always be griefing possibilities; designing around multiple-rounds of
> negotiation amongst not always-on participants is even more yucky, so
> just having a 'taker is in charge of network fee; if it's slow or gets
> double spent out causing time delay then just wait', combined with
> 'there really isn't any economic incentive for an attacker' (i.e.
> ignoring griefing) might sound crappy but it's probably just being
> realistic.
>
> Of course, off-chain contracting has more sophisticated considerations
> than this.
>
> Cheers,
> AdamISZ/waxwing
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Links:
------
[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
[-- Attachment #2: Type: text/html, Size: 6661 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-08 14:54 ` email
@ 2022-11-09 12:05 ` email
0 siblings, 0 replies; 33+ messages in thread
From: email @ 2022-11-09 12:05 UTC (permalink / raw)
To: email, Bitcoin Protocol Discussion; +Cc: Anthony Towns
[-- Attachment #1: Type: text/plain, Size: 3197 bytes --]
> Bob has staked liquidity in a payment channel with Alice who later
> double spends the same inputs (at a very low feerate) resulting in a
> stalemate where neither can spend the UTXOs.
I just realized I made a mistake. RBF will always mine the higher fee
transaction, so in this case, full-rbf would prevent a transaction from
being pinned.
On 2022-11-08 15:54, yancy via bitcoin-dev wrote:
> Peter,
>
> It sounds like there are two attack vectors; neither of which require
> full-rbf (correct me if I'm wrong).
>
> 1) Bob has staked liquidity in a payment channel with Alice who later
> double spends the same inputs (at a very low feerate) resulting in a
> stalemate where neither can spend the UTXOs. The TX that creates the
> payment channel with Bob will never be mined since the mining pool
> sees the double spend?
>
> 2) Alice spams the network with a double spend wide enough that the
> double spend makes it into a block before the remainder of the network
> sees the first spend.
>
> In that case of 1), what if Bob required a opt-in rbf? Wouldn't that
> solve the issue? Bob could just create a replacement transaction with
> enough fee to get back his UTXO?
>
> For 2) it seems to me that neither full-rbf or opt-in rbf resolves
> this, although it's a probabilistic attack and requires spamming many
> nodes.
>
> Cheers,
> -Yancy
>
> On 2022-11-07 15:32, Peter Todd wrote:
>
>> On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev
>> <bitcoin-dev@lists•linuxfoundation.org> wrote:
>> AJ/Antoine et al
>>
>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
>> solve that problem if they have only opt-in RBF available?
>> Assuming Alice is a well funded advisory, with enough resources to
>> spam the network so that enough nodes see her malicious transaction
>> first, how does full-rbf solve this vs. opt-in rbf?
>
> First of all, to make things clear, remember that the attacks were
> talking about are aimed at _preventing_ a transaction from getting
> mined. Alice wants to cheaply broadcast something with low fees that
> won't get mined soon (if ever), that prevents a protocol from making
> forward progress.
>
> With full-rbf, who saw what transaction first doesn't matter: the
> higher fee paying transaction will always(*) replace the lower fee
> one. With opt-in RBF, spamming the network can beat out the
> alternative.
>
> *) So what's the catch? Well, due to limitations in today's mempool
> implementation, sometimes we can't fully evaluate which tx pays the
> higher fee. For example, if Alice spams the network with very _large_
> numbers transactions spending that input, the current mempool code
> doesn't even try to figure out if a replacement is better.
>
> But those limitations are likely to be fixable. And even right now,
> without fixing them, Alice still has to use a lot more money to pull
> off these attacks with full-rbf. So full-rbf definitely improves the
> situation even if it doesn't solve the problem completely.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[-- Attachment #2: Type: text/html, Size: 4542 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-07 14:32 ` Peter Todd
2022-11-07 14:47 ` Erik Aronesty
@ 2022-11-08 14:54 ` email
2022-11-09 12:05 ` email
1 sibling, 1 reply; 33+ messages in thread
From: email @ 2022-11-08 14:54 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Protocol Discussion, Anthony Towns
[-- Attachment #1: Type: text/plain, Size: 2518 bytes --]
Peter,
It sounds like there are two attack vectors; neither of which require
full-rbf (correct me if I'm wrong).
1) Bob has staked liquidity in a payment channel with Alice who later
double spends the same inputs (at a very low feerate) resulting in a
stalemate where neither can spend the UTXOs. The TX that creates the
payment channel with Bob will never be mined since the mining pool sees
the double spend?
2) Alice spams the network with a double spend wide enough that the
double spend makes it into a block before the remainder of the network
sees the first spend.
In that case of 1), what if Bob required a opt-in rbf? Wouldn't that
solve the issue? Bob could just create a replacement transaction with
enough fee to get back his UTXO?
For 2) it seems to me that neither full-rbf or opt-in rbf resolves this,
although it's a probabilistic attack and requires spamming many nodes.
Cheers,
-Yancy
On 2022-11-07 15:32, Peter Todd wrote:
> On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:
> AJ/Antoine et al
>
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
> Assuming Alice is a well funded advisory, with enough resources to spam
> the network so that enough nodes see her malicious transaction first,
> how does full-rbf solve this vs. opt-in rbf?
First of all, to make things clear, remember that the attacks were
talking about are aimed at _preventing_ a transaction from getting
mined. Alice wants to cheaply broadcast something with low fees that
won't get mined soon (if ever), that prevents a protocol from making
forward progress.
With full-rbf, who saw what transaction first doesn't matter: the
higher fee paying transaction will always(*) replace the lower fee
one. With opt-in RBF, spamming the network can beat out the
alternative.
*) So what's the catch? Well, due to limitations in today's mempool
implementation, sometimes we can't fully evaluate which tx pays the
higher fee. For example, if Alice spams the network with very _large_
numbers transactions spending that input, the current mempool code
doesn't even try to figure out if a replacement is better.
But those limitations are likely to be fixable. And even right now,
without fixing them, Alice still has to use a lot more money to pull
off these attacks with full-rbf. So full-rbf definitely improves the
situation even if it doesn't solve the problem completely.
[-- Attachment #2: Type: text/html, Size: 4493 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 17:21 ` Anthony Towns
2022-10-27 17:35 ` Suhas Daftuar
@ 2022-11-08 9:28 ` AdamISZ
2022-11-10 14:38 ` email
1 sibling, 1 reply; 33+ messages in thread
From: AdamISZ @ 2022-11-08 9:28 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
Hi aj and list,
(questions inline)
------- Original Message -------
On Thursday, October 27th, 2022 at 18:21, Anthony Towns via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
>
> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
> a DoS issue when utxos are jointly funded by untrusting partners, and,
> aiui, that's the main motivation for addressing this now.
>
> [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> The scenario he describes is: A, B, C create a tx:
>
> inputs: A1, B1, C1 [opts in to RBF]
> fees: normal
> outputs:
> [lightning channel, DLC, etc, who knows]
>
> they all analyse the tx, and agree it looks great; however just before
> publishing it, A spams the network with an alternative tx, double
> spending her input:
>
> inputs: A1 [does not opt in to RBF]
> fees: low
> outputs: A
>
> If A gets the timing right, that's bad for B and C because they've
> populated their mempool with the 1st transaction, while everyone else
> sees the 2nd one instead; and neither tx will replace the other. B and
> C can't know that they should just cancel their transaction, eg:
>
> inputs: B1, C1 [opts in to RBF]
> fees: 50% above normal
> outputs:
> [smaller channel, refund, whatever]
>
> and might instead waste time trying to fee bump the tx to get it mined,
> or similar.
>
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
>
<snip>
>
I read Antoine's original post on this and got the general gist, and here also, it makes sense, but I'd like to ask: is it necessary that (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs: A1, outputs: A, fees: low; call it TX_2)? I get that they cannot replace it, but the idea that they suffer financial loss from "ignorant" fee bumping is the part that seems weird to me. Clearly TX_2 gets gossiped to other mempools; and understood that it does not replace the TX_1 (the 3-input) in B's mempool, say, but why should they not even hear about it? Is it just a matter of engineering, or is there some deeper problem with that.
About this general flavour of attack, it's never been a *big* concern in Joinmarket imo (though, we did until recently have a bug that made this happen *by accident*, i.e. people double spending an input out of a negotiated join, albeit it was rare; but it's ofc definitely *possible* to 'grief' like this, given the ordering of events; maker sends signature, maker broadcasts double spend - 95% of the time they will be first). Interactive protocols are yucky and I think there'll always be griefing possibilities; designing around multiple-rounds of negotiation amongst not always-on participants is even more yucky, so just having a 'taker is in charge of network fee; if it's slow or gets double spent out causing time delay then just wait', combined with 'there really isn't any economic incentive for an attacker' (i.e. ignoring griefing) might sound crappy but it's probably just being realistic.
Of course, off-chain contracting has more sophisticated considerations than this.
Cheers,
AdamISZ/waxwing
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-07 14:32 ` Peter Todd
@ 2022-11-07 14:47 ` Erik Aronesty
2022-11-08 14:54 ` email
1 sibling, 0 replies; 33+ messages in thread
From: Erik Aronesty @ 2022-11-07 14:47 UTC (permalink / raw)
To: Peter Todd, Bitcoin Protocol Discussion; +Cc: Anthony Towns
[-- Attachment #1: Type: text/plain, Size: 513 bytes --]
>
>
> With full-rbf, who saw what transaction first doesn't matter: the higher
> fee paying transaction will always(*) replace the lower fee one. With
> opt-in RBF, spamming the network can beat out the alternative.
>
incentivised predictability is critical when designing low level protocols,
like bitcoin. the knock-on effects of deeper, network-wide predictability
are likely beneficial in ways that are hard to predict. for example stuff
like the "sabu" protocol might even work if full-rbf is the norm.
[-- Attachment #2: Type: text/html, Size: 765 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-11-03 21:06 email
@ 2022-11-07 14:32 ` Peter Todd
2022-11-07 14:47 ` Erik Aronesty
2022-11-08 14:54 ` email
0 siblings, 2 replies; 33+ messages in thread
From: Peter Todd @ 2022-11-07 14:32 UTC (permalink / raw)
To: email, Bitcoin Protocol Discussion, yancy via bitcoin-dev, Anthony Towns
On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>AJ/Antoine et al
>
>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
>> solve that problem if they have only opt-in RBF available?
>
>Assuming Alice is a well funded advisory, with enough resources to spam the network so that enough nodes see her malicious transaction first, how does full-rbf solve this vs. opt-in rbf?
First of all, to make things clear, remember that the attacks were talking about are aimed at _preventing_ a transaction from getting mined. Alice wants to cheaply broadcast something with low fees that won't get mined soon (if ever), that prevents a protocol from making forward progress.
With full-rbf, who saw what transaction first doesn't matter: the higher fee paying transaction will always(*) replace the lower fee one. With opt-in RBF, spamming the network can beat out the alternative.
*) So what's the catch? Well, due to limitations in today's mempool implementation, sometimes we can't fully evaluate which tx pays the higher fee. For example, if Alice spams the network with very _large_ numbers transactions spending that input, the current mempool code doesn't even try to figure out if a replacement is better.
But those limitations are likely to be fixable. And even right now, without fixing them, Alice still has to use a lot more money to pull off these attacks with full-rbf. So full-rbf definitely improves the situation even if it doesn't solve the problem completely.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
@ 2022-11-03 21:06 email
2022-11-07 14:32 ` Peter Todd
0 siblings, 1 reply; 33+ messages in thread
From: email @ 2022-11-03 21:06 UTC (permalink / raw)
To: Anthony Towns, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 6033 bytes --]
AJ/Antoine et al
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
Assuming Alice is a well funded advisory, with enough resources to spam
the network so that enough nodes see her malicious transaction first,
how does full-rbf solve this vs. opt-in rbf?
Cheers,
-Yancy
On 2022-10-27 19:21, Anthony Towns via bitcoin-dev wrote:
> On Thu, Oct 27, 2022 at 11:56:45AM +0200, John Carvalho via bitcoin-dev
> wrote:
>
>> I took the time to read your whole post. Despite a diplomatic tone, I
>> find
>> your takeaways from all your references to remain conveniently biased
>> for
>> protecting the plan of RBF
>
> Yes, I am heavily biased against zeroconf: there's no way I'd
> personally
> be willing to trust it for my own incoming funds, no matter how much
> evidence you show me that it's safe in practice. Show me a million
> transactions where every single one worked fine, and I'm still going to
> assume that the payment going to me is going to be the one that makes
> the error rate tick up from 0% to 0.0001%. That's okay; just because I
> wouldn't do something, doesn't mean other people shouldn't.
>
> It does mean I'm not going to be a particularly good advocate for
> zeroconf
> though. I mean, I might still be a fine advocate for giving people time
> to react, making it clear what's going on, finding ways that might make
> everyone happy, or just digging it to random technical details; but,
> for me, I'm more interested in a world where chargebacks are
> impossible,
> not where we just make the best of what was possible with technology
> from five or ten years ago.
>
> But that's fine: it just means that people, like yourself, who will
> tolerate the risks of zeroconf, should be involved in the discussion.
>
>> You show multiple examples where, when I read them, I assume the next
>> thing
>> you will say will be "so we really should stop trying to impose
>> optional
>> features, particularly when they affect existing use cases" but
>> instead you
>> persist.
>
> Sure, that's natural: you read a sign saying "you can have any ice
> cream
> you want for 5c" and think "Awesome, who wouldn't want cheap chocolate
> ice cream!!" and see me going for a Golden Gaytime and think "wtf
> dude".
> Different strokes.
>
> For me, I see the gmaxwell github comment I quoted saying:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works.
>
> and think "yeah, okay, maybe we should be working harder to push
> lightning
> adoption, rather than letting people stick with wallet UX from 2015"
> and have altcoins take over >50% of payment volume.
>
> Likewise,
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it.
>
> seems a pretty good match against your claim "I expect the things I do
> with Bitcoin today to work FOREVER." Better to nip that thinking in the
> bud; and even if the best time to do that was years ago, the second
> best
> time to do it is still now.
>
> By contrast, from the same post, I'd guess you're focussing on:
>
> Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public".
>
> and thinking "yeah, move fast, break things, force everyone else --
> that's exactly what's going on here, and shouldn't be".
>
> But that's also okay: even when there is common ground to be found,
> sometimes it requires actual work to get people who start from
> different
> views to get there.
>
>> The problem is that RBF has already been an option for years, and
>> anyone
>> that wants to use it can.
>
> Is that true? Antoine claims [1 [1]] that opt-in RBF isn't enough to
> avoid
> a DoS issue when utxos are jointly funded by untrusting partners, and,
> aiui, that's the main motivation for addressing this now.
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> The scenario he describes is: A, B, C create a tx:
>
> inputs: A1, B1, C1 [opts in to RBF]
> fees: normal
> outputs:
> [lightning channel, DLC, etc, who knows]
>
> they all analyse the tx, and agree it looks great; however just before
> publishing it, A spams the network with an alternative tx, double
> spending her input:
>
> inputs: A1 [does not opt in to RBF]
> fees: low
> outputs: A
>
> If A gets the timing right, that's bad for B and C because they've
> populated their mempool with the 1st transaction, while everyone else
> sees the 2nd one instead; and neither tx will replace the other. B and
> C can't know that they should just cancel their transaction, eg:
>
> inputs: B1, C1 [opts in to RBF]
> fees: 50% above normal
> outputs:
> [smaller channel, refund, whatever]
>
> and might instead waste time trying to fee bump the tx to get it mined,
> or similar.
>
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
>
> If you're right that opt-in RBF is enough, that question has a good
> answer. I don't believe anyone's presented an answer to it in the 17
> months since Antoine raised the concern.
>
>> passive aggression
>> escalation
>> unfair advantage
>> oppressive, dark-pattern design
>> strong-arming and shoe-horning
>
> Do you really think any of that was helping your cause?
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Links:
------
[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
[-- Attachment #2: Type: text/html, Size: 8609 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 17:44 ` Greg Sanders
@ 2022-10-27 19:00 ` Greg Sanders
0 siblings, 0 replies; 33+ messages in thread
From: Greg Sanders @ 2022-10-27 19:00 UTC (permalink / raw)
To: Suhas Daftuar, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 4433 bytes --]
During off-channel discussion, Suhas made a great point that even with
fullrbf, you can get stuck by bip125 rule#5 pinning if an adversary
controls a number of inputs(4 with default mempool settings).
Implication being, while we can mitigate rule#3 damage potentially with
fullrbf, we cannot actually make promises about mempool entry beyond quite
small transaction sizes. Adversary has to make 100 transactions, 4 chains
of 25, but it achieves the original pin.
On Thu, Oct 27, 2022 at 1:44 PM Greg Sanders <gsanders87@gmail•com> wrote:
> > For instance, the double-spend could be low-feerate and large, and
> effectively pin any attempt to replace it.
>
> Yes, this is the biggest hole left. You *could* replace it with RBF when
> before you simply could not, so perhaps the pinning door is slightly
> smaller in scenarios where going feerates are significantly higher than min.
>
> > Or it could be higher feerate and confirm and B/C have to start all over.
>
> Coinjoins have "blame rounds" exactly for this. Ruling out the above hole
> where you don't want to pay the 100kvb rule#3 penalty, you can kick the
> griefer out. Without replacement, you likely can not.
>
> > Or, A could stall things in the signing phase and B/C have to figure out
> when to give up on the channel with A.
>
> Again, blame rounds solve this.
>
> So to recap, it makes it *possible* to over-bid your griefer, vs simply
> not able to and have funds tied up for weeks(or guess you're being pinned
> and double-spend your input, which again looks blame-worthy).
>
> Properly replacing rule#3 would give these protocols higher assurances,
> but this is where we're at now.
>
> On Thu, Oct 27, 2022 at 1:35 PM Suhas Daftuar via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> I have more to say on this broader topic, but since you've brought up
>> this particular example I think it's worth commenting:
>>
>> On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev <
>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>
>>> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
>>> a DoS issue when utxos are jointly funded by untrusting partners, and,
>>> aiui, that's the main motivation for addressing this now.
>>>
>>> [1]
>>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>>>
>>> The scenario he describes is: A, B, C create a tx:
>>>
>>> inputs: A1, B1, C1 [opts in to RBF]
>>> fees: normal
>>> outputs:
>>> [lightning channel, DLC, etc, who knows]
>>>
>>> they all analyse the tx, and agree it looks great; however just before
>>> publishing it, A spams the network with an alternative tx, double
>>> spending her input:
>>>
>>> inputs: A1 [does not opt in to RBF]
>>> fees: low
>>> outputs: A
>>>
>>> If A gets the timing right, that's bad for B and C because they've
>>> populated their mempool with the 1st transaction, while everyone else
>>> sees the 2nd one instead; and neither tx will replace the other. B and
>>> C can't know that they should just cancel their transaction, eg:
>>>
>>> inputs: B1, C1 [opts in to RBF]
>>> fees: 50% above normal
>>> outputs:
>>> [smaller channel, refund, whatever]
>>>
>>> and might instead waste time trying to fee bump the tx to get it mined,
>>> or similar.
>>>
>>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
>>> solve that problem if they have only opt-in RBF available?
>>>
>>
>> I think this is not a real example of a DoS vector that is available
>> because we support non-rbf signaling transactions. Even in a world where
>> all transactions are replaceable, person A could double-spend their input
>> in a way that is annoying for B and C. For instance, the double-spend
>> could be low-feerate and large, and effectively pin any attempt to replace
>> it. Or it could be higher feerate and confirm and B/C have to start all
>> over. Or, A could stall things in the signing phase and B/C have to figure
>> out when to give up on the channel with A.
>>
>> So I find this example to be unconvincing. Are there any other examples
>> where having a non-replacement policy for some transactions causes problems
>> for protocols people are trying to build?
>>
>> Thanks,
>> Suhas
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
[-- Attachment #2: Type: text/html, Size: 5958 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 17:35 ` Suhas Daftuar
@ 2022-10-27 17:44 ` Greg Sanders
2022-10-27 19:00 ` Greg Sanders
0 siblings, 1 reply; 33+ messages in thread
From: Greg Sanders @ 2022-10-27 17:44 UTC (permalink / raw)
To: Suhas Daftuar, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 3778 bytes --]
> For instance, the double-spend could be low-feerate and large, and
effectively pin any attempt to replace it.
Yes, this is the biggest hole left. You *could* replace it with RBF when
before you simply could not, so perhaps the pinning door is slightly
smaller in scenarios where going feerates are significantly higher than min.
> Or it could be higher feerate and confirm and B/C have to start all over.
Coinjoins have "blame rounds" exactly for this. Ruling out the above hole
where you don't want to pay the 100kvb rule#3 penalty, you can kick the
griefer out. Without replacement, you likely can not.
> Or, A could stall things in the signing phase and B/C have to figure out
when to give up on the channel with A.
Again, blame rounds solve this.
So to recap, it makes it *possible* to over-bid your griefer, vs simply not
able to and have funds tied up for weeks(or guess you're being pinned and
double-spend your input, which again looks blame-worthy).
Properly replacing rule#3 would give these protocols higher assurances, but
this is where we're at now.
On Thu, Oct 27, 2022 at 1:35 PM Suhas Daftuar via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> I have more to say on this broader topic, but since you've brought up this
> particular example I think it's worth commenting:
>
> On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
>> a DoS issue when utxos are jointly funded by untrusting partners, and,
>> aiui, that's the main motivation for addressing this now.
>>
>> [1]
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>>
>> The scenario he describes is: A, B, C create a tx:
>>
>> inputs: A1, B1, C1 [opts in to RBF]
>> fees: normal
>> outputs:
>> [lightning channel, DLC, etc, who knows]
>>
>> they all analyse the tx, and agree it looks great; however just before
>> publishing it, A spams the network with an alternative tx, double
>> spending her input:
>>
>> inputs: A1 [does not opt in to RBF]
>> fees: low
>> outputs: A
>>
>> If A gets the timing right, that's bad for B and C because they've
>> populated their mempool with the 1st transaction, while everyone else
>> sees the 2nd one instead; and neither tx will replace the other. B and
>> C can't know that they should just cancel their transaction, eg:
>>
>> inputs: B1, C1 [opts in to RBF]
>> fees: 50% above normal
>> outputs:
>> [smaller channel, refund, whatever]
>>
>> and might instead waste time trying to fee bump the tx to get it mined,
>> or similar.
>>
>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
>> solve that problem if they have only opt-in RBF available?
>>
>
> I think this is not a real example of a DoS vector that is available
> because we support non-rbf signaling transactions. Even in a world where
> all transactions are replaceable, person A could double-spend their input
> in a way that is annoying for B and C. For instance, the double-spend
> could be low-feerate and large, and effectively pin any attempt to replace
> it. Or it could be higher feerate and confirm and B/C have to start all
> over. Or, A could stall things in the signing phase and B/C have to figure
> out when to give up on the channel with A.
>
> So I find this example to be unconvincing. Are there any other examples
> where having a non-replacement policy for some transactions causes problems
> for protocols people are trying to build?
>
> Thanks,
> Suhas
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 5094 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 17:21 ` Anthony Towns
@ 2022-10-27 17:35 ` Suhas Daftuar
2022-10-27 17:44 ` Greg Sanders
2022-11-08 9:28 ` AdamISZ
1 sibling, 1 reply; 33+ messages in thread
From: Suhas Daftuar @ 2022-10-27 17:35 UTC (permalink / raw)
To: Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 2333 bytes --]
I have more to say on this broader topic, but since you've brought up this
particular example I think it's worth commenting:
On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
> a DoS issue when utxos are jointly funded by untrusting partners, and,
> aiui, that's the main motivation for addressing this now.
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> The scenario he describes is: A, B, C create a tx:
>
> inputs: A1, B1, C1 [opts in to RBF]
> fees: normal
> outputs:
> [lightning channel, DLC, etc, who knows]
>
> they all analyse the tx, and agree it looks great; however just before
> publishing it, A spams the network with an alternative tx, double
> spending her input:
>
> inputs: A1 [does not opt in to RBF]
> fees: low
> outputs: A
>
> If A gets the timing right, that's bad for B and C because they've
> populated their mempool with the 1st transaction, while everyone else
> sees the 2nd one instead; and neither tx will replace the other. B and
> C can't know that they should just cancel their transaction, eg:
>
> inputs: B1, C1 [opts in to RBF]
> fees: 50% above normal
> outputs:
> [smaller channel, refund, whatever]
>
> and might instead waste time trying to fee bump the tx to get it mined,
> or similar.
>
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
>
I think this is not a real example of a DoS vector that is available
because we support non-rbf signaling transactions. Even in a world where
all transactions are replaceable, person A could double-spend their input
in a way that is annoying for B and C. For instance, the double-spend
could be low-feerate and large, and effectively pin any attempt to replace
it. Or it could be higher feerate and confirm and B/C have to start all
over. Or, A could stall things in the signing phase and B/C have to figure
out when to give up on the channel with A.
So I find this example to be unconvincing. Are there any other examples
where having a non-replacement policy for some transactions causes problems
for protocols people are trying to build?
Thanks,
Suhas
[-- Attachment #2: Type: text/html, Size: 3016 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
2022-10-27 9:56 ` John Carvalho
@ 2022-10-27 17:21 ` Anthony Towns
2022-10-27 17:35 ` Suhas Daftuar
2022-11-08 9:28 ` AdamISZ
0 siblings, 2 replies; 33+ messages in thread
From: Anthony Towns @ 2022-10-27 17:21 UTC (permalink / raw)
To: John Carvalho, Bitcoin Protocol Discussion
On Thu, Oct 27, 2022 at 11:56:45AM +0200, John Carvalho via bitcoin-dev wrote:
> I took the time to read your whole post. Despite a diplomatic tone, I find
> your takeaways from all your references to remain conveniently biased for
> protecting the plan of RBF
Yes, I am heavily biased against zeroconf: there's no way I'd personally
be willing to trust it for my own incoming funds, no matter how much
evidence you show me that it's safe in practice. Show me a million
transactions where every single one worked fine, and I'm still going to
assume that the payment going to me is going to be the one that makes
the error rate tick up from 0% to 0.0001%. That's okay; just because I
wouldn't do something, doesn't mean other people shouldn't.
It does mean I'm not going to be a particularly good advocate for zeroconf
though. I mean, I might still be a fine advocate for giving people time
to react, making it clear what's going on, finding ways that might make
everyone happy, or just digging it to random technical details; but,
for me, I'm more interested in a world where chargebacks are impossible,
not where we just make the best of what was possible with technology
from five or ten years ago.
But that's fine: it just means that people, like yourself, who will
tolerate the risks of zeroconf, should be involved in the discussion.
> You show multiple examples where, when I read them, I assume the next thing
> you will say will be "so we really should stop trying to impose optional
> features, particularly when they affect existing use cases" but instead you
> persist.
Sure, that's natural: you read a sign saying "you can have any ice cream
you want for 5c" and think "Awesome, who wouldn't want cheap chocolate
ice cream!!" and see me going for a Golden Gaytime and think "wtf dude".
Different strokes.
For me, I see the gmaxwell github comment I quoted saying:
There is also a matter of driving competent design rather than lazy
first thing that works.
and think "yeah, okay, maybe we should be working harder to push lightning
adoption, rather than letting people stick with wallet UX from 2015"
and have altcoins take over >50% of payment volume.
Likewise,
There is also a very clear pattern we've seen in the past where
people take anything the system lets them do as strong evidence that
they have a irrevocable right to use the system in that way, and that
their only responsibility-- and if their usage harms the system it's
the responsibility of the system to not permit it.
seems a pretty good match against your claim "I expect the things I do
with Bitcoin today to work FOREVER." Better to nip that thinking in the
bud; and even if the best time to do that was years ago, the second best
time to do it is still now.
By contrast, from the same post, I'd guess you're focussing on:
Network behavior is one of the few bits of friction
driving good technical design rather than "move fast, break things, and
force everyone else onto my way of doing thing rather than discussing
the design in public".
and thinking "yeah, move fast, break things, force everyone else --
that's exactly what's going on here, and shouldn't be".
But that's also okay: even when there is common ground to be found,
sometimes it requires actual work to get people who start from different
views to get there.
> The problem is that RBF has already been an option for years, and anyone
> that wants to use it can.
Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
a DoS issue when utxos are jointly funded by untrusting partners, and,
aiui, that's the main motivation for addressing this now.
[1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
The scenario he describes is: A, B, C create a tx:
inputs: A1, B1, C1 [opts in to RBF]
fees: normal
outputs:
[lightning channel, DLC, etc, who knows]
they all analyse the tx, and agree it looks great; however just before
publishing it, A spams the network with an alternative tx, double
spending her input:
inputs: A1 [does not opt in to RBF]
fees: low
outputs: A
If A gets the timing right, that's bad for B and C because they've
populated their mempool with the 1st transaction, while everyone else
sees the 2nd one instead; and neither tx will replace the other. B and
C can't know that they should just cancel their transaction, eg:
inputs: B1, C1 [opts in to RBF]
fees: 50% above normal
outputs:
[smaller channel, refund, whatever]
and might instead waste time trying to fee bump the tx to get it mined,
or similar.
What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
solve that problem if they have only opt-in RBF available?
If you're right that opt-in RBF is enough, that question has a good
answer. I don't believe anyone's presented an answer to it in the 17
months since Antoine raised the concern.
> passive aggression
> escalation
> unfair advantage
> oppressive, dark-pattern design
> strong-arming and shoe-horning
Do you really think any of that was helping your cause?
Cheers,
aj
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [bitcoin-dev] On mempool policy consistency
[not found] <mailman.38435.1666828344.956.bitcoin-dev@lists.linuxfoundation.org>
@ 2022-10-27 9:56 ` John Carvalho
2022-10-27 17:21 ` Anthony Towns
0 siblings, 1 reply; 33+ messages in thread
From: John Carvalho @ 2022-10-27 9:56 UTC (permalink / raw)
To: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 25991 bytes --]
Anthony,
I took the time to read your whole post. Despite a diplomatic tone, I find
your takeaways from all your references to remain conveniently biased for
protecting the plan of RBF via passive aggression.
You show multiple examples where, when I read them, I assume the next thing
you will say will be "so we really should stop trying to impose optional
features, particularly when they affect existing use cases" but instead you
persist.
The problem is that RBF has already been an option for years, and anyone
that wants to use it can. Any escalation in Bitcoin Core code to support it
more deeply, or by default, is basically an unfair advantage to force the
market to do what it already has decided not to.
If wallets want to default to RBF, they can already do so, as evidenced by
Green Wallet (which I stopped using because it breaks the UX at Bitrefill).
Instead of Core devs admitting RBF is a minority use case, you seem to be
proposing that the market should now be obligated to prove it can defeat
RBF in a stronger form if it really wants to prove other use cases. This is
oppressive, dark-pattern design. We all know that Core has little ability
to sense the market, and the market has little ability to express itself to
Core. The idea that the market can always downvote or defeat a feature or
new complexity proposal is idealistic and unrealistic.
Superficial features should be decided at the surface (app level) not in
the protocol or node.
The default answer to ALL proposals is "No." Changes need to win market
acceptance, not get special access through Core devs baking them deeper and
deeper into the protocol and policies until everyone is forced into a new
design.
As I mentioned before, this behavior, if we are lucky, will result in more
mempool types, more implementations, and a more-difficult to modify
protocol, but ALL feature changes, default settings that make decisions for
users, and even all scaling changes, are speculative risks with
unpredictable outcomes.
I urge the culture of Core to respect these dynamics and become much more
conservative with proposing change. Please focus on efficiencies, bugs,
cleanup, reducing overhead, etc.
The current RBF movement feels like Core is strong-arming and shoe-horning
in a change that the market is not actually asking for. It is okay to leave
things as they are. It is okay if RBF remains a niche feature. It is not
okay for a small group of RBF-interested engineers to make commercial
Bitcoin use cases worse.
Let us realize the Bitcoin we already have. We already have a largely
unexplored canvas of taproot, lightning, UX, etc.
I expect the things I do with Bitcoin today to work FOREVER.
--
John Carvalho
CEO, Synonym.to <http://synonym.to/>
> Date: Thu, 27 Oct 2022 09:52:10 +1000
> From: Anthony Towns <aj@erisian•com.au>
> To: Bitcoin Protocol Discussion
> <bitcoin-dev@lists•linuxfoundation.org>
> Subject: [bitcoin-dev] On mempool policy consistency
> Message-ID: <Y1nIKjQC3DkiSGyw@erisian•com.au>
> Content-Type: text/plain; charset=us-ascii
>
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
>
> And other plausible options aren't configurable even at compile time
> -- eg, core doesn't implement BIP 125's inherited signalling rule so
> there's no way to enable it; core doesn't allow opting out of BIP 125
> rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
> more than 1 ancestor; core doesn't allow opting out of LOW_S checks
> (even via -acceptnonstdtxn); etc.
>
> We also naturally have different mempool policies between different
> releases: eg, expansions of policy, such as allowing OP_RETURN or
> expanding it from 40 to 80 bytes or new soft forks where old nodes won't
> relay transactions that use the new; and also occassional restrictions
> in policy, such as the LOW_S requirement.
>
>
> While supporting and encouraging different mempool polices might be new
> for core, it's not new for knots: knots changes some of these defaults
> (-permitbaremultisig defaults to false, -datacarriersize is reduced to
> 42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
> options including -spkreuse and -mempoolreplacement (giving the latter
> full rbf behaviour by default). Knots also includes a `-corepolicy`
> option to make it easy to get a configuration matching core's defaults.
>
>
> I think gmaxwell's take from Feb 2015 (in the context of how restrictive
> policy on OP_RETURN data should be) was a reasonable description for
> core's approach up until now:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works. E.g. In stealth addresses the early proposals
> use highly inefficient single ECDH point per output instead of simply
> pooling them. Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public". No one wants to be an outright gatekeeper,
> but the network is a shared resource and it's perfectly reasonable
> node behavior to be stingy about the perpetual storage impact of the
> transactions they're willing to process, especially when it comes to
> neutral technical criteria like the amount of network irrelevant data
> stuffed in transactions.
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it. [...
> ...] For mitigating these risks it's optimal if transactions
> seem as uniform and indistinguishable as reasonably possible.
>
> - https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
>
> Perhaps see also sdaftuar in Nov 2015,
>
> To me the most important question is, is priority something that miners
> want to use?
>
> If a non-negligible amount of hashpower intends to use it in their
> transaction selection, then I think it makes sense for nodes to use it
> too, because it's generally helpful to have your mempool predict the
> UTXO as much as possible, and for nodes to be able to have reasonable
> fee and priority estimates (which won't happen unless they track the
> priority transactions somehow -- I'm presuming that miners run with
> much bigger mempools than regular nodes).
>
> If the answer is no, then that's fine and I don't see a reason to push
> in this direction. I sort of assumed there was enough hashpower mining
> with priority, since last time I checked estimatepriority was still
> giving meaningful results for low-ish blockheights, but I haven't done
> any kind of real analysis.
>
> - https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
>
> or in June 2019,
>
> What this PR is proposing is to get rid of a command-line option that is
> (a) a footgun for users and (b) does not reflect what I believe to be
> the understanding most users have, which is that [X txs] are expected
> to propagate well on the network.
>
> ..
>
> I don't think this rises to the level that Luke is concerned about,
> namely a prelude to forcing a common relay policy on all nodes. In
> particular I do agree it makes sense that we offer some ways of
> customizing policy parameters (eg the mempool size, min relay fee,
> etc). Instead, I think the justification for this change is that we
> should not support behaviors we think are harmful to the ecosystem
> overall and have no legitimate use-case, and we should eliminate ways
> that users might inadvertently shoot themselves in the foot.
>
> - https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
>
> (or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
>
> I don't mean to imply the above are saying "there's one way to do
> things and it's this way", or that the old way of doing things should
> necessarily be the way we keep doing things. Just that previously core
> has tended towards designing a single policy that works as well as it
> can for everyone and the ecosystem as a whole. (I'm also not saying that
> fullrbf can't work well for everyone or the ecosystem as a whole)
>
>
> By contrast, I think the most common response to pushback against the
> full rbf option has been along the lines of "it's just an option, we
> don't want to force people", eg:
>
> Blaming the default false -mempoolfullrbf option for a full RBF network
> would be holding Bitcoin Core developers responsible for the decisions
> of individual node operators and miners. I don't think having the
> option (again, default false) can directly cause a full RBF network,
> and likewise, I don't think removing this option removes the "risk"
> of a full RBF network.
> - glozow
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
>
> NACK. This is a default false option.
> - achow101
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
>
> Erecting artificial barriers to prevent or make it difficult for users
> to do what they want to do, is not appropriate behaviour.
> - luke-jr
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
>
> I'm in general against removing options.
> - instagibbs
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
>
> I think this differs from what core has done in the past, in that
> previously we've tried to ensure a new policy is good for everyone (or as
> nearly as it can be), and then enabled it as soon as it's implemented.
> Any options that have been added have either been to control resource
> usage in ways that don't significantly effect tx propagation, to
> allow people to revert to the old behaviour when the new behaviour is
> controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
> and to make it easier to test/debug the implementation.
>
> Giving people a new relay behaviour they can opt-in to when we aren't
> confident enough to turn on by default doesn't match the approach I've
> seen core take in the past.
>
>
> If this is going to be an ongoing shift in how core sees relay/mempool
> policy, I think that's significant and worth paying attention to.
>
> I don't think it's necessary to have that shift to roll out full rbf.
> The other approach would be either:
>
> * set -mempoolfullrbf=true as the default for 24.0, and just have the
> command line param there in case people want to do a
> "UserRejectedMempoolPolicy" campaign to get everyone to opt-out
>
> * revert it for now because we don't think mainnet is ready for fullrbf
> yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
> to activate at some scheduled date in that timeframe (potentially
> backporting it to previous releases to help with adoption too,
> whatever). same effect as the previous option, just with a bit more
> advanced notice and time to prepare
>
> I don't think anyone's proposed the first (which I interpret as "most of
> us don't think mainnet is ready for fullrbf today"), but the comments
> above are all pushback by people arguing against (the first step of)
> the second approach, and they seem to be winning the day.
>
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case and
> not a precedent, and when people propose other default false options,
> there will be substantially more resistance to them being merged,
> despite all the talk about users having options that's going on right now.
>
>
> Assuming it is the change of direction it appears to be -- and all of
> the above is really just justification for that assumption -- then like
> I said, I think it's worth seriously considering what it means for people
> to choose their own relay/mempool policies and for you to expect to have
> different mempool policies to many or most of your potential peers.
>
>
> One thing maybe worth noting is that is that you can still only choose
> your policy from options that people write code for -- if it wasn't
> something you could get by running knots or compiling a rejected PR
> yourself, it won't magically become more possible now. Presumably it
> would mean that once a PR is written, it might get better review (rather
> than being dismissed as not suitable for everyone), and there would be
> less maintenance burden than if it had to be manually rebased every
> release, though (or at least the maintenance burden would be shared
> across everyone working on the codebase).
>
>
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
>
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance. If erlay allows increasing your outbound count to
> 12 connections instead of 8; that might reduce down to needing just 20%
> of listening nodes (~93%).
>
> But for cases where less than 30% (20%) of network supports your preferred
> policy, you probably need to do something cleverer.
>
> One approach is to set a service bit and preferentially peer with other
> nodes that advertise that service bit; knots does the first half of this
> for fullrbf, and both halves have been proposed for core in #25600.
> Preferential peering was previously done for the segwit deployment,
> though in that case it was necessary not just for tx propogation but
> also for ensuring block propogation, making it effectively a consensus
> critical issue.
>
> Another approach is having a separate relay network -- eg, lightning nodes
> already have a gossip network, and might want to help their own ecosystem
> by ensuring unilateral channel closes and justice transactions are quickly
> relayed. Using their own gossip network to relay the transaction around,
> and each lightning node adding it to their local bitcoind's mempool and
> allowing it to propogate (or not) from there as normal, would also be a
> way of allowing transactions to propogate well. It does mean that miners
> would either need to also participate in lightning gossip directly, or
> that miners would need to connect to *many* peers to be confident of
> seeing those transactions (eg, if only 2% of the network would see a
> tx, you'd need to make 228 connections to have a 99% chance of seeing
> the tx). You can't currently do something like this, because all the
> relay policies are also applied when adding txs to the mempool via RPC,
> and there's no convenient way to remove txs from the mempool.
>
> A case where something like that might occur is in preventing L2
> transactions from pinning attacks -- so you might have a high-fee,
> low-feerate transaction that's been widely propogated, sitting in the
> bottom of people's mempools, and you want to replace it with a smaller,
> higher-feerate transaction, but don't want to pay a higher absolute fee,
> and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
> unwilling to deviate from BIP 125 rule 3 for you; because that would
> make it easy for random griefers to spam their mempool with large txs
> then delete them while only paying a small fee; but your L2 peers may be
> able to decode your replacement transaction and be sure that you aren't
> going to spam them, and thus will happily relay it.
>
> >From a technical point-of-view, that's largely fine; the downside is it
> increases the centralisation pressure on mining: whether that's by having
> to connect to substantially more nodes, or having to parse through more
> spam, you can't just run your mining operation off a standard install
> of bitcoin core anymore, but need to actively opt-in to find all the
> weird unusual ways people are sending transactions around in order to
> actually collect as much in fees as your competitors are.
>
> That's probably moderately bad for privacy as well -- if lightning or
> coinjoins need special relay rules that most nodes haven't opted into,
> it's potentially easy to use that to find the bitcoin nodes on the
> network that are participating in those protocols, and from there to
> either identify the operator, or run a DoS attack to make it hard for you
> to keep doing what you want. Obviously if you're setting a service bit to
> get better routing, you've given up that privacy already. Likewise if the
> government or random vandals are opposed to bitcoin mining, and miners
> have to have special configuration on their nodes that distinguish them
> from regular users, then perhaps that makes it easier to find or shut
> down their operations.
>
> There are a few efficiencies to be gained from similar mempool policies as
> well: more reliable compact block reconstruction (if you're not missing
> any transactions, you avoid a round-trip) and presumably more efficient
> set reconstruction with erlay. You'll also waste less bandwidth sending
> transactions that the other node is only going to reject. Both those
> depend on how many transactions are going to rely on unusual mempool
> policies in the first place though.
>
> ariard wrote:
>
> I know I've advocated in the past to turn RBF support by default in
> the past. Though after gathering a lot of feedbacks, this approach
> of offering the policy flexiblity to the interested users only and
> favoring a full-rbf gradual deployment sounds better to me.
>
> - https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
>
> I guess all the above leads me to think that gradual deployments of
> mempool policies are likely the worse approach: even when they're not
> hurting anyone, it makes them hard to use during the gradual phase,
> and getting around that comes with worrying compromises on privacy and
> centralisation; and when they are problematic for some, the indeterminate
> nature of a gradual deployment means it's hard to plan for when that
> risk is going to eventuate.
>
>
> Theoretically, one way to recover the good parts of core deciding on
> what's good for the network might be for people outside of core to
> recommend a mempool configuration; then core can just have an option
> to make that easy, similar to "-std=c++17" for a C++ compiler, and much
> the same as knots' "-corepolicy" option.
>
> Presuming anyone actually wants to take on that job, and listen to the
> concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
> and can come up with something that keeps most of them happy, and that
> 70% or 90% of the network ends up just following those recommendations
> because it's easy, it works, and it's recommended by all the apps they
> want to use, then that could work great:
>
> * miners don't need to do anything special, so there's no new
> mining centralisation pressure
> * miners and users don't reveal what they're doing with bitcoin by the way
> they configure their nodes, so there's no privacy problems
> * devs can be fairly confident in how they have to design their apps
> in order to get their transactions to most hashpower
> * devs don't have to add new p2p layers to make it happen
> * at least there's someone to talk to when you're trying to figure out
> how to make some new project possible when it's inhibited by current
> relay policies and you don't have to try to convince everyone to
> upgrade on your own
> * core devs just provide options, and don't have to worry about being
> seen as gatekeepers
>
> The "downside" in that scenario is that users/dev aren't making much
> actual use of all the choices core is offering by making different
> options available; but the upside is that that choice is at least readily
> available should whoever is coming up with these policy become out of
> step with what people actually want.
>
> One thing that might make an approach like that difficult is that core
> has historically been happy to remove options that don't seem useful
> anymore: eg the ability to turn of BIP 125 support (#16171), and priority
> transactions (#9602). Perhaps that's fine if you're trying to actively
> craft a single mempool/relay policy that's good enough for almost everyone
> (after all, it makes the code simpler and more efficient, and reduces
> the number of footguns); all you're doing is leaving a minority of people
> who want weird things to run a fork, and that's going to happen anyway.
>
> But if people are following policy developed outside of core, core
> might well disagree with them and decide "no that's a stupid policy,
> no one should do that" and remove some feature that others thing should
> continue to be normal. Beyond the examples above, there's already talk of
> removing the ability to disable fullrbf support in #26305, for instance.
> If that happens, then the people maintaining the policy will instead
> end up maintaining an entire fork of bitcoin core, and all we've done
> is transition to people running software from a different repo, and a
> different set of maintainers.
>
> If we're really going to a world where core's eager to add new options,
> and reluctant to remove them, at least if anyone at all finds them
> interesting, that's presumably a non-issue, though.
>
> Cheers,
> aj
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> ------------------------------
>
> End of bitcoin-dev Digest, Vol 89, Issue 77
> *******************************************
>
[-- Attachment #2: Type: text/html, Size: 30807 bytes --]
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2022-11-10 14:38 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-26 23:52 [bitcoin-dev] On mempool policy consistency Anthony Towns
2022-10-27 12:36 ` Gloria Zhao
2022-10-27 15:37 ` Anthony Towns
2022-10-27 18:17 ` Luke Dashjr
2022-10-27 13:49 ` Greg Sanders
2022-10-27 15:00 ` Peter Todd
2022-10-27 20:29 ` Antoine Riard
2022-10-30 2:24 ` Anthony Towns
2022-10-29 7:45 ` David A. Harding
2022-10-30 1:02 ` Anthony Towns
2022-10-30 2:40 ` Anthony Towns
2022-10-30 11:06 ` email
2022-10-31 13:02 ` Suhas Daftuar
2022-10-31 16:25 ` Greg Sanders
2022-10-31 17:21 ` email
2022-10-31 17:51 ` Peter Todd
2022-11-04 10:28 ` email
2022-11-02 3:07 ` Anthony Towns
2022-11-02 13:32 ` Greg Sanders
2022-11-02 19:50 ` Antoine Riard
2022-11-05 2:35 ` Peter Todd
[not found] <mailman.38435.1666828344.956.bitcoin-dev@lists.linuxfoundation.org>
2022-10-27 9:56 ` John Carvalho
2022-10-27 17:21 ` Anthony Towns
2022-10-27 17:35 ` Suhas Daftuar
2022-10-27 17:44 ` Greg Sanders
2022-10-27 19:00 ` Greg Sanders
2022-11-08 9:28 ` AdamISZ
2022-11-10 14:38 ` email
2022-11-03 21:06 email
2022-11-07 14:32 ` Peter Todd
2022-11-07 14:47 ` Erik Aronesty
2022-11-08 14:54 ` email
2022-11-09 12:05 ` email
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox