Hi Bastien > In the case of LN, an attacker can game this and heavily restrict your RBF attempts if you're only allowed to use confirmed inputs and have many channels (and a limited number of confirmed inputs). Otherwise you'll need node operators to pre-emptively split their utxos into many small utxos just for fee bumping, which is inefficient... I share the concern about splitting utxos into smaller ones. IIRC, the carve-out tolerance is only 2txn/10_000 vb. If one of your counterparties attach a junk branch on her own anchor output, are you allowed to chain your self-owned unconfirmed CPFP ? I'm thinking about the topology "Chained CPFPs" exposed here : https://github.com/rust-bitcoin/rust-lightning/issues/989. Or if you have another L2 broadcast topology which could be safe w.r.t our current mempool logic :) ? Le lun. 27 sept. 2021 à 03:15, Bastien TEINTURIER a écrit : > I think we could restrain package acceptance to only confirmed inputs for >> now and revisit later this point ? For LN-anchor, you can assume that the >> fee-bumping UTXO feeding the CPFP is already >> confirmed. Or are there currently-deployed use-cases which would benefit >> from your proposed Rule #2 ? >> > > I think constraining package acceptance to only confirmed inputs > is very limiting and quite dangerous for L2 protocols. > > In the case of LN, an attacker can game this and heavily restrict > your RBF attempts if you're only allowed to use confirmed inputs > and have many channels (and a limited number of confirmed inputs). > Otherwise you'll need node operators to pre-emptively split their > utxos into many small utxos just for fee bumping, which is inefficient... > > Bastien > > Le lun. 27 sept. 2021 à 00:27, Antoine Riard via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> a écrit : > >> Hi Gloria, >> >> Thanks for your answers, >> >> > In summary, it seems that the decisions that might still need >> > attention/input from devs on this mailing list are: >> > 1. Whether we should start with multiple-parent-1-child or >> 1-parent-1-child. >> > 2. Whether it's ok to require that the child not have conflicts with >> > mempool transactions. >> >> Yes 1) it would be good to have inputs of more potential users of package >> acceptance . And 2) I think it's more a matter of clearer wording of the >> proposal. >> >> However, see my final point on the relaxation around "unconfirmed inputs" >> which might in fact alter our current block construction strategy. >> >> > Right, the fact that we essentially always choose the first-seen >> witness is >> > an unfortunate limitation that exists already. Adding package mempool >> > accept doesn't worsen this, but the procedure in the future is to >> replace >> > the witness when it makes sense economically. We can also add logic to >> > allow package feerate to pay for witness replacements as well. This is >> > pretty far into the future, though. >> >> Yes I agree package mempool doesn't worsen this. And it's not an issue >> for current LN as you can't significantly inflate a spending witness for >> the 2-of-2 funding output. >> However, it might be an issue for multi-party protocol where the spending >> script has alternative branches with asymmetric valid witness weights. >> Taproot should ease that kind of script so hopefully we would deploy >> wtxid-replacement not too far in the future. >> >> > I could be misunderstanding, but an attacker wouldn't be able to >> > batch-attack like this. Alice's package only conflicts with A' + D', >> not A' >> > + B' + C' + D'. She only needs to pay for evicting 2 transactions. >> >> Yeah I can be clearer, I think you have 2 pinning attacks scenarios to >> consider. >> >> In LN, if you're trying to confirm a commitment transaction to time-out >> or claim on-chain a HTLC and the timelock is near-expiration, you should be >> ready to pay in commitment+2nd-stage HTLC transaction fees as much as the >> value offered by the HTLC. >> >> Following this security assumption, an attacker can exploit it by >> targeting together commitment transactions from different channels by >> blocking them under a high-fee child, of which the fee value >> is equal to the top-value HTLC + 1. Victims's fee-bumping logics won't >> overbid as it's not worthy to offer fees beyond their competed HTLCs. Apart >> from observing mempools state, victims can't learn they're targeted by the >> same attacker. >> >> To draw from the aforementioned topology, Mallory broadcasts A' + B' + C' >> + D', where A' conflicts with Alice's P1, B' conflicts with Bob's P2, C' >> conflicts with Caroll's P3. Let's assume P1 is confirming the top-value >> HTLC of the set. If D' fees is higher than P1 + 1, it won't be rational for >> Alice or Bob or Caroll to keep offering competing feerates. Mallory will be >> at loss on stealing P1, as she has paid more in fees but will realize a >> gain on P2+P3. >> >> In this model, Alice is allowed to evict those 2 transactions (A' + D') >> but as she is economically-bounded she won't succeed. >> >> Mallory is maliciously exploiting RBF rule 3 on absolute fee. I think >> this 1st pinning scenario is correct and "lucractive" when you sum the >> global gain/loss. >> >> There is a 2nd attack scenario where A + B + C + D, where D is the child >> of A,B,C. All those transactions are honestly issued by Alice. Once A + B + >> C + D are propagated in network mempools, Mallory is able to replace A + D >> with A' + D' where D' is paying a higher fee. This package A' + D' will >> confirm soon if D feerate was compelling but Mallory succeeds in delaying >> the confirmation >> of B + C for one or more blocks. As B + C are pre-signed commitments with >> a low-fee rate they won't confirm without Alice issuing a new child E. >> Mallory can repeat the same trick by broadcasting >> B' + E' and delay again the confirmation of C. >> >> If the remaining package pending HTLC has a higher-value than all the >> malicious fees over-bid, Mallory should realize a gain. With this 2nd >> pinning attack, the malicious entity buys confirmation delay of your >> packaged-together commitments. >> >> Assuming those attacks are correct, I'm leaning towards being >> conservative with the LDK broadcast backend. Though once again, other L2 >> devs have likely other use-cases and opinions :) >> >> > B' only needs to pay for itself in this case. >> >> Yes I think it's a nice discount when UTXO is single-owned. In the >> context of shared-owned UTXO (e.g LN), you might not if there is an >> in-mempool package already spending the UTXO and have to assume the >> worst-case scenario. I.e have B' committing enough fee to pay for A' >> replacement bandwidth. I think we can't do that much for this case... >> >> > If a package meets feerate requirements as a >> package, the parents in the transaction are allowed to replace-by-fee >> mempool transactions. The child cannot replace mempool transactions." >> >> I agree with the Mallory-vs-Alice case. Though if Alice broadcasts A+B' >> to replace A+B because the first broadcast isn't satisfying anymore due to >> mempool spikes ? Assuming B' fees is enough, I think that case as child B' >> replacing in-mempool transaction B. Which I understand going against "The >> child cannot replace mempool transactions". >> >> Maybe wording could be a bit clearer ? >> >> > While it would be nice to have full RBF, malleability of the child won't >> > block RBF here. If we're trying to replace A', we only require that A' >> > signals replaceability, and don't mind if its child doesn't. >> >> Yes, it sounds good. >> >> > Yes, A+C+D pays 2500sat more in fees, but it is also 1000vB larger. A >> miner >> > should prefer to utilize their block space more effectively. >> >> If your mempool is empty and only composed of A+C+D or A+B, I think >> taking A+C+D is the most efficient block construction you can come up with >> as a miner ? >> >> > No, because we don't use that model. >> >> Can you describe what miner model we are using ? Like the block >> construction strategy implemented by `addPackagesTxs` or also encompassing >> our current mempool acceptance policy, which I think rely on absolute fee >> over ancestor score in case of replacement ? >> >> I think this point is worthy to discuss as otherwise we might downgrade >> the efficiency of our current block construction strategy in periods of >> near-empty mempools. A knowledge which could be discreetly leveraged by a >> miner to gain an advantage on the rest of the mining ecosystem. >> >> Note, I think we *might* have to go in this direction if we want to >> replace replace-by-fee by replace-by-feerate or replace-by-ancestor and >> solve in-depth pinning attacks. Though if we do so, >> IMO we would need more thoughts. >> >> I think we could restrain package acceptance to only confirmed inputs for >> now and revisit later this point ? For LN-anchor, you can assume that the >> fee-bumping UTXO feeding the CPFP is already >> confirmed. Or are there currently-deployed use-cases which would benefit >> from your proposed Rule #2 ? >> >> Antoine >> >> Le jeu. 23 sept. 2021 à 11:36, Gloria Zhao a >> écrit : >> >>> Hi Antoine, >>> >>> Thanks as always for your input. I'm glad we agree on so much! >>> >>> In summary, it seems that the decisions that might still need >>> attention/input from devs on this mailing list are: >>> 1. Whether we should start with multiple-parent-1-child or >>> 1-parent-1-child. >>> 2. Whether it's ok to require that the child not have conflicts with >>> mempool transactions. >>> >>> Responding to your comments... >>> >>> > IIUC, you have package A+B, during the dedup phase early in >>> `AcceptMultipleTransactions` if you observe same-txid-different-wtixd A' >>> and A' is higher feerate than A, you trim A and replace by A' ? >>> >>> > I think this approach is safe, the one who appears unsafe to me is >>> when A' has a _lower_ feerate, even if A' is already accepted by our >>> mempool ? In that case iirc that would be a pinning. >>> >>> Right, the fact that we essentially always choose the first-seen witness >>> is an unfortunate limitation that exists already. Adding package mempool >>> accept doesn't worsen this, but the procedure in the future is to replace >>> the witness when it makes sense economically. We can also add logic to >>> allow package feerate to pay for witness replacements as well. This is >>> pretty far into the future, though. >>> >>> > It sounds uneconomical for an attacker but I think it's not when you >>> consider than you can "batch" attack against multiple honest >>> counterparties. E.g, Mallory broadcast A' + B' + C' + D' where A' conflicts >>> with Alice's honest package P1, B' conflicts with Bob's honest package P2, >>> C' conflicts with Caroll's honest package P3. And D' is a high-fee child of >>> A' + B' + C'. >>> >>> > If D' is higher-fee than P1 or P2 or P3 but inferior to the sum of >>> HTLCs confirmed by P1+P2+P3, I think it's lucrative for the attacker ? >>> >>> I could be misunderstanding, but an attacker wouldn't be able to >>> batch-attack like this. Alice's package only conflicts with A' + D', not A' >>> + B' + C' + D'. She only needs to pay for evicting 2 transactions. >>> >>> > Do we assume that broadcasted packages are "honest" by default and >>> that the parent(s) always need the child to pass the fee checks, that way >>> saving the processing of individual transactions which are expected to fail >>> in 99% of cases or more ad hoc composition of packages at relay ? >>> > I think this point is quite dependent on the p2p packages format/logic >>> we'll end up on and that we should feel free to revisit it later ? >>> >>> I think it's the opposite; there's no way for us to assume that p2p >>> packages will be "honest." I'd like to have two things before we expose on >>> P2P: (1) ensure that the amount of resources potentially allocated for >>> package validation isn't disproportionately higher than that of single >>> transaction validation and (2) only use package validation when we're >>> unsatisifed with the single validation result, e.g. we might get better >>> fees. >>> Yes, let's revisit this later :) >>> >>> > Yes, if you receive A+B, and A is already in-mempoo, I agree you can >>> discard its feerate as B should pay for all fees checked on its own. Where >>> I'm unclear is when you have in-mempool A+B and receive A+B'. Should B' >>> have a fee high enough to cover the bandwidth penalty replacement >>> (`PaysForRBF`, 2nd check) of both A+B' or only B' ? >>> >>> B' only needs to pay for itself in this case. >>> >>> > > Do we want the child to be able to replace mempool transactions as >>> well? >>> >>> > If we mean when you have replaceable A+B then A'+B' try to replace >>> with a higher-feerate ? I think that's exactly the case we need for >>> Lightning as A+B is coming from Alice and A'+B' is coming from Bob :/ >>> >>> Let me clarify this because I can see that my wording was ambiguous, and >>> then please let me know if it fits Lightning's needs? >>> >>> In my proposal, I wrote "If a package meets feerate requirements as a >>> package, the parents in the transaction are allowed to replace-by-fee >>> mempool transactions. The child cannot replace mempool transactions." What >>> I meant was: the package can replace mempool transactions if any of the >>> parents conflict with mempool transactions. The child cannot not conflict >>> with any mempool transactions. >>> The Lightning use case this attempts to address is: Alice and Mallory >>> are LN counterparties, and have packages A+B and A'+B', respectively. A and >>> A' are their commitment transactions and conflict with each other; they >>> have shared inputs and different txids. >>> B spends Alice's anchor output from A. B' spends Mallory's anchor output >>> from A'. Thus, B and B' do not conflict with each other. >>> Alice can broadcast her package, A+B, to replace Mallory's package, >>> A'+B', since B doesn't conflict with the mempool. >>> >>> Would this be ok? >>> >>> > The second option, a child of A', In the LN case I think the CPFP is >>> attached on one's anchor output. >>> >>> While it would be nice to have full RBF, malleability of the child won't >>> block RBF here. If we're trying to replace A', we only require that A' >>> signals replaceability, and don't mind if its child doesn't. >>> >>> > > B has an ancestor score of 10sat/vb and D has an >>> > > ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than >>> B's, >>> > > it fails the proposed package RBF Rule #2, so this package would be >>> > > rejected. Does this meet your expectations? >>> >>> > Well what sounds odd to me, in my example, we fail D even if it has a >>> higher-fee than B. Like A+B absolute fees are 2000 sats and A+C+D absolute >>> fees are 4500 sats ? >>> >>> Yes, A+C+D pays 2500sat more in fees, but it is also 1000vB larger. A >>> miner should prefer to utilize their block space more effectively. >>> >>> > Is this compatible with a model where a miner prioritizes absolute >>> fees over ancestor score, in the case that mempools aren't full-enough to >>> fulfill a block ? >>> >>> No, because we don't use that model. >>> >>> Thanks, >>> Gloria >>> >>> On Thu, Sep 23, 2021 at 5:29 AM Antoine Riard >>> wrote: >>> >>>> > Correct, if B+C is too low feerate to be accepted, we will reject it. >>>> I >>>> > prefer this because it is incentive compatible: A can be mined by >>>> itself, >>>> > so there's no reason to prefer A+B+C instead of A. >>>> > As another way of looking at this, consider the case where we do >>>> accept >>>> > A+B+C and it sits at the "bottom" of our mempool. If our mempool >>>> reaches >>>> > capacity, we evict the lowest descendant feerate transactions, which >>>> are >>>> > B+C in this case. This gives us the same resulting mempool, with A >>>> and not >>>> > B+C. >>>> >>>> I agree here. Doing otherwise, we might evict other transactions >>>> mempool in `MempoolAccept::Finalize` with a higher-feerate than B+C while >>>> those evicted transactions are the most compelling for block construction. >>>> >>>> I thought at first missing this acceptance requirement would break a >>>> fee-bumping scheme like Parent-Pay-For-Child where a high-fee parent is >>>> attached to a child signed with SIGHASH_ANYONECANPAY but in this case the >>>> child fee is capturing the parent value. I can't think of other fee-bumping >>>> schemes potentially affected. If they do exist I would say they're wrong in >>>> their design assumptions. >>>> >>>> > If or when we have witness replacement, the logic is: if the >>>> individual >>>> > transaction is enough to replace the mempool one, the replacement will >>>> > happen during the preceding individual transaction acceptance, and >>>> > deduplication logic will work. Otherwise, we will try to deduplicate >>>> by >>>> > wtxid, see that we need a package witness replacement, and use the >>>> package >>>> > feerate to evaluate whether this is economically rational. >>>> >>>> IIUC, you have package A+B, during the dedup phase early in >>>> `AcceptMultipleTransactions` if you observe same-txid-different-wtixd A' >>>> and A' is higher feerate than A, you trim A and replace by A' ? >>>> >>>> I think this approach is safe, the one who appears unsafe to me is when >>>> A' has a _lower_ feerate, even if A' is already accepted by our mempool ? >>>> In that case iirc that would be a pinning. >>>> >>>> Good to see progress on witness replacement before we see usage of >>>> Taproot tree in the context of multi-party, where a malicious counterparty >>>> inflates its witness to jam a honest spending. >>>> >>>> (Note, the commit linked currently points nowhere :)) >>>> >>>> >>>> > Please note that A may replace A' even if A' has higher fees than A >>>> > individually, because the proposed package RBF utilizes the fees and >>>> size >>>> > of the entire package. This just requires E to pay enough fees, >>>> although >>>> > this can be pretty high if there are also potential B' and C' >>>> competing >>>> > commitment transactions that we don't know about. >>>> >>>> Ah right, if the package acceptance waives `PaysMoreThanConflicts` for >>>> the individual check on A, the honest package should replace the pinning >>>> attempt. I've not fully parsed the proposed implementation yet. >>>> >>>> Though note, I think it's still unsafe for a Lightning >>>> multi-commitment-broadcast-as-one-package as a malicious A' might have an >>>> absolute fee higher than E. It sounds uneconomical for >>>> an attacker but I think it's not when you consider than you can "batch" >>>> attack against multiple honest counterparties. E.g, Mallory broadcast A' + >>>> B' + C' + D' where A' conflicts with Alice's honest package P1, B' >>>> conflicts with Bob's honest package P2, C' conflicts with Caroll's honest >>>> package P3. And D' is a high-fee child of A' + B' + C'. >>>> >>>> If D' is higher-fee than P1 or P2 or P3 but inferior to the sum of >>>> HTLCs confirmed by P1+P2+P3, I think it's lucrative for the attacker ? >>>> >>>> > So far, my understanding is that multi-parent-1-child is desired for >>>> > batched fee-bumping ( >>>> > https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-897951289) >>>> and >>>> > I've also seen your response which I have less context on ( >>>> > https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-900352202). >>>> That >>>> > being said, I am happy to create a new proposal for 1 parent + 1 child >>>> > (which would be slightly simpler) and plan for moving to >>>> > multi-parent-1-child later if that is preferred. I am very interested >>>> in >>>> > hearing feedback on that approach. >>>> >>>> I think batched fee-bumping is okay as long as you don't have >>>> time-sensitive outputs encumbering your commitment transactions. For the >>>> reasons mentioned above, I think that's unsafe. >>>> >>>> What I'm worried about is L2 developers, potentially not aware about >>>> all the mempool subtleties blurring the difference and always batching >>>> their broadcast by default. >>>> >>>> IMO, a good thing by restraining to 1-parent + 1 child, we >>>> artificially constraint L2 design space for now and minimize risks of >>>> unsafe usage of the package API :) >>>> >>>> I think that's a point where it would be relevant to have the opinion >>>> of more L2 devs. >>>> >>>> > I think there is a misunderstanding here - let me describe what I'm >>>> > proposing we'd do in this situation: we'll try individual submission >>>> for A, >>>> > see that it fails due to "insufficient fees." Then, we'll try package >>>> > validation for A+B and use package RBF. If A+B pays enough, it can >>>> still >>>> > replace A'. If A fails for a bad signature, we won't look at B or >>>> A+B. Does >>>> > this meet your expectations? >>>> >>>> Yes there was a misunderstanding, I think this approach is correct, >>>> it's more a question of performance. Do we assume that broadcasted packages >>>> are "honest" by default and that the parent(s) always need the child to >>>> pass the fee checks, that way saving the processing of individual >>>> transactions which are expected to fail in 99% of cases or more ad hoc >>>> composition of packages at relay ? >>>> >>>> I think this point is quite dependent on the p2p packages format/logic >>>> we'll end up on and that we should feel free to revisit it later ? >>>> >>>> >>>> > What problem are you trying to solve by the package feerate *after* >>>> dedup >>>> rule ? >>>> > My understanding is that an in-package transaction might be already in >>>> the mempool. Therefore, to compute a correct RBF penalty replacement, >>>> the >>>> vsize of this transaction could be discarded lowering the cost of >>>> package >>>> RBF. >>>> >>>> > I'm proposing that, when a transaction has already been submitted to >>>> > mempool, we would ignore both its fees and vsize when calculating >>>> package >>>> > feerate. >>>> >>>> Yes, if you receive A+B, and A is already in-mempoo, I agree you can >>>> discard its feerate as B should pay for all fees checked on its own. Where >>>> I'm unclear is when you have in-mempool A+B and receive A+B'. Should B' >>>> have a fee high enough to cover the bandwidth penalty replacement >>>> (`PaysForRBF`, 2nd check) of both A+B' or only B' ? >>>> >>>> If you have a second-layer like current Lightning, you might have a >>>> counterparty commitment to replace and should always expect to have to pay >>>> for parent replacement bandwidth. >>>> >>>> Where a potential discount sounds interesting is when you have an >>>> univoque state on the first-stage of transactions. E.g DLC's funding >>>> transaction which might be CPFP by any participant iirc. >>>> >>>> > Note that, if C' conflicts with C, it also conflicts with D, since D >>>> is a >>>> > descendant of C and would thus need to be evicted along with it. >>>> >>>> Ah once again I think it's a misunderstanding without the code under my >>>> eyes! If we do C' `PreChecks`, solve the conflicts provoked by it, i.e mark >>>> for potential eviction D and don't consider it for future conflicts in the >>>> rest of the package, I think D' `PreChecks` should be good ? >>>> >>>> > More generally, this example is surprising to me because I didn't >>>> think >>>> > packages would be used to fee-bump replaceable transactions. Do we >>>> want the >>>> > child to be able to replace mempool transactions as well? >>>> >>>> If we mean when you have replaceable A+B then A'+B' try to replace with >>>> a higher-feerate ? I think that's exactly the case we need for Lightning as >>>> A+B is coming from Alice and A'+B' is coming from Bob :/ >>>> >>>> > I'm not sure what you mean? Let's say we have a package of parent A + >>>> child >>>> > B, where A is supposed to replace a mempool transaction A'. Are you >>>> saying >>>> > that counterparties are able to malleate the package child B, or a >>>> child of >>>> > A'? >>>> >>>> The second option, a child of A', In the LN case I think the CPFP is >>>> attached on one's anchor output. >>>> >>>> I think it's good if we assume the >>>> solve-conflicts-after-parent's`'PreChecks` mentioned above or fixing >>>> inherited signaling or full-rbf ? >>>> >>>> > Sorry, I don't understand what you mean by "preserve the package >>>> > integrity?" Could you elaborate? >>>> >>>> After thinking the relaxation about the "new" unconfirmed input is not >>>> linked to trimming but I would say more to the multi-parent support. >>>> >>>> Let's say you have A+B trying to replace C+D where B is also spending >>>> already in-mempool E. To succeed, you need to waive the no-new-unconfirmed >>>> input as D isn't spending E. >>>> >>>> So good, I think we agree on the problem description here. >>>> >>>> > I am in agreement with your calculations but unsure if we disagree on >>>> the >>>> > expected outcome. Yes, B has an ancestor score of 10sat/vb and D has >>>> an >>>> > ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than >>>> B's, >>>> > it fails the proposed package RBF Rule #2, so this package would be >>>> > rejected. Does this meet your expectations? >>>> >>>> Well what sounds odd to me, in my example, we fail D even if it has a >>>> higher-fee than B. Like A+B absolute fees are 2000 sats and A+C+D absolute >>>> fees are 4500 sats ? >>>> >>>> Is this compatible with a model where a miner prioritizes absolute fees >>>> over ancestor score, in the case that mempools aren't full-enough to >>>> fulfill a block ? >>>> >>>> Let me know if I can clarify a point. >>>> >>>> Antoine >>>> >>>> Le lun. 20 sept. 2021 à 11:10, Gloria Zhao a >>>> écrit : >>>> >>>>> >>>>> Hi Antoine, >>>>> >>>>> First of all, thank you for the thorough review. I appreciate your >>>>> insight on LN requirements. >>>>> >>>>> > IIUC, you have a package A+B+C submitted for acceptance and A is >>>>> already in your mempool. You trim out A from the package and then evaluate >>>>> B+C. >>>>> >>>>> > I think this might be an issue if A is the higher-fee element of the >>>>> ABC package. B+C package fees might be under the mempool min fee and will >>>>> be rejected, potentially breaking the acceptance expectations of the >>>>> package issuer ? >>>>> >>>>> Correct, if B+C is too low feerate to be accepted, we will reject it. >>>>> I prefer this because it is incentive compatible: A can be mined by itself, >>>>> so there's no reason to prefer A+B+C instead of A. >>>>> As another way of looking at this, consider the case where we do >>>>> accept A+B+C and it sits at the "bottom" of our mempool. If our mempool >>>>> reaches capacity, we evict the lowest descendant feerate transactions, >>>>> which are B+C in this case. This gives us the same resulting mempool, with >>>>> A and not B+C. >>>>> >>>>> >>>>> > Further, I think the dedup should be done on wtxid, as you might >>>>> have multiple valid witnesses. Though with varying vsizes and as such >>>>> offering different feerates. >>>>> >>>>> I agree that variations of the same package with different witnesses >>>>> is a case that must be handled. I consider witness replacement to be a >>>>> project that can be done in parallel to package mempool acceptance because >>>>> being able to accept packages does not worsen the problem of a >>>>> same-txid-different-witness "pinning" attack. >>>>> >>>>> If or when we have witness replacement, the logic is: if the >>>>> individual transaction is enough to replace the mempool one, the >>>>> replacement will happen during the preceding individual transaction >>>>> acceptance, and deduplication logic will work. Otherwise, we will try to >>>>> deduplicate by wtxid, see that we need a package witness replacement, and >>>>> use the package feerate to evaluate whether this is economically rational. >>>>> >>>>> See the #22290 "handle package transactions already in mempool" commit >>>>> ( >>>>> https://github.com/bitcoin/bitcoin/pull/22290/commits/fea75a2237b46cf76145242fecad7e274bfcb5ff), >>>>> which handles the case of same-txid-different-witness by simply using the >>>>> transaction in the mempool for now, with TODOs for what I just described. >>>>> >>>>> >>>>> > I'm not clearly understanding the accepted topologies. By "parent >>>>> and child to share a parent", do you mean the set of transactions A, B, C, >>>>> where B is spending A and C is spending A and B would be correct ? >>>>> >>>>> Yes, that is what I meant. Yes, that would a valid package under these >>>>> rules. >>>>> >>>>> > If yes, is there a width-limit introduced or we fallback on >>>>> MAX_PACKAGE_COUNT=25 ? >>>>> >>>>> No, there is no limit on connectivity other than "child with all >>>>> unconfirmed parents." We will enforce MAX_PACKAGE_COUNT=25 and child's >>>>> in-mempool + in-package ancestor limits. >>>>> >>>>> >>>>> > Considering the current Core's mempool acceptance rules, I think >>>>> CPFP batching is unsafe for LN time-sensitive closure. A malicious tx-relay >>>>> jamming successful on one channel commitment transaction would contamine >>>>> the remaining commitments sharing the same package. >>>>> >>>>> > E.g, you broadcast the package A+B+C+D+E where A,B,C,D are >>>>> commitment transactions and E a shared CPFP. If a malicious A' transaction >>>>> has a better feerate than A, the whole package acceptance will fail. Even >>>>> if A' confirms in the following block, >>>>> the propagation and confirmation of B+C+D have been delayed. This >>>>> could carry on a loss of funds. >>>>> >>>>> Please note that A may replace A' even if A' has higher fees than A >>>>> individually, because the proposed package RBF utilizes the fees and size >>>>> of the entire package. This just requires E to pay enough fees, although >>>>> this can be pretty high if there are also potential B' and C' competing >>>>> commitment transactions that we don't know about. >>>>> >>>>> >>>>> > IMHO, I'm leaning towards deploying during a first phase >>>>> 1-parent/1-child. I think it's the most conservative step still improving >>>>> second-layer safety. >>>>> >>>>> So far, my understanding is that multi-parent-1-child is desired for >>>>> batched fee-bumping ( >>>>> https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-897951289) >>>>> and I've also seen your response which I have less context on ( >>>>> https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-900352202). >>>>> That being said, I am happy to create a new proposal for 1 parent + 1 child >>>>> (which would be slightly simpler) and plan for moving to >>>>> multi-parent-1-child later if that is preferred. I am very interested in >>>>> hearing feedback on that approach. >>>>> >>>>> >>>>> > If A+B is submitted to replace A', where A pays 0 sats, B pays 200 >>>>> sats and A' pays 100 sats. If we apply the individual RBF on A, A+B >>>>> acceptance fails. For this reason I think the individual RBF should be >>>>> bypassed and only the package RBF apply ? >>>>> >>>>> I think there is a misunderstanding here - let me describe what I'm >>>>> proposing we'd do in this situation: we'll try individual submission for A, >>>>> see that it fails due to "insufficient fees." Then, we'll try package >>>>> validation for A+B and use package RBF. If A+B pays enough, it can still >>>>> replace A'. If A fails for a bad signature, we won't look at B or A+B. Does >>>>> this meet your expectations? >>>>> >>>>> >>>>> > What problem are you trying to solve by the package feerate *after* >>>>> dedup rule ? >>>>> > My understanding is that an in-package transaction might be already >>>>> in the mempool. Therefore, to compute a correct RBF penalty replacement, >>>>> the vsize of this transaction could be discarded lowering the cost of >>>>> package RBF. >>>>> >>>>> I'm proposing that, when a transaction has already been submitted to >>>>> mempool, we would ignore both its fees and vsize when calculating package >>>>> feerate. In example G2, we shouldn't count M1 fees after its submission to >>>>> mempool, since M1's fees have already been used to pay for its individual >>>>> bandwidth, and it shouldn't be used again to pay for P2 and P3's bandwidth. >>>>> We also shouldn't count its vsize, since it has already been paid for. >>>>> >>>>> >>>>> > I think this is a footgunish API, as if a package issuer send the >>>>> multiple-parent-one-child package A,B,C,D where D is the child of A,B,C. >>>>> Then try to broadcast the higher-feerate C'+D' package, it should be >>>>> rejected. So it's breaking the naive broadcaster assumption that a >>>>> higher-feerate/higher-fee package always replaces ? >>>>> >>>>> Note that, if C' conflicts with C, it also conflicts with D, since D >>>>> is a descendant of C and would thus need to be evicted along with it. >>>>> Implicitly, D' would not be in conflict with D. >>>>> More generally, this example is surprising to me because I didn't >>>>> think packages would be used to fee-bump replaceable transactions. Do we >>>>> want the child to be able to replace mempool transactions as well? This can >>>>> be implemented with a bit of additional logic. >>>>> >>>>> > I think this is unsafe for L2s if counterparties have malleability >>>>> of the child transaction. They can block your package replacement by >>>>> opting-out from RBF signaling. IIRC, LN's "anchor output" presents such an >>>>> ability. >>>>> >>>>> I'm not sure what you mean? Let's say we have a package of parent A + >>>>> child B, where A is supposed to replace a mempool transaction A'. Are you >>>>> saying that counterparties are able to malleate the package child B, or a >>>>> child of A'? If they can malleate a child of A', that shouldn't matter as >>>>> long as A' is signaling replacement. This would be handled identically with >>>>> full RBF and what Core currently implements. >>>>> >>>>> > I think this is an issue brought by the trimming during the dedup >>>>> phase. If we preserve the package integrity, only re-using the tx-level >>>>> checks results of already in-mempool transactions to gain in CPU time we >>>>> won't have this issue. Package childs can add unconfirmed inputs as long as >>>>> they're in-package, the bip125 rule2 is only evaluated against parents ? >>>>> >>>>> Sorry, I don't understand what you mean by "preserve the package >>>>> integrity?" Could you elaborate? >>>>> >>>>> > Let's say you have in-mempool A, B where A pays 10 sat/vb for 100 >>>>> vbytes and B pays 10 sat/vb for 100 vbytes. You have the candidate >>>>> replacement D spending both A and C where D pays 15sat/vb for 100 vbytes >>>>> and C pays 1 sat/vb for 1000 vbytes. >>>>> >>>>> > Package A + B ancestor score is 10 sat/vb. >>>>> >>>>> > D has a higher feerate/absolute fee than B. >>>>> >>>>> > Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's >>>>> 1000 sats + D's 1500 sats) / A's 100 vb + C's 1000 vb + D's 100 vb) >>>>> >>>>> I am in agreement with your calculations but unsure if we disagree on >>>>> the expected outcome. Yes, B has an ancestor score of 10sat/vb and D has an >>>>> ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than B's, >>>>> it fails the proposed package RBF Rule #2, so this package would be >>>>> rejected. Does this meet your expectations? >>>>> >>>>> Thank you for linking to projects that might be interested in package >>>>> relay :) >>>>> >>>>> Thanks, >>>>> Gloria >>>>> >>>>> On Mon, Sep 20, 2021 at 12:16 AM Antoine Riard < >>>>> antoine.riard@gmail.com> wrote: >>>>> >>>>>> Hi Gloria, >>>>>> >>>>>> > A package may contain transactions that are already in the mempool. >>>>>> We >>>>>> > remove >>>>>> > ("deduplicate") those transactions from the package for the >>>>>> purposes of >>>>>> > package >>>>>> > mempool acceptance. If a package is empty after deduplication, we do >>>>>> > nothing. >>>>>> >>>>>> IIUC, you have a package A+B+C submitted for acceptance and A is >>>>>> already in your mempool. You trim out A from the package and then evaluate >>>>>> B+C. >>>>>> >>>>>> I think this might be an issue if A is the higher-fee element of the >>>>>> ABC package. B+C package fees might be under the mempool min fee and will >>>>>> be rejected, potentially breaking the acceptance expectations of the >>>>>> package issuer ? >>>>>> >>>>>> Further, I think the dedup should be done on wtxid, as you might have >>>>>> multiple valid witnesses. Though with varying vsizes and as such offering >>>>>> different feerates. >>>>>> >>>>>> E.g you're going to evaluate the package A+B and A' is already in >>>>>> your mempool with a bigger valid witness. You trim A based on txid, then >>>>>> you evaluate A'+B, which fails the fee checks. However, evaluating A+B >>>>>> would have been a success. >>>>>> >>>>>> AFAICT, the dedup rationale would be to save on CPU time/IO disk, to >>>>>> avoid repeated signatures verification and parent UTXOs fetches ? Can we >>>>>> achieve the same goal by bypassing tx-level checks for already-in txn while >>>>>> conserving the package integrity for package-level checks ? >>>>>> >>>>>> > Note that it's possible for the parents to be >>>>>> > indirect >>>>>> > descendants/ancestors of one another, or for parent and child to >>>>>> share a >>>>>> > parent, >>>>>> > so we cannot make any other topology assumptions. >>>>>> >>>>>> I'm not clearly understanding the accepted topologies. By "parent and >>>>>> child to share a parent", do you mean the set of transactions A, B, C, >>>>>> where B is spending A and C is spending A and B would be correct ? >>>>>> >>>>>> If yes, is there a width-limit introduced or we fallback on >>>>>> MAX_PACKAGE_COUNT=25 ? >>>>>> >>>>>> IIRC, one rationale to come with this topology limitation was to >>>>>> lower the DoS risks when potentially deploying p2p packages. >>>>>> >>>>>> Considering the current Core's mempool acceptance rules, I think CPFP >>>>>> batching is unsafe for LN time-sensitive closure. A malicious tx-relay >>>>>> jamming successful on one channel commitment transaction would contamine >>>>>> the remaining commitments sharing the same package. >>>>>> >>>>>> E.g, you broadcast the package A+B+C+D+E where A,B,C,D are commitment >>>>>> transactions and E a shared CPFP. If a malicious A' transaction has a >>>>>> better feerate than A, the whole package acceptance will fail. Even if A' >>>>>> confirms in the following block, >>>>>> the propagation and confirmation of B+C+D have been delayed. This >>>>>> could carry on a loss of funds. >>>>>> >>>>>> That said, if you're broadcasting commitment transactions without >>>>>> time-sensitive HTLC outputs, I think the batching is effectively a fee >>>>>> saving as you don't have to duplicate the CPFP. >>>>>> >>>>>> IMHO, I'm leaning towards deploying during a first phase >>>>>> 1-parent/1-child. I think it's the most conservative step still improving >>>>>> second-layer safety. >>>>>> >>>>>> > *Rationale*: It would be incorrect to use the fees of transactions >>>>>> that are >>>>>> > already in the mempool, as we do not want a transaction's fees to be >>>>>> > double-counted for both its individual RBF and package RBF. >>>>>> >>>>>> I'm unsure about the logical order of the checks proposed. >>>>>> >>>>>> If A+B is submitted to replace A', where A pays 0 sats, B pays 200 >>>>>> sats and A' pays 100 sats. If we apply the individual RBF on A, A+B >>>>>> acceptance fails. For this reason I think the individual RBF should be >>>>>> bypassed and only the package RBF apply ? >>>>>> >>>>>> Note this situation is plausible, with current LN design, your >>>>>> counterparty can have a commitment transaction with a better fee just by >>>>>> selecting a higher `dust_limit_satoshis` than yours. >>>>>> >>>>>> > Examples F and G [14] show the same package, but P1 is submitted >>>>>> > individually before >>>>>> > the package in example G. In example F, we can see that the 300vB >>>>>> package >>>>>> > pays >>>>>> > an additional 200sat in fees, which is not enough to pay for its own >>>>>> > bandwidth >>>>>> > (BIP125#4). In example G, we can see that P1 pays enough to replace >>>>>> M1, but >>>>>> > using P1's fees again during package submission would make it look >>>>>> like a >>>>>> > 300sat >>>>>> > increase for a 200vB package. Even including its fees and size >>>>>> would not be >>>>>> > sufficient in this example, since the 300sat looks like enough for >>>>>> the 300vB >>>>>> > package. The calculcation after deduplication is 100sat increase >>>>>> for a >>>>>> > package >>>>>> > of size 200vB, which correctly fails BIP125#4. Assume all >>>>>> transactions have >>>>>> > a >>>>>> > size of 100vB. >>>>>> >>>>>> What problem are you trying to solve by the package feerate *after* >>>>>> dedup rule ? >>>>>> >>>>>> My understanding is that an in-package transaction might be already >>>>>> in the mempool. Therefore, to compute a correct RBF penalty replacement, >>>>>> the vsize of this transaction could be discarded lowering the cost of >>>>>> package RBF. >>>>>> >>>>>> If we keep a "safe" dedup mechanism (see my point above), I think >>>>>> this discount is justified, as the validation cost of node operators is >>>>>> paid for ? >>>>>> >>>>>> > The child cannot replace mempool transactions. >>>>>> >>>>>> Let's say you issue package A+B, then package C+B', where B' is a >>>>>> child of both A and C. This rule fails the acceptance of C+B' ? >>>>>> >>>>>> I think this is a footgunish API, as if a package issuer send the >>>>>> multiple-parent-one-child package A,B,C,D where D is the child of A,B,C. >>>>>> Then try to broadcast the higher-feerate C'+D' package, it should be >>>>>> rejected. So it's breaking the naive broadcaster assumption that a >>>>>> higher-feerate/higher-fee package always replaces ? And it might be unsafe >>>>>> in protocols where states are symmetric. E.g a malicious counterparty >>>>>> broadcasts first S+A, then you honestly broadcast S+B, where B pays better >>>>>> fees. >>>>>> >>>>>> > All mempool transactions to be replaced must signal replaceability. >>>>>> >>>>>> I think this is unsafe for L2s if counterparties have malleability of >>>>>> the child transaction. They can block your package replacement by >>>>>> opting-out from RBF signaling. IIRC, LN's "anchor output" presents such an >>>>>> ability. >>>>>> >>>>>> I think it's better to either fix inherited signaling or move towards >>>>>> full-rbf. >>>>>> >>>>>> > if a package parent has already been submitted, it would >>>>>> > look >>>>>> >like the child is spending a "new" unconfirmed input. >>>>>> >>>>>> I think this is an issue brought by the trimming during the dedup >>>>>> phase. If we preserve the package integrity, only re-using the tx-level >>>>>> checks results of already in-mempool transactions to gain in CPU time we >>>>>> won't have this issue. Package childs can add unconfirmed inputs as long as >>>>>> they're in-package, the bip125 rule2 is only evaluated against parents ? >>>>>> >>>>>> > However, we still achieve the same goal of requiring the >>>>>> > replacement >>>>>> > transactions to have a ancestor score at least as high as the >>>>>> original >>>>>> > ones. >>>>>> >>>>>> I'm not sure if this holds... >>>>>> >>>>>> Let's say you have in-mempool A, B where A pays 10 sat/vb for 100 >>>>>> vbytes and B pays 10 sat/vb for 100 vbytes. You have the candidate >>>>>> replacement D spending both A and C where D pays 15sat/vb for 100 vbytes >>>>>> and C pays 1 sat/vb for 1000 vbytes. >>>>>> >>>>>> Package A + B ancestor score is 10 sat/vb. >>>>>> >>>>>> D has a higher feerate/absolute fee than B. >>>>>> >>>>>> Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's >>>>>> 1000 sats + D's 1500 sats) / >>>>>> A's 100 vb + C's 1000 vb + D's 100 vb) >>>>>> >>>>>> Overall, this is a review through the lenses of LN requirements. I >>>>>> think other L2 protocols/applications >>>>>> could be candidates to using package accept/relay such as: >>>>>> * https://github.com/lightninglabs/pool >>>>>> * https://github.com/discreetlogcontracts/dlcspecs >>>>>> * https://github.com/bitcoin-teleport/teleport-transactions/ >>>>>> * https://github.com/sapio-lang/sapio >>>>>> * >>>>>> https://github.com/commerceblock/mercury/blob/master/doc/statechains.md >>>>>> * https://github.com/revault/practical-revault >>>>>> >>>>>> Thanks for rolling forward the ball on this subject. >>>>>> >>>>>> Antoine >>>>>> >>>>>> Le jeu. 16 sept. 2021 à 03:55, Gloria Zhao via bitcoin-dev < >>>>>> bitcoin-dev@lists.linuxfoundation.org> a écrit : >>>>>> >>>>>>> Hi there, >>>>>>> >>>>>>> I'm writing to propose a set of mempool policy changes to enable >>>>>>> package >>>>>>> validation (in preparation for package relay) in Bitcoin Core. These >>>>>>> would not >>>>>>> be consensus or P2P protocol changes. However, since mempool policy >>>>>>> significantly affects transaction propagation, I believe this is >>>>>>> relevant for >>>>>>> the mailing list. >>>>>>> >>>>>>> My proposal enables packages consisting of multiple parents and 1 >>>>>>> child. If you >>>>>>> develop software that relies on specific transaction relay >>>>>>> assumptions and/or >>>>>>> are interested in using package relay in the future, I'm very >>>>>>> interested to hear >>>>>>> your feedback on the utility or restrictiveness of these package >>>>>>> policies for >>>>>>> your use cases. >>>>>>> >>>>>>> A draft implementation of this proposal can be found in [Bitcoin Core >>>>>>> PR#22290][1]. >>>>>>> >>>>>>> An illustrated version of this post can be found at >>>>>>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a. >>>>>>> I have also linked the images below. >>>>>>> >>>>>>> ## Background >>>>>>> >>>>>>> Feel free to skip this section if you are already familiar with >>>>>>> mempool policy >>>>>>> and package relay terminology. >>>>>>> >>>>>>> ### Terminology Clarifications >>>>>>> >>>>>>> * Package = an ordered list of related transactions, representable >>>>>>> by a Directed >>>>>>> Acyclic Graph. >>>>>>> * Package Feerate = the total modified fees divided by the total >>>>>>> virtual size of >>>>>>> all transactions in the package. >>>>>>> - Modified fees = a transaction's base fees + fee delta applied >>>>>>> by the user >>>>>>> with `prioritisetransaction`. As such, we expect this to vary >>>>>>> across >>>>>>> mempools. >>>>>>> - Virtual Size = the maximum of virtual sizes calculated using >>>>>>> [BIP141 >>>>>>> virtual size][2] and sigop weight. [Implemented here in >>>>>>> Bitcoin Core][3]. >>>>>>> - Note that feerate is not necessarily based on the base fees >>>>>>> and serialized >>>>>>> size. >>>>>>> >>>>>>> * Fee-Bumping = user/wallet actions that take advantage of miner >>>>>>> incentives to >>>>>>> boost a transaction's candidacy for inclusion in a block, >>>>>>> including Child Pays >>>>>>> for Parent (CPFP) and [BIP125][12] Replace-by-Fee (RBF). Our >>>>>>> intention in >>>>>>> mempool policy is to recognize when the new transaction is more >>>>>>> economical to >>>>>>> mine than the original one(s) but not open DoS vectors, so there are >>>>>>> some >>>>>>> limitations. >>>>>>> >>>>>>> ### Policy >>>>>>> >>>>>>> The purpose of the mempool is to store the best (to be most >>>>>>> incentive-compatible >>>>>>> with miners, highest feerate) candidates for inclusion in a block. >>>>>>> Miners use >>>>>>> the mempool to build block templates. The mempool is also useful as >>>>>>> a cache for >>>>>>> boosting block relay and validation performance, aiding transaction >>>>>>> relay, and >>>>>>> generating feerate estimations. >>>>>>> >>>>>>> Ideally, all consensus-valid transactions paying reasonable fees >>>>>>> should make it >>>>>>> to miners through normal transaction relay, without any special >>>>>>> connectivity or >>>>>>> relationships with miners. On the other hand, nodes do not have >>>>>>> unlimited >>>>>>> resources, and a P2P network designed to let any honest node >>>>>>> broadcast their >>>>>>> transactions also exposes the transaction validation engine to DoS >>>>>>> attacks from >>>>>>> malicious peers. >>>>>>> >>>>>>> As such, for unconfirmed transactions we are considering for our >>>>>>> mempool, we >>>>>>> apply a set of validation rules in addition to consensus, primarily >>>>>>> to protect >>>>>>> us from resource exhaustion and aid our efforts to keep the highest >>>>>>> fee >>>>>>> transactions. We call this mempool _policy_: a set of (configurable, >>>>>>> node-specific) rules that transactions must abide by in order to be >>>>>>> accepted >>>>>>> into our mempool. Transaction "Standardness" rules and mempool >>>>>>> restrictions such >>>>>>> as "too-long-mempool-chain" are both examples of policy. >>>>>>> >>>>>>> ### Package Relay and Package Mempool Accept >>>>>>> >>>>>>> In transaction relay, we currently consider transactions one at a >>>>>>> time for >>>>>>> submission to the mempool. This creates a limitation in the node's >>>>>>> ability to >>>>>>> determine which transactions have the highest feerates, since we >>>>>>> cannot take >>>>>>> into account descendants (i.e. cannot use CPFP) until all the >>>>>>> transactions are >>>>>>> in the mempool. Similarly, we cannot use a transaction's descendants >>>>>>> when >>>>>>> considering it for RBF. When an individual transaction does not meet >>>>>>> the mempool >>>>>>> minimum feerate and the user isn't able to create a replacement >>>>>>> transaction >>>>>>> directly, it will not be accepted by mempools. >>>>>>> >>>>>>> This limitation presents a security issue for applications and users >>>>>>> relying on >>>>>>> time-sensitive transactions. For example, Lightning and other >>>>>>> protocols create >>>>>>> UTXOs with multiple spending paths, where one counterparty's >>>>>>> spending path opens >>>>>>> up after a timelock, and users are protected from cheating scenarios >>>>>>> as long as >>>>>>> they redeem on-chain in time. A key security assumption is that all >>>>>>> parties' >>>>>>> transactions will propagate and confirm in a timely manner. This >>>>>>> assumption can >>>>>>> be broken if fee-bumping does not work as intended. >>>>>>> >>>>>>> The end goal for Package Relay is to consider multiple transactions >>>>>>> at the same >>>>>>> time, e.g. a transaction with its high-fee child. This may help us >>>>>>> better >>>>>>> determine whether transactions should be accepted to our mempool, >>>>>>> especially if >>>>>>> they don't meet fee requirements individually or are better RBF >>>>>>> candidates as a >>>>>>> package. A combination of changes to mempool validation logic, >>>>>>> policy, and >>>>>>> transaction relay allows us to better propagate the transactions >>>>>>> with the >>>>>>> highest package feerates to miners, and makes fee-bumping tools more >>>>>>> powerful >>>>>>> for users. >>>>>>> >>>>>>> The "relay" part of Package Relay suggests P2P messaging changes, >>>>>>> but a large >>>>>>> part of the changes are in the mempool's package validation logic. >>>>>>> We call this >>>>>>> *Package Mempool Accept*. >>>>>>> >>>>>>> ### Previous Work >>>>>>> >>>>>>> * Given that mempool validation is DoS-sensitive and complex, it >>>>>>> would be >>>>>>> dangerous to haphazardly tack on package validation logic. Many >>>>>>> efforts have >>>>>>> been made to make mempool validation less opaque (see [#16400][4], >>>>>>> [#21062][5], >>>>>>> [#22675][6], [#22796][7]). >>>>>>> * [#20833][8] Added basic capabilities for package validation, test >>>>>>> accepts only >>>>>>> (no submission to mempool). >>>>>>> * [#21800][9] Implemented package ancestor/descendant limit checks >>>>>>> for arbitrary >>>>>>> packages. Still test accepts only. >>>>>>> * Previous package relay proposals (see [#16401][10], [#19621][11]). >>>>>>> >>>>>>> ### Existing Package Rules >>>>>>> >>>>>>> These are in master as introduced in [#20833][8] and [#21800][9]. >>>>>>> I'll consider >>>>>>> them as "given" in the rest of this document, though they can be >>>>>>> changed, since >>>>>>> package validation is test-accept only right now. >>>>>>> >>>>>>> 1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and >>>>>>> `MAX_PACKAGE_SIZE=101KvB` total size [8] >>>>>>> >>>>>>> *Rationale*: This is already enforced as mempool >>>>>>> ancestor/descendant limits. >>>>>>> Presumably, transactions in a package are all related, so exceeding >>>>>>> this limit >>>>>>> would mean that the package can either be split up or it wouldn't >>>>>>> pass this >>>>>>> mempool policy. >>>>>>> >>>>>>> 2. Packages must be topologically sorted: if any dependencies exist >>>>>>> between >>>>>>> transactions, parents must appear somewhere before children. [8] >>>>>>> >>>>>>> 3. A package cannot have conflicting transactions, i.e. none of them >>>>>>> can spend >>>>>>> the same inputs. This also means there cannot be duplicate >>>>>>> transactions. [8] >>>>>>> >>>>>>> 4. When packages are evaluated against ancestor/descendant limits in >>>>>>> a test >>>>>>> accept, the union of all of their descendants and ancestors is >>>>>>> considered. This >>>>>>> is essentially a "worst case" heuristic where every transaction in >>>>>>> the package >>>>>>> is treated as each other's ancestor and descendant. [8] >>>>>>> Packages for which ancestor/descendant limits are accurately >>>>>>> captured by this >>>>>>> heuristic: [19] >>>>>>> >>>>>>> There are also limitations such as the fact that CPFP carve out is >>>>>>> not applied >>>>>>> to package transactions. #20833 also disables RBF in package >>>>>>> validation; this >>>>>>> proposal overrides that to allow packages to use RBF. >>>>>>> >>>>>>> ## Proposed Changes >>>>>>> >>>>>>> The next step in the Package Mempool Accept project is to implement >>>>>>> submission >>>>>>> to mempool, initially through RPC only. This allows us to test the >>>>>>> submission >>>>>>> logic before exposing it on P2P. >>>>>>> >>>>>>> ### Summary >>>>>>> >>>>>>> - Packages may contain already-in-mempool transactions. >>>>>>> - Packages are 2 generations, Multi-Parent-1-Child. >>>>>>> - Fee-related checks use the package feerate. This means that >>>>>>> wallets can >>>>>>> create a package that utilizes CPFP. >>>>>>> - Parents are allowed to RBF mempool transactions with a set of >>>>>>> rules similar >>>>>>> to BIP125. This enables a combination of CPFP and RBF, where a >>>>>>> transaction's descendant fees pay for replacing mempool conflicts. >>>>>>> >>>>>>> There is a draft implementation in [#22290][1]. It is WIP, but >>>>>>> feedback is >>>>>>> always welcome. >>>>>>> >>>>>>> ### Details >>>>>>> >>>>>>> #### Packages May Contain Already-in-Mempool Transactions >>>>>>> >>>>>>> A package may contain transactions that are already in the mempool. >>>>>>> We remove >>>>>>> ("deduplicate") those transactions from the package for the purposes >>>>>>> of package >>>>>>> mempool acceptance. If a package is empty after deduplication, we do >>>>>>> nothing. >>>>>>> >>>>>>> *Rationale*: Mempools vary across the network. It's possible for a >>>>>>> parent to be >>>>>>> accepted to the mempool of a peer on its own due to differences in >>>>>>> policy and >>>>>>> fee market fluctuations. We should not reject or penalize the entire >>>>>>> package for >>>>>>> an individual transaction as that could be a censorship vector. >>>>>>> >>>>>>> #### Packages Are Multi-Parent-1-Child >>>>>>> >>>>>>> Only packages of a specific topology are permitted. Namely, a >>>>>>> package is exactly >>>>>>> 1 child with all of its unconfirmed parents. After deduplication, >>>>>>> the package >>>>>>> may be exactly the same, empty, 1 child, 1 child with just some of >>>>>>> its >>>>>>> unconfirmed parents, etc. Note that it's possible for the parents to >>>>>>> be indirect >>>>>>> descendants/ancestors of one another, or for parent and child to >>>>>>> share a parent, >>>>>>> so we cannot make any other topology assumptions. >>>>>>> >>>>>>> *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple >>>>>>> parents >>>>>>> makes it possible to fee-bump a batch of transactions. Restricting >>>>>>> packages to a >>>>>>> defined topology is also easier to reason about and simplifies the >>>>>>> validation >>>>>>> logic greatly. Multi-parent-1-child allows us to think of the >>>>>>> package as one big >>>>>>> transaction, where: >>>>>>> >>>>>>> - Inputs = all the inputs of parents + inputs of the child that come >>>>>>> from >>>>>>> confirmed UTXOs >>>>>>> - Outputs = all the outputs of the child + all outputs of the >>>>>>> parents that >>>>>>> aren't spent by other transactions in the package >>>>>>> >>>>>>> Examples of packages that follow this rule (variations of example A >>>>>>> show some >>>>>>> possibilities after deduplication): ![image][15] >>>>>>> >>>>>>> #### Fee-Related Checks Use Package Feerate >>>>>>> >>>>>>> Package Feerate = the total modified fees divided by the total >>>>>>> virtual size of >>>>>>> all transactions in the package. >>>>>>> >>>>>>> To meet the two feerate requirements of a mempool, i.e., the >>>>>>> pre-configured >>>>>>> minimum relay feerate (`minRelayTxFee`) and dynamic mempool minimum >>>>>>> feerate, the >>>>>>> total package feerate is used instead of the individual feerate. The >>>>>>> individual >>>>>>> transactions are allowed to be below feerate requirements if the >>>>>>> package meets >>>>>>> the feerate requirements. For example, the parent(s) in the package >>>>>>> can have 0 >>>>>>> fees but be paid for by the child. >>>>>>> >>>>>>> *Rationale*: This can be thought of as "CPFP within a package," >>>>>>> solving the >>>>>>> issue of a parent not meeting minimum fees on its own. This allows L2 >>>>>>> applications to adjust their fees at broadcast time instead of >>>>>>> overshooting or >>>>>>> risking getting stuck/pinned. >>>>>>> >>>>>>> We use the package feerate of the package *after deduplication*. >>>>>>> >>>>>>> *Rationale*: It would be incorrect to use the fees of transactions >>>>>>> that are >>>>>>> already in the mempool, as we do not want a transaction's fees to be >>>>>>> double-counted for both its individual RBF and package RBF. >>>>>>> >>>>>>> Examples F and G [14] show the same package, but P1 is submitted >>>>>>> individually before >>>>>>> the package in example G. In example F, we can see that the 300vB >>>>>>> package pays >>>>>>> an additional 200sat in fees, which is not enough to pay for its own >>>>>>> bandwidth >>>>>>> (BIP125#4). In example G, we can see that P1 pays enough to replace >>>>>>> M1, but >>>>>>> using P1's fees again during package submission would make it look >>>>>>> like a 300sat >>>>>>> increase for a 200vB package. Even including its fees and size would >>>>>>> not be >>>>>>> sufficient in this example, since the 300sat looks like enough for >>>>>>> the 300vB >>>>>>> package. The calculcation after deduplication is 100sat increase for >>>>>>> a package >>>>>>> of size 200vB, which correctly fails BIP125#4. Assume all >>>>>>> transactions have a >>>>>>> size of 100vB. >>>>>>> >>>>>>> #### Package RBF >>>>>>> >>>>>>> If a package meets feerate requirements as a package, the parents in >>>>>>> the >>>>>>> transaction are allowed to replace-by-fee mempool transactions. The >>>>>>> child cannot >>>>>>> replace mempool transactions. Multiple transactions can replace the >>>>>>> same >>>>>>> transaction, but in order to be valid, none of the transactions can >>>>>>> try to >>>>>>> replace an ancestor of another transaction in the same package >>>>>>> (which would thus >>>>>>> make its inputs unavailable). >>>>>>> >>>>>>> *Rationale*: Even if we are using package feerate, a package will >>>>>>> not propagate >>>>>>> as intended if RBF still requires each individual transaction to >>>>>>> meet the >>>>>>> feerate requirements. >>>>>>> >>>>>>> We use a set of rules slightly modified from BIP125 as follows: >>>>>>> >>>>>>> ##### Signaling (Rule #1) >>>>>>> >>>>>>> All mempool transactions to be replaced must signal replaceability. >>>>>>> >>>>>>> *Rationale*: Package RBF signaling logic should be the same for >>>>>>> package RBF and >>>>>>> single transaction acceptance. This would be updated if single >>>>>>> transaction >>>>>>> validation moves to full RBF. >>>>>>> >>>>>>> ##### New Unconfirmed Inputs (Rule #2) >>>>>>> >>>>>>> A package may include new unconfirmed inputs, but the ancestor >>>>>>> feerate of the >>>>>>> child must be at least as high as the ancestor feerates of every >>>>>>> transaction >>>>>>> being replaced. This is contrary to BIP125#2, which states "The >>>>>>> replacement >>>>>>> transaction may only include an unconfirmed input if that input was >>>>>>> included in >>>>>>> one of the original transactions. (An unconfirmed input spends an >>>>>>> output from a >>>>>>> currently-unconfirmed transaction.)" >>>>>>> >>>>>>> *Rationale*: The purpose of BIP125#2 is to ensure that the >>>>>>> replacement >>>>>>> transaction has a higher ancestor score than the original >>>>>>> transaction(s) (see >>>>>>> [comment][13]). Example H [16] shows how adding a new unconfirmed >>>>>>> input can lower the >>>>>>> ancestor score of the replacement transaction. P1 is trying to >>>>>>> replace M1, and >>>>>>> spends an unconfirmed output of M2. P1 pays 800sat, M1 pays 600sat, >>>>>>> and M2 pays >>>>>>> 100sat. Assume all transactions have a size of 100vB. While, in >>>>>>> isolation, P1 >>>>>>> looks like a better mining candidate than M1, it must be mined with >>>>>>> M2, so its >>>>>>> ancestor feerate is actually 4.5sat/vB. This is lower than M1's >>>>>>> ancestor >>>>>>> feerate, which is 6sat/vB. >>>>>>> >>>>>>> In package RBF, the rule analogous to BIP125#2 would be "none of the >>>>>>> transactions in the package can spend new unconfirmed inputs." >>>>>>> Example J [17] shows >>>>>>> why, if any of the package transactions have ancestors, package >>>>>>> feerate is no >>>>>>> longer accurate. Even though M2 and M3 are not ancestors of P1 >>>>>>> (which is the >>>>>>> replacement transaction in an RBF), we're actually interested in the >>>>>>> entire >>>>>>> package. A miner should mine M1 which is 5sat/vB instead of M2, M3, >>>>>>> P1, P2, and >>>>>>> P3, which is only 4sat/vB. The Package RBF rule cannot be loosened >>>>>>> to only allow >>>>>>> the child to have new unconfirmed inputs, either, because it can >>>>>>> still cause us >>>>>>> to overestimate the package's ancestor score. >>>>>>> >>>>>>> However, enforcing a rule analogous to BIP125#2 would not only make >>>>>>> Package RBF >>>>>>> less useful, but would also break Package RBF for packages with >>>>>>> parents already >>>>>>> in the mempool: if a package parent has already been submitted, it >>>>>>> would look >>>>>>> like the child is spending a "new" unconfirmed input. In example K >>>>>>> [18], we're >>>>>>> looking to replace M1 with the entire package including P1, P2, and >>>>>>> P3. We must >>>>>>> consider the case where one of the parents is already in the mempool >>>>>>> (in this >>>>>>> case, P2), which means we must allow P3 to have new unconfirmed >>>>>>> inputs. However, >>>>>>> M2 lowers the ancestor score of P3 to 4.3sat/vB, so we should not >>>>>>> replace M1 >>>>>>> with this package. >>>>>>> >>>>>>> Thus, the package RBF rule regarding new unconfirmed inputs is less >>>>>>> strict than >>>>>>> BIP125#2. However, we still achieve the same goal of requiring the >>>>>>> replacement >>>>>>> transactions to have a ancestor score at least as high as the >>>>>>> original ones. As >>>>>>> a result, the entire package is required to be a higher feerate >>>>>>> mining candidate >>>>>>> than each of the replaced transactions. >>>>>>> >>>>>>> Another note: the [comment][13] above the BIP125#2 code in the >>>>>>> original RBF >>>>>>> implementation suggests that the rule was intended to be temporary. >>>>>>> >>>>>>> ##### Absolute Fee (Rule #3) >>>>>>> >>>>>>> The package must increase the absolute fee of the mempool, i.e. the >>>>>>> total fees >>>>>>> of the package must be higher than the absolute fees of the mempool >>>>>>> transactions >>>>>>> it replaces. Combined with the CPFP rule above, this differs from >>>>>>> BIP125 Rule #3 >>>>>>> - an individual transaction in the package may have lower fees than >>>>>>> the >>>>>>> transaction(s) it is replacing. In fact, it may have 0 fees, and >>>>>>> the child >>>>>>> pays for RBF. >>>>>>> >>>>>>> ##### Feerate (Rule #4) >>>>>>> >>>>>>> The package must pay for its own bandwidth; the package feerate must >>>>>>> be higher >>>>>>> than the replaced transactions by at least minimum relay feerate >>>>>>> (`incrementalRelayFee`). Combined with the CPFP rule above, this >>>>>>> differs from >>>>>>> BIP125 Rule #4 - an individual transaction in the package can have a >>>>>>> lower >>>>>>> feerate than the transaction(s) it is replacing. In fact, it may >>>>>>> have 0 fees, >>>>>>> and the child pays for RBF. >>>>>>> >>>>>>> ##### Total Number of Replaced Transactions (Rule #5) >>>>>>> >>>>>>> The package cannot replace more than 100 mempool transactions. This >>>>>>> is identical >>>>>>> to BIP125 Rule #5. >>>>>>> >>>>>>> ### Expected FAQs >>>>>>> >>>>>>> 1. Is it possible for only some of the package to make it into the >>>>>>> mempool? >>>>>>> >>>>>>> Yes, it is. However, since we evict transactions from the mempool >>>>>>> by >>>>>>> descendant score and the package child is supposed to be sponsoring >>>>>>> the fees of >>>>>>> its parents, the most common scenario would be all-or-nothing. This >>>>>>> is >>>>>>> incentive-compatible. In fact, to be conservative, package >>>>>>> validation should >>>>>>> begin by trying to submit all of the transactions individually, and >>>>>>> only use the >>>>>>> package mempool acceptance logic if the parents fail due to low >>>>>>> feerate. >>>>>>> >>>>>>> 2. Should we allow packages to contain already-confirmed >>>>>>> transactions? >>>>>>> >>>>>>> No, for practical reasons. In mempool validation, we actually >>>>>>> aren't able to >>>>>>> tell with 100% confidence if we are looking at a transaction that >>>>>>> has already >>>>>>> confirmed, because we look up inputs using a UTXO set. If we have >>>>>>> historical >>>>>>> block data, it's possible to look for it, but this is inefficient, >>>>>>> not always >>>>>>> possible for pruning nodes, and unnecessary because we're not going >>>>>>> to do >>>>>>> anything with the transaction anyway. As such, we already have the >>>>>>> expectation >>>>>>> that transaction relay is somewhat "stateful" i.e. nobody should be >>>>>>> relaying >>>>>>> transactions that have already been confirmed. Similarly, we >>>>>>> shouldn't be >>>>>>> relaying packages that contain already-confirmed transactions. >>>>>>> >>>>>>> [1]: https://github.com/bitcoin/bitcoin/pull/22290 >>>>>>> [2]: >>>>>>> https://github.com/bitcoin/bips/blob/1f0b563738199ca60d32b4ba779797fc97d040fe/bip-0141.mediawiki#transaction-size-calculations >>>>>>> [3]: >>>>>>> https://github.com/bitcoin/bitcoin/blob/94f83534e4b771944af7d9ed0f40746f392eb75e/src/policy/policy.cpp#L282 >>>>>>> [4]: https://github.com/bitcoin/bitcoin/pull/16400 >>>>>>> [5]: https://github.com/bitcoin/bitcoin/pull/21062 >>>>>>> [6]: https://github.com/bitcoin/bitcoin/pull/22675 >>>>>>> [7]: https://github.com/bitcoin/bitcoin/pull/22796 >>>>>>> [8]: https://github.com/bitcoin/bitcoin/pull/20833 >>>>>>> [9]: https://github.com/bitcoin/bitcoin/pull/21800 >>>>>>> [10]: https://github.com/bitcoin/bitcoin/pull/16401 >>>>>>> [11]: https://github.com/bitcoin/bitcoin/pull/19621 >>>>>>> [12]: https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki >>>>>>> [13]: >>>>>>> https://github.com/bitcoin/bitcoin/pull/6871/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R1101-R1104 >>>>>>> [14]: >>>>>>> https://user-images.githubusercontent.com/25183001/133567078-075a971c-0619-4339-9168-b41fd2b90c28.png >>>>>>> [15]: >>>>>>> https://user-images.githubusercontent.com/25183001/132856734-fc17da75-f875-44bb-b954-cb7a1725cc0d.png >>>>>>> [16]: >>>>>>> https://user-images.githubusercontent.com/25183001/133567347-a3e2e4a8-ae9c-49f8-abb9-81e8e0aba224.png >>>>>>> [17]: >>>>>>> https://user-images.githubusercontent.com/25183001/133567370-21566d0e-36c8-4831-b1a8-706634540af3.png >>>>>>> [18]: >>>>>>> https://user-images.githubusercontent.com/25183001/133567444-bfff1142-439f-4547-800a-2ba2b0242bcb.png >>>>>>> [19]: >>>>>>> https://user-images.githubusercontent.com/25183001/133456219-0bb447cb-dcb4-4a31-b9c1-7d86205b68bc.png >>>>>>> [20]: >>>>>>> https://user-images.githubusercontent.com/25183001/132857787-7b7c6f56-af96-44c8-8d78-983719888c19.png >>>>>>> _______________________________________________ >>>>>>> bitcoin-dev mailing list >>>>>>> bitcoin-dev@lists.linuxfoundation.org >>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>>>>> >>>>>> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >