Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions

From: Alex Morcos <morcos@gmail•com>
To: Danny Thorpe <danny.thorpe@gmail•com>
Cc: Bitcoin Dev <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions
Date: Mon, 5 Oct 2015 16:02:40 -0400	[thread overview]
Message-ID: <CAPWm=eWfnuW98aLhaULCM5BAXThOz1E_APLC+Yd2uOJ4-B4PsA@mail.gmail.com> (raw)
In-Reply-To: <CAJN5wHXJkFefminbnY6O93PUkQKCyLpvM57TCp+4q6p6CEzhzg@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 8316 bytes --]

Yes, total number of dependent transactions regardless of chain depth.

A descendant package means all the transactions that can not be included in
a block before the transaction in question.

An ancestor package means all the transactions that are required to be
included in a block before the transaction in question can be.

On Mon, Oct 5, 2015 at 2:51 PM, Danny Thorpe <danny.thorpe@gmail•com> wrote:

> What does "package" mean here?
>
> When you say 25 txs, does that mean maximum linked chain depth, or total
> number of dependent transactions regardless of chain depth?
>
> Thanks,
> -Danny
>
>
>
> On Mon, Oct 5, 2015 at 11:45 AM, Alex Morcos via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> I'd like to propose updates to the new policy limits on unconfirmed
>> transaction chains.
>>
>> The existing limits in master and scheduled for release in 0.12 are:
>> Ancestor packages = 100 txs and 900kb total size
>> Descendant packages = 1000 txs and 2500kb total size
>>
>> Before 0.12 is released I would like to propose a significant reduction
>> in these limits. In the course of analyzing algorithms for mempool
>> limiting, it became clear that large packages of unconfirmed transactions
>> were the primary vector for mempool clogging or relay fee boosting attacks.
>> Feedback from the initial proposed limits was that they were too generous
>> anyway.
>>
>> The proposed new limits are:
>> Ancestor packages = 25 txs and 100kb total size
>> Descendant packages = 25 txs and 100kb total size
>>
>> Based on historical transaction data, the most restrictive of these
>> limits is the 25 transaction count on descendant packages. Over the period
>> of April and May of this year (before stress tests), 5.8% of transactions
>> would have violated this limit alone. Applying all the limits together
>> would have affected 6.1% of transactions.
>>
>> Please keep in mind these are policy limits that affect transactions
>> which depend on other unconfirmed transactions only. They are not a change
>> to consensus rules and do not affect how many chained txs a valid block may
>> contain. Furthermore, any transaction that was unable to be relayed due to
>> these limits need only wait for some of its unconfirmed ancestors to be
>> included in a block and then it could be successfully broadcast. This is
>> unlikely to affect the total time from creation to inclusion in a block.
>> Finally, these limits are command line arguments that can easily be changed
>> on an individual node basis in Bitcoin Core.
>>
>> Please give your feedback if you know of legitimate use cases that would
>> be hindered by these limits.
>>
>> Thanks,
>> Alex
>>
>> On Mon, Sep 21, 2015 at 11:02 AM, Alex Morcos <morcos@gmail•com> wrote:
>>
>>> Thanks for everyone's review.  These policy changes have been merged in
>>> to master in 6654 <https://github.com/bitcoin/bitcoin/pull/6654>, which
>>> just implements these limits and no mempool limiting yet.  The default
>>> ancestor package size limit is 900kb not 1MB.
>>>
>>> Yes I think these limits are generous, but they were designed to be as
>>> generous as was computationally feasible so they were unobjectionable
>>> (since the existing policy was no limits).  This does not preclude future
>>> changes to policy that would reduce these limits.
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 21, 2015 at 3:52 PM, Danny Thorpe <danny.thorpe@gmail•com>
>>> wrote:
>>>
>>>> The limits Alex proposed are generous (bordering on obscene!), but
>>>> dropping that down to allowing only two levels of chained unconfirmed
>>>> transactions is too tight.
>>>>
>>>> Use case: Brokered asset transfers may require sets of transactions
>>>> with a dependency tree depth of 3 to be published together. ( N seller txs,
>>>> 1 broker bridge tx, M buyer txs )
>>>>
>>>> If the originally proposed depth limit of 100 does not provide a
>>>> sufficient cap on memory consumption or loop/recursion depth, a depth limit
>>>> of 10 would provide plenty of headroom for this 3 level use case and
>>>> similar patterns.
>>>>
>>>> -Danny
>>>>
>>>> On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev <
>>>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>>>
>>>>> I dont see any problem with such limits. Though, hell, if you limited
>>>>> entire tx dependency trees (ie transactions and all required
>>>>> unconfirmed
>>>>> transactions for them) to something like 10 txn, maximum two levels
>>>>> deep, I also wouldnt have a problem.
>>>>>
>>>>> Matt
>>>>>
>>>>> On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote:
>>>>> > Hi everyone,
>>>>> >
>>>>> >
>>>>> > I'd like to propose a new set of requirements as a policy on when to
>>>>> > accept new transactions into the mempool and relay them.  This policy
>>>>> > would affect transactions which have as inputs other transactions
>>>>> which
>>>>> > are not yet confirmed in the blockchain.
>>>>> >
>>>>> > The motivation for this policy is 6470
>>>>> > <https://github.com/bitcoin/bitcoin/pull/6470> which aims to limit
>>>>> the
>>>>> > size of a mempool.  As discussed in that pull
>>>>> > <https://github.com/bitcoin/bitcoin/pull/6470#issuecomment-125324736
>>>>> >,
>>>>> > once the mempool is full a new transaction must be able to pay not
>>>>> only
>>>>> > for the transaction it would evict, but any dependent transactions
>>>>> that
>>>>> > would be removed from the mempool as well.  In order to make sure
>>>>> this
>>>>> > is always feasible, I'm proposing 4 new policy limits.
>>>>> >
>>>>> > All limits are command line configurable.
>>>>> >
>>>>> > The first two limits are required to make sure no chain of
>>>>> transactions
>>>>> > will be too large for the eviction code to handle:
>>>>> >
>>>>> > Max number of descendant txs : No transaction shall be accepted if it
>>>>> > would cause another transaction in the mempool to have too many
>>>>> > descendant transactions (all of which would have to be evicted if the
>>>>> > ancestor transaction was evicted).  Default: 1000
>>>>> >
>>>>> > Max descendant size : No transaction shall be accepted if it would
>>>>> cause
>>>>> > another transaction in the mempool to have the total size of all its
>>>>> > descendant transactions be too great.  Default : maxmempool / 200
>>>>> =  2.5MB
>>>>> >
>>>>> > The third limit is required to make sure calculating the state
>>>>> required
>>>>> > for sorting and limiting the mempool and enforcing the first 2
>>>>> limits is
>>>>> > computationally feasible:
>>>>> >
>>>>> > Max number of ancestor txs:  No transaction shall be accepted if it
>>>>> has
>>>>> > too many ancestor transactions which are not yet confirmed (ie, in
>>>>> the
>>>>> > mempool). Default: 100
>>>>> >
>>>>> > The fourth limit is required to maintain the pre existing policy goal
>>>>> > that all transactions in the mempool should be mineable in the next
>>>>> block.
>>>>> >
>>>>> > Max ancestor size: No transaction shall be accepted if the total
>>>>> size of
>>>>> > all its unconfirmed ancestor transactions is too large.  Default: 1MB
>>>>> >
>>>>> > (All limits include the transaction itself.)
>>>>> >
>>>>> > For reference, these limits would have affected less than 2% of
>>>>> > transactions entering the mempool in April or May of this year.
>>>>> During
>>>>> > the period of 7/6 through 7/14, while the network was under stress
>>>>> test,
>>>>> > as many as 25% of the transactions would have been affected.
>>>>> >
>>>>> > The code to implement the descendant package tracking and new policy
>>>>> > limits can be found in 6557
>>>>> > <https://github.com/bitcoin/bitcoin/pull/6557> which is built off
>>>>> of 6470.
>>>>> >
>>>>> > Thanks,
>>>>> > Alex
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > bitcoin-dev mailing list
>>>>> > bitcoin-dev@lists•linuxfoundation.org
>>>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>> >
>>>>> _______________________________________________
>>>>> bitcoin-dev mailing list
>>>>> bitcoin-dev@lists•linuxfoundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>>
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 11250 bytes --]