Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions

From: Matt Corallo <lf-lists@mattcorallo•com>
To: Alex Morcos <morcos@gmail•com>,
	Alex Morcos via bitcoin-dev
	<bitcoin-dev@lists•linuxfoundation.org>,
	Danny Thorpe <danny.thorpe@gmail•com>
Cc: Bitcoin Dev <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions
Date: Thu, 08 Oct 2015 03:33:07 +0000	[thread overview]
Message-ID: <272A5405-676C-4CC9-8E17-F3149DDB18AA@mattcorallo.com> (raw)
In-Reply-To: <CAPWm=eWfnuW98aLhaULCM5BAXThOz1E_APLC+Yd2uOJ4-B4PsA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 9094 bytes --]

There is a PR up for this change at https://github.com/bitcoin/bitcoin/pull/6771, which is getting some discussion for those following along.

On October 5, 2015 1:02:40 PM PDT, Alex Morcos via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
>Yes, total number of dependent transactions regardless of chain depth.
>
>A descendant package means all the transactions that can not be
>included in
>a block before the transaction in question.
>
>An ancestor package means all the transactions that are required to be
>included in a block before the transaction in question can be.
>
>
>
>
>On Mon, Oct 5, 2015 at 2:51 PM, Danny Thorpe <danny.thorpe@gmail•com>
>wrote:
>
>> What does "package" mean here?
>>
>> When you say 25 txs, does that mean maximum linked chain depth, or
>total
>> number of dependent transactions regardless of chain depth?
>>
>> Thanks,
>> -Danny
>>
>>
>>
>> On Mon, Oct 5, 2015 at 11:45 AM, Alex Morcos via bitcoin-dev <
>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>
>>> I'd like to propose updates to the new policy limits on unconfirmed
>>> transaction chains.
>>>
>>> The existing limits in master and scheduled for release in 0.12 are:
>>> Ancestor packages = 100 txs and 900kb total size
>>> Descendant packages = 1000 txs and 2500kb total size
>>>
>>> Before 0.12 is released I would like to propose a significant
>reduction
>>> in these limits. In the course of analyzing algorithms for mempool
>>> limiting, it became clear that large packages of unconfirmed
>transactions
>>> were the primary vector for mempool clogging or relay fee boosting
>attacks.
>>> Feedback from the initial proposed limits was that they were too
>generous
>>> anyway.
>>>
>>> The proposed new limits are:
>>> Ancestor packages = 25 txs and 100kb total size
>>> Descendant packages = 25 txs and 100kb total size
>>>
>>> Based on historical transaction data, the most restrictive of these
>>> limits is the 25 transaction count on descendant packages. Over the
>period
>>> of April and May of this year (before stress tests), 5.8% of
>transactions
>>> would have violated this limit alone. Applying all the limits
>together
>>> would have affected 6.1% of transactions.
>>>
>>> Please keep in mind these are policy limits that affect transactions
>>> which depend on other unconfirmed transactions only. They are not a
>change
>>> to consensus rules and do not affect how many chained txs a valid
>block may
>>> contain. Furthermore, any transaction that was unable to be relayed
>due to
>>> these limits need only wait for some of its unconfirmed ancestors to
>be
>>> included in a block and then it could be successfully broadcast.
>This is
>>> unlikely to affect the total time from creation to inclusion in a
>block.
>>> Finally, these limits are command line arguments that can easily be
>changed
>>> on an individual node basis in Bitcoin Core.
>>>
>>> Please give your feedback if you know of legitimate use cases that
>would
>>> be hindered by these limits.
>>>
>>> Thanks,
>>> Alex
>>>
>>> On Mon, Sep 21, 2015 at 11:02 AM, Alex Morcos <morcos@gmail•com>
>wrote:
>>>
>>>> Thanks for everyone's review.  These policy changes have been
>merged in
>>>> to master in 6654 <https://github.com/bitcoin/bitcoin/pull/6654>,
>which
>>>> just implements these limits and no mempool limiting yet.  The
>default
>>>> ancestor package size limit is 900kb not 1MB.
>>>>
>>>> Yes I think these limits are generous, but they were designed to be
>as
>>>> generous as was computationally feasible so they were
>unobjectionable
>>>> (since the existing policy was no limits).  This does not preclude
>future
>>>> changes to policy that would reduce these limits.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 21, 2015 at 3:52 PM, Danny Thorpe
><danny.thorpe@gmail•com>
>>>> wrote:
>>>>
>>>>> The limits Alex proposed are generous (bordering on obscene!), but
>>>>> dropping that down to allowing only two levels of chained
>unconfirmed
>>>>> transactions is too tight.
>>>>>
>>>>> Use case: Brokered asset transfers may require sets of
>transactions
>>>>> with a dependency tree depth of 3 to be published together. ( N
>seller txs,
>>>>> 1 broker bridge tx, M buyer txs )
>>>>>
>>>>> If the originally proposed depth limit of 100 does not provide a
>>>>> sufficient cap on memory consumption or loop/recursion depth, a
>depth limit
>>>>> of 10 would provide plenty of headroom for this 3 level use case
>and
>>>>> similar patterns.
>>>>>
>>>>> -Danny
>>>>>
>>>>> On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev <
>>>>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>>>>
>>>>>> I dont see any problem with such limits. Though, hell, if you
>limited
>>>>>> entire tx dependency trees (ie transactions and all required
>>>>>> unconfirmed
>>>>>> transactions for them) to something like 10 txn, maximum two
>levels
>>>>>> deep, I also wouldnt have a problem.
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>> On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote:
>>>>>> > Hi everyone,
>>>>>> >
>>>>>> >
>>>>>> > I'd like to propose a new set of requirements as a policy on
>when to
>>>>>> > accept new transactions into the mempool and relay them.  This
>policy
>>>>>> > would affect transactions which have as inputs other
>transactions
>>>>>> which
>>>>>> > are not yet confirmed in the blockchain.
>>>>>> >
>>>>>> > The motivation for this policy is 6470
>>>>>> > <https://github.com/bitcoin/bitcoin/pull/6470> which aims to
>limit
>>>>>> the
>>>>>> > size of a mempool.  As discussed in that pull
>>>>>> >
><https://github.com/bitcoin/bitcoin/pull/6470#issuecomment-125324736
>>>>>> >,
>>>>>> > once the mempool is full a new transaction must be able to pay
>not
>>>>>> only
>>>>>> > for the transaction it would evict, but any dependent
>transactions
>>>>>> that
>>>>>> > would be removed from the mempool as well.  In order to make
>sure
>>>>>> this
>>>>>> > is always feasible, I'm proposing 4 new policy limits.
>>>>>> >
>>>>>> > All limits are command line configurable.
>>>>>> >
>>>>>> > The first two limits are required to make sure no chain of
>>>>>> transactions
>>>>>> > will be too large for the eviction code to handle:
>>>>>> >
>>>>>> > Max number of descendant txs : No transaction shall be accepted
>if it
>>>>>> > would cause another transaction in the mempool to have too many
>>>>>> > descendant transactions (all of which would have to be evicted
>if the
>>>>>> > ancestor transaction was evicted).  Default: 1000
>>>>>> >
>>>>>> > Max descendant size : No transaction shall be accepted if it
>would
>>>>>> cause
>>>>>> > another transaction in the mempool to have the total size of
>all its
>>>>>> > descendant transactions be too great.  Default : maxmempool /
>200
>>>>>> =  2.5MB
>>>>>> >
>>>>>> > The third limit is required to make sure calculating the state
>>>>>> required
>>>>>> > for sorting and limiting the mempool and enforcing the first 2
>>>>>> limits is
>>>>>> > computationally feasible:
>>>>>> >
>>>>>> > Max number of ancestor txs:  No transaction shall be accepted
>if it
>>>>>> has
>>>>>> > too many ancestor transactions which are not yet confirmed (ie,
>in
>>>>>> the
>>>>>> > mempool). Default: 100
>>>>>> >
>>>>>> > The fourth limit is required to maintain the pre existing
>policy goal
>>>>>> > that all transactions in the mempool should be mineable in the
>next
>>>>>> block.
>>>>>> >
>>>>>> > Max ancestor size: No transaction shall be accepted if the
>total
>>>>>> size of
>>>>>> > all its unconfirmed ancestor transactions is too large. 
>Default: 1MB
>>>>>> >
>>>>>> > (All limits include the transaction itself.)
>>>>>> >
>>>>>> > For reference, these limits would have affected less than 2% of
>>>>>> > transactions entering the mempool in April or May of this year.
>>>>>> During
>>>>>> > the period of 7/6 through 7/14, while the network was under
>stress
>>>>>> test,
>>>>>> > as many as 25% of the transactions would have been affected.
>>>>>> >
>>>>>> > The code to implement the descendant package tracking and new
>policy
>>>>>> > limits can be found in 6557
>>>>>> > <https://github.com/bitcoin/bitcoin/pull/6557> which is built
>off
>>>>>> of 6470.
>>>>>> >
>>>>>> > Thanks,
>>>>>> > Alex
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > bitcoin-dev mailing list
>>>>>> > bitcoin-dev@lists•linuxfoundation.org
>>>>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>>> >
>>>>>> _______________________________________________
>>>>>> bitcoin-dev mailing list
>>>>>> bitcoin-dev@lists•linuxfoundation.org
>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists•linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists•linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 12351 bytes --]