Subject: Re: [bitcoindev] Mandatory Inclusion of Old Transactions in Blocks

From: developer <[email-masked]>
Date: Sun, 29 Dec 2024 08:35:01 -0800 (PST)

I would like to explain the concept better. I am not very technical, but I 
would like to leave a contribution by talking about this topic that could 
become reality. In a context of censorship and control, even a seemingly 
infallible system like Bitcoin could be subject to impositions by strong 
powers. The goal is to make the entire structure more robust, proposing a 
desirable improvement for the future. How invasive it can be is to be 
tested, but the idea is to exploit everything that already exists without 
introducing new things.

Assumption ==================

I start from the basic assumption that a wallet, when signing and sending 
the transaction, inserts the sending timestamp in the "nLockTime" field 
(usually set to 0).

“nLockTime” is designed to indicate the earliest time a transaction is 
eligible to be included in a block (USUALLY USED FOR THEFT) and is only 
considered if the nSequence field is less than 0xFFFFFFFF (for example, 
0xFFFFFFFE). Today I imagine that most transactions with RBF already set 
this field in a way to allow users to create a transaction that can be 
replaced by a new transaction with a higher fee if necessary.

Using “nLockTime” is therefore possible and legal.

Inserting the timestamp would not be an improper use at all, but only a 
confirmation of the will to insert the block starting from the timestamp 
(immediately).

Analysis ==================

Getting into practice, the ordering of transactions occurs mainly in the 
data structure called indexed_transaction_set, defined using Boost 
MultiIndex. The mempool uses an index based on the fee rate to classify the 
transactions. The CompareTxMemPoolEntryByFee function compares transactions 
based on the fee rate, which is the ratio of the total transaction fee 
(GetFee()) to the size of the transaction (GetTxSize()). This structure 
allows the mempool to maintain a natural order based on fees per byte 
(sat/vByte).

The txmempool.cpp file implements the main functions for adding and 
removing transactions from the mempool, maintaining the order. The size_t 
CTxMemPool::TrimToSize(size_t sizelimit) function adds an unchecked 
transaction to the mempool and updates the indexes (including the fee rate 
index).

(Removing Transactions)

Transactions can be removed from the mempool if:

• They are included in a block.

• They exceed the space limits of the mempool.

• They are replaced by other transactions with a higher fee rate 
(Replace-by-Fee, or RBF).

When the mempool reaches the memory limit (mempoollimit), transactions with 
lower fee rates are removed to make room for those with higher fee rates. 
The TrimToSize() function in txmempool.cpp handles this logic, keeping only 
the transactions with the highest fee rates.

Intervention ==================

Integrating nLockTime-based sorting along with fee rate into the Boost 
MultiIndex indexed_transaction_set structure in the mempool would require 
some fine-grained, but not overly invasive, changes. The complexity depends 
on how you want to balance the two criteria (fee rate and nLockTime) and 
the philosophy with which these rules should interact. Here is one possible 
version:

1. Updating the data structure

The indexed_transaction_set uses Boost MultiIndex, allowing you to define 
multiple indexes for ranking. Currently, one of the main indexes is the 
CompareTxMemPoolEntryByFee for fee rate. To add nLockTime support, we need 
to implement a new comparator that takes both the fee rate and the 
nLockTime into account.

To implement the logic that removes the transactions with the lowest fee 
rate and the most recent timestamp from the mempool (so that older 
transactions are taken into account), we need to change the eviction 
criterion in the TrimToSize function.

2. Updating the TrimToSize function

In the txmempool.cpp file, I modify the TrimToSize function to use a new 
sorting that considers the fee rate first (in ascending order) and then the 
nLockTime (in descending order, to give importance to older transactions).

Example of a hypothetical implementation of the TrimToSize function:

size_t CTxMemPool::TrimToSize(size_t sizelimit) {

LOCK(cs);

while (DynamicMemoryUsage() > sizelimit) {

// Use the primary index to get the transaction with the lowest fee rate

// and the most recent nLockTime

auto it = mapTx.get<0>().begin(); // The order is based on the new 
comparator

// Remove the selected transaction

removeUnchecked(mapTx.project<0>(it));

}

return DynamicMemoryUsage();

}

This code assumes that the sort order of mapTx has already been updated to 
reflect the desired order.

3. Modify the CompareTxMemPoolEntryByFeeAndLockTime comparator

To ensure that the transaction selected by mapTx.get<0>() is the one with 
the lowest fee rate and the most recent timestamp, we update the comparator:

struct CompareTxMemPoolEntryByFeeAndLockTime {

bool operator()(const CTxMemPoolEntry& a, const CTxMemPoolEntry& b) const {

// First priority: Fee rate (increasing, to remove the lowest fees first)

if (a.GetFeeRate() != b.GetFeeRate()) {

return a.GetFeeRate() < b.GetFeeRate();

}

// Second priority: nLockTime (decreasing, to remove the most recent 
timestamps first)

return a.GetTx().nLockTime > b.GetTx().nLockTime;

}

};

With this comparator:

1. Transactions with the lowest fee rate are selected first.

2. For the same fee rate, those with the most recent nLockTime (highest 
timestamp) are selected.

Conclusion

With these changes:

• The mempool removes transactions with the lowest fee rate first.

• Among transactions with the same fee rate, those with the most recent 
nLockTime (highest timestamp) are removed first.

• This ensures that older transactions, even with low fees, have a higher 
chance of being included in a block, reducing the risk of stagnating in the 
mempool.

This could be a first approach, it would then be possible to evaluate a 
greater weight to the timestamp in order to introduce at least a part of 
stagnant transactions in new blocks. It is not a question of forcing the 
mempool to implement the changes since these are ethical and cooperation 
issues. Nature is cooperative, there is no free market.
Il giorno sabato 28 dicembre 2024 alle 19:50:43 UTC+1 Ethan Heilman ha 
scritto:

> You say:
>
> > Bitcoin network nodes will validate blocks only if they contain the 
> required percentage of old transactions. If a block fails to meet this 
> criterion, it will be deemed invalid and rejected by the network.
>
> This requires that network nodes reach consensus on what the oldest
> transactions are. This is the technical problem you need to solve to
> make your proposal practical, but I don't see that as a bad thing as
> it is a very interesting problem. If you can solve this problem, you
> can probably also solve the problem of how to enforce that Bitcoin
> miners only include the transactions with the highest fee rate.
> Enforcing the highest fee rate at the consensus level may be an
> effective tool against MEVil.
>
> This does seem like a hard problem to solve, because reaching
> consensus on transactions in mempool is very similar to what bitcoin
> blocks are already trying to achieve. Perhaps there is a more creative
> solution. It is worth looking at but it doesn't seem ready for a BIP.
>
> On Sat, Dec 28, 2024 at 11:26 AM Michael Cassano <[email-masked]> 
> wrote:
> >
> > I reject the premise of this proposed BIP. Mandating miners to include a 
> specific percentage of transactions based on age fundamentally undermines 
> the core principles of Bitcoin: decentralization, voluntary participation, 
> and free market dynamics.
> >
> > Bitcoin thrives because of its permissionless, free-market system. 
> Miners are incentivized to prioritize transactions based on fees and 
> network conditions, not arbitrary mandates. Imposing a rule like this 
> introduces central planning into what is a decentralized system.
> >
> > The proposal claims to fight centralization, but will likely backfire. 
> Mandates like this add operational complexity and reduce efficiency for 
> miners. Smaller miners, who are already operating on thin margins, will be 
> disproportionately impacted, driving them out of the market and further 
> centralizing mining power. If censorship-resistant mining is valuable, let 
> the free market reward those who provide it. If there’s demand for miners 
> to include old or low-fee transactions, let someone build tools and pools 
> that prioritize this voluntarily. Solutions shall arise from innovation, 
> not coercion.
> >
> > Best regards,
> > Mike
> >
> > On Sat, Dec 28, 2024 at 8:58 AM developer <[email-masked]> wrote:
> >>
> >> Status: Draft
> >> Type: Standards Track
> >> Created: December 27, 2024
> >> Abstract
> >>
> >> This proposal mandates miners to include at least 0.1% of transactions 
> in their blocks from the oldest transactions by date, even if they have low 
> fees. This mechanism helps prevent mining centralization and censorship, 
> encouraging miners not to exclude certain transactions.
> >> Motivation
> >>
> >> The increasing centralization of Bitcoin mining and potential 
> regulations that may require miners to censor or exclude certain 
> transactions pose a threat to the Bitcoin network. Mandating the inclusion 
> of a small percentage of old transactions, even with low fees, ensures that 
> no single miner can censor block contents without sacrificing their own 
> rewards.
> >> Specification
> >>
> >> Mandatory Inclusion of Old even if with Low-Fee Transactions
> >> Each miner is required to include at least 0.1% of the total 
> transactions in a block from the oldest transactions in the mempool, even 
> if their fees are below the current market average.
> >> These transactions must be added to blocks regardless of their fees, 
> prioritizing their age.
> >>
> >> Block Validation
> >> Bitcoin network nodes will validate blocks only if they contain the 
> required percentage of old transactions.
> >> If a block fails to meet this criterion, it will be deemed invalid and 
> rejected by the network.
> >>
> >> Incentives
> >> Miners are incentivized to include these transactions to ensure their 
> blocks are valid and to avoid losing block rewards.
> >>
> >> Advantages
> >>
> >> Censorship Resistance: Miners cannot censor transactions without 
> forfeiting their rewards.
> >> Greater Inclusivity: Old and low-fee transactions are assured of being 
> confirmed.
> >> Decentralization Prevention: Reducing the potential for centralized 
> censorship keeps the Bitcoin network decentralized.
> >>
> >> Considerations
> >>
> >> Impact on the Mempool: The mempool may become more dynamic and 
> up-to-date with fewer old, stagnant transactions.
> >> Resource Management: Miners will need to adjust their systems to 
> automatically identify and include relevant transactions.
> >>
> >> Conclusion
> >>
> >> Implementing this BIP will help maintain the integrity and 
> decentralization of the Bitcoin network, preventing censorship and ensuring 
> all transactions have a fair chance of confirmation.
> >>
> >> --
> >> You received this message because you are subscribed to the Google 
> Groups "Bitcoin Development Mailing List" group.
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an email to [email-masked].
> >> To view this discussion visit 
> https://groups.google.com/d/msgid/bitcoindev/fa4a8cd3-778c-4793-8dd4-5662475b6601n%40googlegroups.com
> .
> >
> > --
> > You received this message because you are subscribed to the Google 
> Groups "Bitcoin Development Mailing List" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to [email-masked].
> > To view this discussion visit 
> https://groups.google.com/d/msgid/bitcoindev/CAAg3Je3k4RrQzUQ-x-D81NeMPsFuZTVYFKem9uN9MYP-CnmdRg%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email-masked].
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/fa02c975-8cac-40ec-8497-71f288dda177n%40googlegroups.com.