From: <eric@voskuil•org>
To: "'Ruben Somsen'" <rsomsen@gmail•com>, <bitcoindev@googlegroups.com>
Subject: RE: [bitcoindev] The Tragic Tale of BIP30
Date: Sun, 27 Apr 2025 14:30:29 -0400 [thread overview]
Message-ID: <002201dbb7a2$74676640$5d3632c0$@voskuil.org> (raw)
In-Reply-To: <CAPv7TjZTWhgzzdps3vb0YoU3EYJwThDFhNLkf4XmmdfhbORTaw@mail.gmail.com>
> The problem occurs when we reorg back to a point between block 91880 and
> 91722. When we rewind the blockchain, previously created outputs get
> removed from the UTXO set again.
Consider that a UTXO set (accumulator) is an implementation detail. One of its underlying assumptions is that a given txid cannot repeat in confirmed blocks. This assumption did not hold, and the bip30 workaround for the two exceptions failed to consider the effect of the above reorg given a UTXO set design. Instead of removing the second instance, it removes all instances.
> we could fix the bug by no longer removing the coinbase transaction in case of a reorg of block 91880 and 91842.
IMO this would be the most reasonable resolution, as it would produce the outcome that in fact exists independent of the UTXO set. The previous blocks still actually have the first instance of each duplicated coinbase.
This is also inconsequential from a performance standpoint. It only affects reorganization (infrequent), and only proceeds for these two specific blocks (rare). Furthermore these two specific blocks are already exceptions and the implementation is trivial.
> However, it seems this never occurred.
Correct, only the exception coinbases have been duplicated in the current strong chain.
> Aside from checking for coinbase uniqueness, we could also check that the
> coinbase will not conflict with any future coinbases (i.e. not conflict with
> BIP34 as well as the Consensus Cleanup BIP).
The relationship between BIP34 and BIP30 is also a bit sordid, but the presumption is that the Consensus Cleanup would resolve the existing flaw in BIP34, and that the combination would effectively obsolete BIP30. Under the assumption that this does in fact produce the intended outcome - that BIP34 (presently) Consensus Cleanup (forever) makes further coinbase duplication impossible, BIP30 can remain deactivated to the extent that these are active. Nothing additional is required to avoid the presumably inefficient BIP30 checks.
> Once we fully [remove the checkpoints][3], the bug becomes theoretically (not practically) exploitable.
I would never advocate for adding more, but I'm not aware of any compelling argument to hard fork out the existing checkpoints. The top checkpoint is consensus for over 11 years and materially reduces the validation cost of 295,000 blocks.
> Doing this until block 227931 results in a modest ~7MB cache. However,
> BIP30 might not deactivate, in which case we'd have an ever-growing cache.
> This is solvable as follows....
This is a consequence of the presumed removal of checkpoints above BIP34 activation height. IOW, removing the checkpoints makes it necessary to validate BIP30 until BIP34 activates (block 227,931). The obvious solution to this problem is to not create the problem in the first place.
e
> -----Original Message-----
> From: bitcoindev@googlegroups.com <bitcoindev@googlegroups.com> On
> Behalf Of Ruben Somsen
> Sent: Sunday, April 27, 2025 12:45 PM
> To: bitcoindev@googlegroups.com
> Subject: [bitcoindev] The Tragic Tale of BIP30
>
> Markdown version:
> https://gist.github.com/RubenSomsen/a02b9071bf81b922dcc9edea7d810b
> 7c
>
> ### Introduction
>
> In my recent exploration of [SwiftSync][1], I came to the realization that
> [BIP30][2] has an unresolved consensus bug. It seems this bug can't be
> triggered without a reorg back to 2010, so its seriousness is debatable. We
> currently have checkpoints up to 2013, preventing such a reorg. Once we fully
> [remove the checkpoints][3], the bug becomes theoretically (not practically)
> exploitable.
>
> BIP30 is also a bit of an odd duck in terms of consensus checks in that it
> involves the entire UTXO set without being related to the spending of inputs.
> This is inefficient, and complicates the implementation of alternative validation
> methods such as utreexo, SwiftSync and quite possibly ZKP systems such as
> ZeroSync. It would be nice if we could sunset BIP30 altogether.
>
> Without necessarily advocating for action (the status quo seems quite
> tenable), I'd like to lay out possible solutions for both and open up the
> discussion.
>
> ### 1. Consensus bug
>
> There are two duplicate transactions that BIP30 treats like exceptions. The last
> duplicate was the coinbase transaction in block 91880. When this transaction
> gets processed, the coinbase transaction in block 91722 is overwritten. The
> other instance occurs between these two blocks (91812, 91842).
>
> The problem occurs when we reorg back to a point between block 91880 and
> 91722. When we rewind the blockchain, previously created outputs get
> removed from the UTXO set again. As a result, the overwritten output
> disappears from the UTXO set completely. A node that never witnessed the
> reorg, however, will still have the UTXO in its set. If subsequently the UTXO is
> ever spent, this would result in a fork.
>
> #### Solution A
>
> We could enforce that no reorg can take place between block 91722 and
> 91880 - you'd either have to reorg all of them, or none. This ensures both
> reorged and fresh nodes will not have the problematic outputs in their UTXO
> set. Considering this is only ~160 blocks at the low mining difficulty of 2010,
> this wouldn't be a big constraint.
>
> #### Solution B
>
> When discussing my findings with Sjors Provoost, he pointed out that the
> removal of the checkpoints (which can be seen as a hard fork) [that is being
> considered][3] also presents a window of opportunity to change the pre-
> checkpoint consensus rules - we could fix the bug by no longer removing the
> coinbase transaction in case of a reorg of block 91880 and 91842. Aside from
> that, Sjors' observation also opens up the question whether there are other
> pre-2013 consensus changes we'd want to consider making.
>
> ### 2. Sunsetting BIP30's UTXO set check
>
> BIP30 is currently active from genesis until [BIP34][4] activates at block height
> 227931 (March 2013). If this block is reorged out, BIP30 remains active
> indefinitely. BIP34 has issues of its own that are being addressed in the
> [Consensus Cleanup BIP][5] - you could go and read that, I won't go into too
> much detail here.
>
> Technically, BIP30 only prevents duplicate _unspent_ outputs. It does this by
> checking for each output whether it's already in the UTXO set (this is the
> inefficient part), and rejecting the block if it is. The two 2010 duplicates are
> hard-coded in as exceptions. Under these rules, spending an output and
> recreating it is allowed. However, it seems this never occurred.
>
> One last point to address is why BIP34 gets deactivated if block 227931 is
> reorged out. The reason for this is because otherwise it'd open the door to
> possibly creating outputs prior to BIP34's activation that will conflict with
> BIP34's rules for ensuring coinbase transaction uniqueness (the exact issue the
> Consensus Cleanup is seeking to resolve).
>
> Ideally, it'd be nice to be able to sunset the BIP30 UTXO set check completely,
> ensuring it's no longer required, even in case of a reorg.
>
> #### Solution
>
> Given that we have no duplicates, barring the two exceptions, we could
> replace the inefficient BIP30 UTXO set check with a coinbase uniqueness
> check. We simply cache the coinbase TXIDs and ensure there are no duplicates.
> Doing this until block 227931 results in a modest ~7MB cache. However,
> BIP30 might not deactivate, in which case we'd have an ever-growing cache.
> This is solvable as follows.
>
> Aside from checking for coinbase uniqueness, we could also check that the
> coinbase will not conflict with any future coinbases (i.e. not conflict with
> BIP34 as well as the Consensus Cleanup BIP). This ensures BIP34 can simply
> activate at block height 227931, regardless of whether there's a reorg.
>
> ### In closing
>
> These were some of the issues with BIP30 and possible solutions. Regardless
> of whether we choose to take action, this write-up will serve as a reference.
> Thanks to Antoine Poinsot, Pieter Wuille, and Sjors Provoost for the
> discussions prior to publishing.
>
> -- Ruben Somsen
>
>
> [1]:
> https://gist.github.com/RubenSomsen/a61a37d14182ccd78760e477c7813
> 3cd
> [2]: https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki
> [3]: https://github.com/bitcoin/bitcoin/pull/31649
> [4]: https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki
> [5]: https://github.com/bitcoin/bips/pull/1800
>
> --
> You received this message because you are subscribed to the Google Groups
> "Bitcoin Development Mailing List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to bitcoindev+unsubscribe@googlegroups•com
> <mailto:bitcoindev+unsubscribe@googlegroups•com> .
> To view this discussion visit
> https://groups.google.com/d/msgid/bitcoindev/CAPv7TjZTWhgzzdps3vb0Yo
> U3EYJwThDFhNLkf4XmmdfhbORTaw%40mail.gmail.com
> <https://groups.google.com/d/msgid/bitcoindev/CAPv7TjZTWhgzzdps3vb0Y
> oU3EYJwThDFhNLkf4XmmdfhbORTaw%40mail.gmail.com?utm_medium=em
> ail&utm_source=footer> .
--
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/002201dbb7a2%2474676640%245d3632c0%24%40voskuil.org.
next prev parent reply other threads:[~2025-04-27 18:32 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-27 16:45 Ruben Somsen
2025-04-27 18:20 ` Luke Dashjr
2025-04-27 18:30 ` eric [this message]
2025-04-27 21:01 ` eric
2025-04-28 11:48 ` Sjors Provoost
2025-04-28 12:39 ` Eric Voskuil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='002201dbb7a2$74676640$5d3632c0$@voskuil.org' \
--to=eric@voskuil$(echo .)org \
--cc=bitcoindev@googlegroups.com \
--cc=rsomsen@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox