public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Fabian <fjahr@protonmail•com>
To: "bitcoin-dev@lists•linuxfoundation.org"
	<bitcoin-dev@lists•linuxfoundation.org>
Subject: [bitcoin-dev] Breaking change in calculation of hash_serialized_2
Date: Fri, 20 Oct 2023 17:19:19 +0000	[thread overview]
Message-ID: <kxXtwQMByYbMavS5P9a2tAUd8wz0yTUifost_txwTiQfNKTBtgdepLmAyV4XN6m4wY74cdZLX4EtsiEJ-jpZsnSxPIrCAN5wK8eK8xx1WGw=@protonmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 2470 bytes --]

Hello list,

on Wednesday I found a potential malleability issue in the UTXO set dump files
generated for and used by assumeutxo [1]. On Thursday morning theStack had
found the cause of the issue [2]: A bug in the serialization of UTXOs for the
calculation of hash_serialized_2. This is the value used by Bitcoin Core to
check if the UTXO set loaded from a dump file matches what is expected. The
value of hash_serialized_2 expected for a particular block is hardcoded into
the chainparams of each chain.

Implications:
We have been working on a fix [3] for the serialization and aim to include it
in v26.0 (aimed to be released in November). The serialization must change
which means that all historical UTXO set hash results will change after you
upgrade your node to v26.0. To further highlight this, we will also increment
the version, i.e., the value returned in gettxoutset will be renamed to
hash_serialized_3.
It should also be noted that there were additional potentially problematic
issues found from fuzz testing by dergoegge which is why we decided to switch
the serialization completely rather than implementing a minimal fix. The
serialization format is now the same as used by MuHash.

How this may concern you:
1. If you are using hash_serialized_2 for any security critical purposes, you
should check if the bugs in the serialization code could cause issues for you.
You may switch to using hash_serialized_3 as soon as possible (or maybe
consider using MuHash).
2. If you are utilizing hash_serialized_2 for anything critical in your project
in general and require time to upgrade and adapt to the change described above,
please let us know. While we usually try to avoid breaking changes in our APIs without deprecation warning, we currently tend to think it is not necessary to
keep the buggy hash_serialized_2 around since we don’t know of any substantial
use cases and using it may even pose security risks. Furthermore, keeping the
old code around comes at some additional review and maintenance burden and may
lead to some delay in the release of v26.0. But we are happy to reconsider if
keeping hash_serialized_2 around holds serious value for downstream projects.

Feel free to reach out to me directly or comment in the PR [3] or here on the
list.

Cheers,
Fabian

[1] https://github.com/bitcoin/bitcoin/issues/28675
[2] https://github.com/bitcoin/bitcoin/issues/28675#issuecomment-1770389468[3] https://github.com/bitcoin/bitcoin/pull/28685

[-- Attachment #2: Type: text/html, Size: 4039 bytes --]

             reply	other threads:[~2023-10-20 17:19 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-20 17:19 Fabian [this message]
2023-10-20 17:34 ` Peter Todd
2023-10-20 22:01   ` Fabian

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='kxXtwQMByYbMavS5P9a2tAUd8wz0yTUifost_txwTiQfNKTBtgdepLmAyV4XN6m4wY74cdZLX4EtsiEJ-jpZsnSxPIrCAN5wK8eK8xx1WGw=@protonmail.com' \
    --to=fjahr@protonmail$(echo .)com \
    --cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox