> In terms of adding more taproot specific functionality, I've had a number
of
> items in my laundry list such as:

Forgot to add this other item (also the list wasn't meant to be only tapoot
stuff):
  * reviving old projects to include a micropayment-for-data layer to
    incentivize nodes to serve the filters and other data

On Thu, Nov 4, 2021 at 3:01 PM Olaoluwa Osuntokun <laolu32@gmail.com> wrote:

> Hi y'all,
>
> If you're an active user of neutrino [8], then you probably heard about
> what
> went down over the past week or so on testnet as related to Taproot. First,
> i just wanted to reassure everyone that nothing is fundamentally broken
> with
> BIP 157/158 as it relates to taproot, and we already have a mitigation
> patch
> in place for the issue we encountered.
>
> The rest of this mail is structured in a FAQ style to make it easy to skim
> and extract the information that may be relevant to the reader.
>
> ## What happened on testnet?
>
> Neutrino nodes on testnet rejected a filter (thinking it was invalid) due
> to this transaction spending a taproot input [1]. This was due to a faulty
> heuristics in the neutrino _client code_ (not part of the protocol) that
> attempted to verify the contents of a filter more completely.
>
> In retrospect, the heuristic in question wasn't full proof, as it attempted
> to derive the _pk script_ of a transaction based on its input
> witness/sigScript. This worked pretty well in the context of segwit v0, but
> it isn't possible to exhaustively do as we don't know what future spends
> will look like.
>
> ## Is neutrino broken?
>
> No, the client side is fine, and the protocol is fine.
>
> The problematic heuristic has been removed in this PR [2], which will be
> included in lnd 0.14, and has been tagged with neutrino 0.13 [3].
>
> To dig into _why_ we attempted to use such a heuristic, we'll need to
> revisit how BIP 158 works briefly. For each block, we insert the
> `pkScript`s
> of all the outputs, and also the prev out's pkScript (the script being
> spent) as well. This lets the filter compress script re-use in both inputs
> and outputs, and also makes it possible to implement some protocols in a
> more light-client friendly manner (for example Loop uses this to has the
> client watch HTLC _scripts_ being spent, as it doesn't necessarily know the
> txid/outpoint).
>
> The one issue with this, is that while clients can ensure that all the
> `pkScripts` of outputs have been inserted, they can't do the same for the
> inputs (which is why we added that heuristic in the client code). Luckily
> we
> know how to properly fix this at the protocol level, more on that below.
>
> ## How can I make sure my neutrino clients handle the Taproot upgrade on
> mainnet smoothly?
>
> Upgrade to 0.14 (assuming it's out in time), or apply this small patch [4].
> The patch just demotes an error case to a warning message, so anyone
> running
> a fork should be able to easily apply the fix.
>
> Alongside, optionally extend these filter header guides [7].
>
> We're looking into some intermediate ground where we can verify the scripts
> that we know are relevant to the node.
>
> ## How will BIP 158/157 evolve post taproot?
>
> In terms of adding more taproot specific functionality, I've had a number
> of
> items in my laundry list such as:
>
>   * creating new segwit-only filters with re-parameterized fp rates (also
>     examine other filter types such as pure outpoints, etc)
>
>   * creating filters that include witness data to allow matching on
>     internal/external keys, the control block, merkle root, annex, etc
>
>   * add a new protocol extension to btcd (with a corresponding BIP) to
>     allow notes to fetch block undo data (as described here [5]) to fully
>     verify fetched filters or a node needs to reconcile conflicting filters
>
>   * new filters that span across multiple blocks as previously worked on by
>     Kalle Alm (couldn't find a link to his PR when typing this...)
>
> Make further progress towards a proposal that allows filters to be
> committed
> either as a soft-fork, or a "velvet fork" [6] where miners optionally
> commit to
> the past filter header chain.
>
>
> -- Laolu
>
> [1]:
> https://mempool.space/testnet/tx/4b425a1f5c0fcf4794c48b810c53078773fb768acd2be1398e3f561cc3f19fb8
> [2]: https://github.com/lightninglabs/neutrino/pull/234
> [3]: https://github.com/lightninglabs/neutrino/releases/tag/v0.13.0
> [4]: https://github.com/lightninglabs/neutrino/pull/234/files
> [5]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-February/016649.html
> [6]: https://eprint.iacr.org/2018/087
> [7]:
> https://github.com/lightninglabs/neutrino/blob/5e09bd9b5d65e90c6ff07aa11b3b9d80d42afb86/chainsync/filtercontrol.go#L15
> [8]: https://github.com/lightninglabs/neutrino
>