Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default

From: Matt Corallo <lf-lists@mattcorallo•com>
To: Jonas Schnelli <dev@jonasschnelli•ch>,
	Bitcoin Protocol Discussion
	<bitcoin-dev@lists•linuxfoundation.org>
Cc: Andreas Schildbach <andreas@schildbach•de>
Subject: Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Date: Sat, 27 Jul 2019 12:10:03 -0400	[thread overview]
Message-ID: <4661BB32-3725-4E10-9698-25CDBA2F7ACB@mattcorallo.com> (raw)
In-Reply-To: <0DD0A8C8-374E-4E4F-AB08-C51612A28E16@jonasschnelli.ch>

[-- Attachment #1: Type: text/plain, Size: 6888 bytes --]

This conversation went off the rails somewhat. I don't think there's any immediate risk of NODE_BLOOM peers being unavailable. This is a defaults change, not a removal of the code to serve BIP 37 peers (nor would I suggest removing said code while people still want to use them - the maintenance burden isn't much). Looking at historical upgrade cycles, ignoring any other factors, there will be a large number of nodes serving NODE_BLOOM for many years.

Even more importantly, if you need them, run a node or two. As long as no one is exploiting the issues with them such a node isn't *too* expensive. Or don't, I guarantee you chainanalysis or some competitor of theirs will very very happily serve bloom-filtered clients as long as such clients want to deanonymize themselves. We already see a plurality of nodes on the network are clearly not run-of-the-mill Core nodes, many of which are likely deanonimization efforts.

In some cases BIP 137 is a replacement, in some cases, indeed, it is not. I agree at a protocol level we shouldn't be passing judgement about how users wish to interact with the Bitcoin system (aside from not putting our own, personal, effort into building such things) but that isn't what's happening here. This is an important DoS fix for the average node, and I don't really understand the argument that this is going to break existing BIP 37 wallets, but if it makes you feel any better I can run some beefy BIP 37 nodes.

Matt

> On Jul 26, 2019, at 06:04, Jonas Schnelli via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
> 
> 
>> 1) It causes way too much traffic for mobile users, and likely even too
>> much traffic for fixed lines in not so developed parts of the world.
> 
> Yes. It causes more traffic than BIP37.
> Basic block filters for current last ~7 days (1008 blocks) are about 19MB (just the filters).
> On top, you will probably fetch a handful of irrelevant blocks due to the FPs and due to true relevant txns.
> A over-the-thumb estimation: ~25MB per week of catch-up.
> If you where offline for a month: ~108MB
> 
> Thats certainly more then BIP37 BF (measured 1.6MB total traffic with android schildbach wallet restore blockchain for 8 week [7 weeks headers, 1week merkleblocks]).
> 
> But lets look at it like this: for an additional, say 25MB per week (maybe a bit more), you get the ability to filter blocks without depending on serving peers who may compromise your financial privacy.
> Also, if you keep the filters, further rescans do consume the same or less bandwidth than BF BIP37.
> In other words: you have the chance to potentially increase privacy by consuming bandwidth in the range of a single audio podcast per week.
> 
> I would say the job of protocol developers is protect users privacy where it’s possible (as a default).
> It’s probably a debatable point wether 25MB per week of traffic is worth a potential increase in privacy, though I absolutely think 25MB/week is an acceptable tradeoff.
> Saving traffic is possible by using BIP37 or stratum/electrum… but developers should make sure users are __warned about the consequences__!
> 
> Additionally, it looks like, peer operators are not endless being willing to serve – for free – a CPU/disk intense service with no benefits for the network. I would question wether a decentralised form of BIP37 is sustainable in the long run (if SPV wallet provider bootstrap a net range of NODE_BLOOM peers to make it more reliable on the network would be snake-oil).
> 
> 
>> 
>> 2) It filters blocks only. It doesn't address unconfirmed transactions.
> 
> Well, unconfirmed transaction are uncertain for various reasons.
> 
> BIP158 won't allow you to filter the mempool.
> But as soon as you are connected to the network, you may fetch tx with inv/getdata and pick out the relevant ones (causes also traffic).
> Unclear and probably impossible with the current BIP158 specs to fetch transactions that are not in active relay and are not in a block (mempool txns, at least this is true with the current observed relay tactics).
> 
> 
>> 3) Afaik, it enforces/encourages address re-use. This stems from the
>> fact that the server decides on the filter and in particular on the
>> false positive rate. On wallets with many addresses, a hardcoded filter
>> will be too blurry and thus each block will be matched. So wallets that
>> follow the "one address per incoming payment" pattern (e.g. HD wallets)
>> at some point will be forced to wrap their key chains back to the
>> beginning. If I'm wrong on this one please let me know.
> 
> I’m probably the wrong guy to ask (haven’t made the numbers) but last time I rescanned a Core wallet (in my dev branch) with block filters (and a Core wallet has >2000 addresses by default) it fetched a low and acceptable amount of false positive blocks.
> (Maybe someone who made the numbers step in here.)
> 
> Though, large wallets – AFAIK – also operate badly with BIP37.
> 
>> 
>> 4) The filters are not yet committed to the blockchain. Until that
>> happens we'd have to trust a server to provide correct filters.
> 
> I wouldn’t say so. It’s on a similar level than BIP37.
> BIP37 is not – and can not – be committed to the blockchain.
> You fully trust the peer that it won’t…
> a) create fake unconfirmed transactions (would be the same if a BIP158 wallet would show you unconfirmed transaction)
> b) lies by omission (you will miss relevant transactions, eventually swipe your wallet and loose coins)
> 
> IMO, the point b) is true for BIP37 and BIP158 (as long as not commited).
> In both cases, you can reduce the trust by comparing between peers / filter-providers.
> 
> b) is conceptually solvable with a soft-fork (commitment) in BIP158 (not with BIP37).
> 
> Additionally, block-filters will, very likely, be useful for other features (load/rescan an [old] wallet on a prune peer that has the filters constructed).
> 
> 
> 
> There is probably no clear answer like „X is better than Y“.
> 
> Personally I would like to see developers being more honest/transparent to users about the implications of the used filtering,... and giving them choices.
> Imagine a user can choose between „Electrum / BIP37 / BIP158“ depending on his needs for privacy and availability of bandwidth. Eventually also taking the future usage of this wallet (will he load old private keys, will he receive money daily, etc.) into account.
> 
> Plus, full node hardware appliances that run at home (or in a trusted environment) are solving many of these issues plus adding a bunch of great features – if done right.
> 
> /jonas
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 17831 bytes --]