* [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message
@ 2023-04-18 6:37 Will Clark
2023-04-18 16:57 ` David A. Harding
2023-05-01 13:24 ` 0xB10C
0 siblings, 2 replies; 3+ messages in thread
From: Will Clark @ 2023-04-18 6:37 UTC (permalink / raw)
To: bitcoin-dev
I'd like to discuss potential removal of the BIP35 P2P `mempool`
message.
Supporting the BIP35 `mempool` message for arbitrary peers is bad for
privacy while being relatively inefficient for trusted peers. Its
original intention was to be publicly callable, but it is now (in
Bitcoin Core) gated behind stricter Net Permissions which make it
accessible to trusted peers only.
When serving trusted clients one alternative might be to use the
`savemempool` RPC, which can then be loaded directly (in whole) by the
client. This does currently have some shortcomings, namely that the
client will lose the contents of their mempool in the process, so if
they have different policies some transactions may be lost. Currently
there is no way to load and de-duplicate a mempool dumped by this RPC
into an existing mempool, although a PR has been opened to Bitcoin Core
enabling this functionality so it may be available in the
not-too-distant future:
https://github.com/bitcoin/bitcoin/pull/27460
In my view dumping and loading a dumped mempool to sync two trusted
nodes (or bootstrap one node) makes more sense via RPC as more
transaction data can be included in the sync (e.g. transaction expiry
time), but there is an argument to be made that syncing via P2P message
would be more convenient.
N.B. that two (un-patched) bitcoin nodes cannot currently sync from each
other using the `mempool` P2P message as there is no functionality to
_send_ these messages, only to service them.
Removing this message would also provide an (albeit small) clean-up to
the P2P codebase, bringing with it the usual benefits in terms of
maintainability etc.
I have a draft PR open for the removal of the mempool message here:
https://github.com/bitcoin/bitcoin/pull/27426
However, before moving forward, I want to ensure that there are no
active use cases or technical opposition to its removal from readers of
this list. To that end, I kindly request your input on the following
questions:
1. Are there any parties who still directly rely on the BIP35 P2P
`mempool` message? If so, please share your use case and any potential
impact that the removal might have on your operations.
2. Do you foresee any issues or negative consequences resulting from the
removal of the `mempool` message? If so, please elaborate on the
potential problems and their severity.
From a quick search of node implementations I can see `btcd`,
`libbitcoin` and `BitcoinJ` all have BIP35 messages specified, but I
have not checked more deeply to see if they are using/servicing/ignoring
them, and it's difficult to gauge upstream usage by other projects
without outreach like this...
I look forward to hearing your thoughts.
--
Cheers,
Will
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message
2023-04-18 6:37 [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message Will Clark
@ 2023-04-18 16:57 ` David A. Harding
2023-05-01 13:24 ` 0xB10C
1 sibling, 0 replies; 3+ messages in thread
From: David A. Harding @ 2023-04-18 16:57 UTC (permalink / raw)
To: bitcoin-dev
> When serving trusted clients one alternative might be to use the
> `savemempool` RPC, which can then be loaded directly (in whole) by the
> client.
It was common in the past for lightweight clients to load a BIP37 filter
and then send a `getdata` for requesting `mempool`. In that case, the
node would filter the mempool transactions and only send transactions
matching the filter to the client (plus false positives, which the
client could choose to keep very low).
The above approach minimized the amount of data that needed to be
transferred, which can be very important for lite clients on metered or
bandwidth-limited connections---especially considering that lite clients
on poor connections (e.g. mobile) might get disconnected frequently and
so need to re-request the filtered mempool every time they reconnect to
acquire any new unconfirmed transactions that arrived while they were
disconnected.
By comparison, during a period of backlog (the natural state, we hope),
the mempool contents in the `savemempool` format are about 300 MB. I
think that's a bit much to potentially be sending to lite clients just
so they can learn about any unconfirmed transactions which arrived since
they last connected.
Although I understand and support the desire to remove problematic
public interfaces like BIP37 and BIP35, I think we should also be aiming
to build interfaces which make it easier for people to use third-party
wallets with their own trusted nodes. Right now, it's possible to
use[*]
`getheaders`, BIP157/8, and `getdata(block)` with your own node to learn
about all confirmed transactions affecting your wallet. It's also
possible now to use BIP37 and BIP35 to get unconfirmed transactions in
a bandwidth-efficient manner, if your connection is allowlisted.
I would personally like to see lite clients that use a trusted node
receive a replacement for BIP35/7 support before those protocols are
removed. (Of course, I'd also like to see support for BIP324 and for
something like countersign so that authenticated and encrypted
connections from a lite client to a trusted node are easy to setup.)
Thanks,
-Dave
[*]: Requires an authenticated connection to be secure (and should
ideally be encrypted).
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message
2023-04-18 6:37 [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message Will Clark
2023-04-18 16:57 ` David A. Harding
@ 2023-05-01 13:24 ` 0xB10C
1 sibling, 0 replies; 3+ messages in thread
From: 0xB10C @ 2023-05-01 13:24 UTC (permalink / raw)
To: bitcoin-dev
[-- Attachment #1.1.1: Type: text/plain, Size: 1324 bytes --]
Hi Will,
I shared some numbers and feedback as comment [0] on your PR wanted to
answer question 1. for completeness here too.
> Its original intention was to be publicly callable, but it is now (in
Bitcoin Core) gated behind stricter Net Permissions which make it
accessible to trusted peers only.
Bitcoin Core does only gate processing of mempool messages on
NetPermissionFlags::Mempool when bloom filters are disabled [1]. While
these are disabled by default, more than 20% (see PR comment) of nodes
on the network have bloom filters enabled. These nodes all respond to
mempool messages with INV messages.
> 1. Are there any parties who still directly rely on the BIP35 P2P
`mempool` message?
I've been receiving on average about 20 mempool messages per hour to a
well-connected NODE_BLOOM Bitcoin Core node. I've seen multiple messages
from the user agent /BitcoinKit:0.1.0/, /bitcoinj:0.*.*/Bitcoin
Wallet:*/, /WalletKit:0.1.0/, and /bread:2.1/. Similarly, the node
responds to the clients with INVs up to the max number of 50k entries
and with smaller (bloom) filtered INVs.
0xB10C
[0]: https://github.com/bitcoin/bitcoin/pull/27426#issuecomment-1529678174
[1]:
https://github.com/bitcoin/bitcoin/blob/d89aca1bdbe52406f000e3fa8dda12c46dca9bdd/src/net_processing.cpp#LL4603C52-L4603
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 10965 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-05-01 13:30 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-18 6:37 [bitcoin-dev] Proposal to Remove BIP35 P2P 'mempool' Message Will Clark
2023-04-18 16:57 ` David A. Harding
2023-05-01 13:24 ` 0xB10C
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox