From: Mike Hearn <mike@plan99•net>
To: Jeff Garzik <jgarzik@exmulti•com>
Cc: Bitcoin Dev <bitcoin-development@lists•sourceforge.net>
Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering
Date: Fri, 11 Jan 2013 15:11:44 +0100 [thread overview]
Message-ID: <CANEZrP2k30UsWFYSZ7Bh5Hm4LJ9vEAMEUgYSrYkcXcDTY2Z79Q@mail.gmail.com> (raw)
In-Reply-To: <CA+8xBpcB6kXWyRbeUknK6gkcrFMV6YtrDk0c938q1_32U6GtRw@mail.gmail.com>
I did some very rough initial performance tests.
Syncing from a local peer gives me about 50 blocks per second in the
later parts of the chain (post SD), which is about a 10-20x speedup
over what I could do before. This is on a MacBook Pro. But at those
points it's clearly bottlenecked by bitcoind which has saturated its
CPU core. This makes sense - the filtering is much more server than
client intensive because every transaction in every block has to be
loaded and checked.
I think filtering can be fairly well parallelized on the server side.
So the current 10-20x speedup could potentially be larger if the
server becomes more efficient at scanning and filtering blocks. It's
still a very nice win for now, especially bandwidth wise. And if Matt
makes the mempool command filtered it solves a common usability
problem as well.
Once we get this code in, merged and rolled out I think what we need
for bloom v2 is clear:
- Multi-thread the filtering process in bitcoind so transactions can
be checked in parallel. A 4-core server would then get 4x faster at
filtering blocks and assuming it's not too busy doing other stuff we
could maybe sync at more like 200 blocks per second, which is cool ...
more than a days worth of history for each second of syncing.
- Make the client smarter so the FP rate is adapted during the sync
process. An FP rate that makes sense post-SD results in no false
positives pre-SD, more or less.
- Make the client shard its wallet keys over multiple peers, for
better privacy.
- Make the client suck down filtered blocks in parallel from multiple
peers, for better speed.
As it seems the bottleneck for chain sync is now CPU time, the latter
point may be the most important from a practical perspective.
On Fri, Jan 11, 2013 at 6:02 AM, Jeff Garzik <jgarzik@exmulti•com> wrote:
> On Thu, Jan 10, 2013 at 10:59 PM, Matt Corallo <bitcoin-list@bluematt•me> wrote:
>> Ive been missing lately, when is 0.8 targeted for freeze?
>
> 0.8rc1 will probably happen when the core ultraprune/leveldb stuff is stable.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti•com
next prev parent reply other threads:[~2013-01-11 14:11 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-24 15:56 Mike Hearn
2012-10-24 16:22 ` Pieter Wuille
2012-10-24 16:35 ` Mike Hearn
2012-10-24 17:11 ` Pieter Wuille
2012-10-24 18:54 ` Gavin Andresen
2012-10-24 19:00 ` Matt Corallo
2012-10-24 19:10 ` Mike Hearn
2012-10-24 20:29 ` Gavin Andresen
2012-10-24 20:58 ` Mike Hearn
2012-10-24 21:55 ` Jeff Garzik
2012-10-25 16:56 ` Gregory Maxwell
2012-10-25 17:01 ` Gregory Maxwell
2012-10-26 14:01 ` Mike Hearn
2012-10-26 14:17 ` Gregory Maxwell
2012-10-26 14:21 ` Mike Hearn
2012-10-26 14:34 ` Gregory Maxwell
2012-11-06 19:14 ` Pieter Wuille
2012-11-21 15:15 ` Pieter Wuille
2012-11-21 18:38 ` Matt Corallo
2012-11-27 21:10 ` Pieter Wuille
2013-01-10 15:21 ` Mike Hearn
2013-01-11 3:59 ` Matt Corallo
2013-01-11 5:02 ` Jeff Garzik
2013-01-11 14:11 ` Mike Hearn [this message]
2013-01-11 14:13 ` Mike Hearn
2013-01-16 10:43 ` Mike Hearn
2013-01-16 15:00 ` Matt Corallo
2013-01-18 16:38 ` Mike Hearn
2013-01-19 9:51 ` Andreas Schildbach
2013-01-30 11:09 ` Mike Hearn
2013-01-30 11:13 ` Mike Hearn
2013-02-06 16:33 ` Mike Hearn
2013-02-06 16:45 ` Gregory Maxwell
2013-02-20 12:44 ` Mike Hearn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CANEZrP2k30UsWFYSZ7Bh5Hm4LJ9vEAMEUgYSrYkcXcDTY2Z79Q@mail.gmail.com \
--to=mike@plan99$(echo .)net \
--cc=bitcoin-development@lists$(echo .)sourceforge.net \
--cc=jgarzik@exmulti$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox