From: Aaron Voisine <voisine@gmail•com>
To: Pieter Wuille <pieter.wuille@gmail•com>
Cc: Bitcoin Dev <bitcoin-development@lists•sourceforge.net>
Subject: Re: [Bitcoin-development] Request for review/testing: headers-first synchronization in Bitcoin Core
Date: Sat, 11 Oct 2014 22:51:41 -0700 [thread overview]
Message-ID: <CACq0ZD4XMSFfNbLkX2jdnuGC3ocSSCTkhN_8ba=gL3RVOdmcqQ@mail.gmail.com> (raw)
In-Reply-To: <CAPg+sBjG86ZYKZZXA4VCMiFhz0sdxr_so41Zr+DATOXyRa9_Jw@mail.gmail.com>
This is great Pieter. I was able to sync the entire blockchain from
scratch in a little over 4 hours on a laptop over cable modem. :) No
issues to report. Even my family photos are intact! This makes it
practical to run a full node, part time on a laptop again.
Aaron Voisine
breadwallet.com
On Sat, Oct 11, 2014 at 4:34 PM, Pieter Wuille <pieter.wuille@gmail•com> wrote:
> Hi all,
>
> I believe that a large change that I've been working on for Bitcoin
> Core is ready for review and testing: headers-first synchronization.
> In short, it changes the way the best chain is discovered, downloaded
> and verified, with several advantages:
> * Parallel block downloading (much faster sync on typical network connections).
> * No more stalled downloads.
> * Much more robust against unresponsive or slow peers.
> * Removes a class of DoS attacks related to peers feeding you
> low-difficulty valid large blocks on a side branch.
> * Reduces the need for checkpoints in the code.
> * No orphan blocks stored in memory anymore (reducing memory usage during sync).
> * A major step step towards an SPV mode using the reference codebase.
>
> Historically, this mode of operation has been known for years (Greg
> Maxwell wrote up a description of a very similar method in
> https://en.bitcoin.it/wiki/User:Gmaxwell/Reverse_header-fetching_sync
> in early 2012, but it was known before that), but it took a long time
> to refactor these code enough to support it.
>
> Technically, it works by replacing the single-peer blocks download by
> a single-peer headers download (which typically takes seconds/minutes)
> and verification, and simultaneously fetching blocks along the best
> known headers chain from all peers that are known to have the relevant
> blocks. Downloading is constrained to a moving window to avoid
> unbounded unordering of blocks on disk (which would interfere with
> pruning later).
>
> At the protocol level, it increases the minimally supported version
> for peers to 31800 (corresponding to bitcoin v3.18, released in
> december 2010), as earlier versions did not support the getheaders P2P
> message.
>
> So, the code is available as a github pull request
> (https://github.com/bitcoin/bitcoin/pull/4468), or packaged on
> http://bitcoin.sipa.be/builds/headersfirst, where you can also find
> binaries to test with.
>
> Known issues:
> * At the very start of the sync, especially before all headers are
> processed, downloading is very slow due to a limited number of blocks
> that are requested per peer simultaneously. The policies around this
> will need some experimentation can certainly be improved.
> * Blocks will be stored on disk out of order (in the order they are
> received, really), which makes it incompatible with some tools or
> other programs. Reindexing using earlier versions will also not work
> anymore as a result of this.
> * The block index database will now hold headers for which no block is
> stored on disk, which earlier versions won't support. If you are fully
> synced, it may still be possible to go back to an earlier version.
>
> Unknown issues:
> * Who knows, maybe it will replace your familiy pictures with Nyan
> Cat? Use at your own risk.
>
> TL;DR: Review/test https://github.com/bitcoin/bitcoin/pull/4468 or
> http://bitcoin.sipa.be/builds/headersfirst.
>
> --
> Pieter
>
> ------------------------------------------------------------------------------
> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
next prev parent reply other threads:[~2014-10-12 5:51 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-10-11 23:34 Pieter Wuille
2014-10-12 5:51 ` Aaron Voisine [this message]
2014-10-12 7:14 ` Gregory Maxwell
2014-10-12 8:41 ` Geir Harald Hansen
2014-10-12 9:44 ` Luke Dashjr
2014-10-12 9:52 ` Wladimir
2014-10-12 10:06 ` Geir Harald Hansen
2014-10-12 19:13 ` Jameson Lopp
2014-10-13 22:43 ` 21E14
2014-10-16 5:05 ` Rebroad (sourceforge)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CACq0ZD4XMSFfNbLkX2jdnuGC3ocSSCTkhN_8ba=gL3RVOdmcqQ@mail.gmail.com' \
--to=voisine@gmail$(echo .)com \
--cc=bitcoin-development@lists$(echo .)sourceforge.net \
--cc=pieter.wuille@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox