* [Bitcoin-development] Bootstrapping full nodes post-pruning
@ 2012-06-10 23:06 Mike Hearn
2012-06-11 15:39 ` Gregory Maxwell
0 siblings, 1 reply; 5+ messages in thread
From: Mike Hearn @ 2012-06-10 23:06 UTC (permalink / raw)
To: Bitcoin Dev
Apologies if this has been discussed elsewhere. I don't recall us ever
reaching a solid conclusion on it.
A node that has pruned its block chain cannot serve the chain to new
nodes. So there are three options for bootstrapping a newly installed
node:
1) Have some kind of special archival nodes that never prune
(advertised via the services field?). Encourage people to run them,
somehow.
2) Ship a post-pruning block chain and tx index with the client
downloads, so the client starts up already bootstrapped.
3) Some combination of both. It's safe to assume some people will keep
unpruned chains around no matter what. But for many users (2) is
easiest and archival nodes would be put under less load if they were
used only by users who wish to fully bootstrap from only the code.
I remember some people, Greg in particular, who were not a fan of
approach (2) at all, though it has the benefit of speeding startup for
new users as there's no indexing overhead.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] Bootstrapping full nodes post-pruning
2012-06-10 23:06 [Bitcoin-development] Bootstrapping full nodes post-pruning Mike Hearn
@ 2012-06-11 15:39 ` Gregory Maxwell
2012-06-11 20:36 ` Mike Hearn
0 siblings, 1 reply; 5+ messages in thread
From: Gregory Maxwell @ 2012-06-11 15:39 UTC (permalink / raw)
To: Mike Hearn; +Cc: Bitcoin Dev
On Sun, Jun 10, 2012 at 7:06 PM, Mike Hearn <mike@plan99•net> wrote:
> I remember some people, Greg in particular, who were not a fan of
> approach (2) at all, though it has the benefit of speeding startup for
> new users as there's no indexing overhead.
I'm not a fan of anything which introduces unauditable single source
material. "Trust us" is a bad place to be because it would greatly
increase the attractiveness of compromising developers.
If we wanted to go the route of shipping pruned chains I'd prefer to
have a deterministic process to produce archival chains and then start
introducing commitments to them in the blockchain or something like
that. Then a client doing a reverse header sync[1] would bump into a
commitment for an archival chain that they have and would simply stop
syncing and use the archival chain for points before that.
This would leave it so that the distribution of the software could
still be audited.
More generally we should start doing something with the service
announcements so that full nodes that don't have enough bandwidth to
support a lot of syncing from new nodes can do so without turning off
listening.
[1] https://en.bitcoin.it/wiki/User:Gmaxwell/Reverse_header-fetching_sync
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] Bootstrapping full nodes post-pruning
2012-06-11 15:39 ` Gregory Maxwell
@ 2012-06-11 20:36 ` Mike Hearn
2012-06-11 20:43 ` Gregory Maxwell
0 siblings, 1 reply; 5+ messages in thread
From: Mike Hearn @ 2012-06-11 20:36 UTC (permalink / raw)
To: Gregory Maxwell; +Cc: Bitcoin Dev
> If we wanted to go the route of shipping pruned chains I'd prefer to
> have a deterministic process to produce archival chains
Yeah, that sounds reasonable. I mean, I can't see why pruning would
not be deterministic. So if you download a binary that contains a
pre-indexed and pruned chain up to block 180,000 or whatever, you
should be able to blow away the data files and run with
"-syncto=180000 -prune", then check the hashes of the newly created
files vs what you downloaded.
Unless BDB has some weird behaviour in it, that shouldn't require any
additional effort, and anyone could set up a cron job to verify the
downloads match what is expected.
Even if a more complex scheme is used whereby commitments are in the
block chain, somebody still has to verify the binaries match the
source. If that isn't true, the software could do anything and you'd
never know.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] Bootstrapping full nodes post-pruning
2012-06-11 20:36 ` Mike Hearn
@ 2012-06-11 20:43 ` Gregory Maxwell
2012-06-11 20:48 ` Mike Hearn
0 siblings, 1 reply; 5+ messages in thread
From: Gregory Maxwell @ 2012-06-11 20:43 UTC (permalink / raw)
To: Mike Hearn; +Cc: Bitcoin Dev
On Mon, Jun 11, 2012 at 4:36 PM, Mike Hearn <mike@plan99•net> wrote:
> Unless BDB has some weird behaviour in it, that shouldn't require any
HAHAHA. Have you consider doing comedy full time?
Actual BDB files are absolutely not deterministic. Nor is the raw
blockchain itself currently, because blocks aren't always added in the
same order (plus they get orphans in them)
But the serious inter-version compatibility problems as well as poor
space efficiency make BDB a poor candidate for read only pruned
indexes.
> Even if a more complex scheme is used whereby commitments are in the
> block chain, somebody still has to verify the binaries match the
> source. If that isn't true, the software could do anything and you'd
> never know.
The binaries distributed by bitcoin.org are all already compiled
deterministically and validated by multiple independent parties. In
the future there will be a downloader tool (e.g. for updates) which
will automatically check for N approvals before accepting an update,
even for technically unsophisticated users.
This will produce a full chain of custody which tracks the actual
binaries people fetch to specific source code which can be audited, so
substitution attacks will at least in theory always be detectable. Of
course, you're left with Ken Thompson's compiler attack but even that
can be substantially closed.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] Bootstrapping full nodes post-pruning
2012-06-11 20:43 ` Gregory Maxwell
@ 2012-06-11 20:48 ` Mike Hearn
0 siblings, 0 replies; 5+ messages in thread
From: Mike Hearn @ 2012-06-11 20:48 UTC (permalink / raw)
To: Gregory Maxwell; +Cc: Bitcoin Dev
> Actual BDB files are absolutely not deterministic. Nor is the raw
> blockchain itself currently, because blocks aren't always added in the
> same order (plus they get orphans in them)
That's true. Though if you prune up to the last checkpoint, orphans
before that point can be safely thrown away.
I wonder if swapping out bdb for LevelDB might make sense at some
point. I'm not sure how deterministic that is either though :)
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-06-11 20:48 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-06-10 23:06 [Bitcoin-development] Bootstrapping full nodes post-pruning Mike Hearn
2012-06-11 15:39 ` Gregory Maxwell
2012-06-11 20:36 ` Mike Hearn
2012-06-11 20:43 ` Gregory Maxwell
2012-06-11 20:48 ` Mike Hearn
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox