From: Jonas Schnelli <dev@jonasschnelli•ch>
To: David Vorick <david.vorick@gmail•com>
Cc: Bitcoin Dev <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] Small Nodes: A Better Alternative to Pruned Nodes
Date: Tue, 18 Apr 2017 09:43:52 +0200 [thread overview]
Message-ID: <CA3B35F8-1E81-4202-9784-9770D22C9310@jonasschnelli.ch> (raw)
In-Reply-To: <CAFVRnypbQQ-vsSLqv48cYaqTCty4R1DmFRqfAvxe4mAqyQNXxQ@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 3926 bytes --]
Hi Dave
> A node that stores the full blockchain (I will use the term archival node) requires over 100GB of disk space, which I believe is one of the most significant barriers to more people running full nodes. And I believe the ecosystem would benefit substantially if more users were running full nodes.
Thanks for your proposal.
I agree that 100GB of data may be cumbersome for some systems, especially if you target end user systems (Laptops/Desktops). Though, in my opinion, for those systems, CPU consumption is the biggest UX blocker.
Bootstrapping a full node on a decent consumer system with default parameters takes days, and, during this period, you probably run at full CPU capacity and you will be disturbed by constant fan noise. Standard tasks may be impossible because your system will be slowed down to a point where even word processing may get difficult.
This is because Core (with its default settings) is made to sync as fast as possible.
Once you have verified the chain and you reach the chain tip, indeed, it will be much better (until you shutdown for a couple of days/hours and have to re-sync/catch-up).
1. I agree that we need to have a way for pruned nodes to partially serve historical blocks.
My personal measurements told me that around ~80% of historical block serving are between tip and -1’000 blocks.
Currently, Core nodes have only two modes of operations, „server all historical blocks“ or „none“.
This makes little sense especially if you prune to a target size of, lets say, 80GB (~80% of the chain).
Ideally, there would be a mode where your full node can signal a third mode „I keep the last 1000 blocks“ (or make this more dynamic).
2. Bootstrapping new peers
I’m not sure if full nodes must be the single point of historical data storage. Full nodes provide a valuable service (verification, relay, filtering, etc.). I’m not sure if serving historical blocks is one of them. Historical blocks could be made available on CDN’s or other file storage networks. You are going to verify them anyways,... the serving part is pure data storage.
I’m also pretty sure that some users have stopping running full nodes because their upstream bandwidth consumption (because of serving historical blocks) was getting intolerable.
Especially „consumer“ peers must have been hit by this (little experience in how to reduce traffic, upstream in general is bad for consumers-connections, little resources in general).
Having a second option built into full nodes (or as an external bootstrap service/app) how to download historical blocks during bootstrapping could probably be a relieve for "small nodes“.
It could be a little daemon that downloads historical blocks from CDN’s, etc. and feeds them into your full node over p2p/8333 and kickstarts your bootstrapping without bothering valuable peers.
Or, the alternative download, could be built into the full nodes main logic.
And, if it wasn’t obvious, this must not bypass the verification!
I’m also aware of the downsides of this. This can eventually reduce decentralisation of the storage of historical bitcoin blockchain data and – eventually – increase the upstream bandwidth of peers willing to serve historical blocks (especially in a transition phase to a second „download“-option).
Maybe it’s a tradeoff between reducing decentralisation by killing low resource nodes because serving historical blocks is getting too resource-intense _or_ reducing decentralisation by moving some percentage of the historical data storage away from the bitcoin p2p network.
The later seems more promising to me.
To your proposal:
- Isn’t there a tiny finger-printing element if peers have to pick an segmentation index?
- SPV bloom filter clients can’t use fragmented blocks to filter txns? Right? How could they avoid connecting to those peers?
</jonas>
[-- Attachment #1.2: Type: text/html, Size: 4726 bytes --]
[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2017-04-18 7:44 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-04-17 6:54 David Vorick
2017-04-17 7:11 ` Danny Thorpe
2017-04-17 7:27 ` David Vorick
2017-04-20 15:50 ` Erik Aronesty
2017-04-20 23:42 ` Aymeric Vitte
2017-04-21 13:35 ` David Kaufman
2017-04-21 15:58 ` Leandro Coutinho
2017-04-17 10:14 ` Aymeric Vitte
2017-04-19 17:30 ` David Vorick
2017-04-20 9:46 ` Tom Zander
2017-04-20 20:32 ` Andrew Poelstra
2017-04-21 8:27 ` Tom Zander
2017-04-20 11:27 ` Aymeric Vitte
2017-04-18 7:43 ` Jonas Schnelli [this message]
2017-04-18 10:50 ` Tom Zander
2017-04-18 13:07 ` Tier Nolan
2017-04-18 23:19 ` Aymeric Vitte
2017-04-19 4:28 ` udevNull
2017-04-19 13:47 ` Angel Leon
2017-04-21 20:38 ` Gregory Maxwell
2017-04-23 16:27 ` Aymeric Vitte
2017-05-03 14:03 ` Erik Aronesty
2017-05-03 19:10 ` Natanael
2017-05-03 22:45 ` Aymeric Vitte
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA3B35F8-1E81-4202-9784-9770D22C9310@jonasschnelli.ch \
--to=dev@jonasschnelli$(echo .)ch \
--cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
--cc=david.vorick@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox