Re: [bitcoin-dev] A design for Probabilistic Partial Pruning

From: Craig Raw <craigraw@gmail•com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] A design for Probabilistic Partial Pruning
Date: Mon, 1 Mar 2021 08:22:01 +0200	[thread overview]
Message-ID: <CAPR5oBND9DS1Ea=RMqrhnMBjFqU6wpCpv-sMeX9mxqgyLJGYiQ@mail.gmail.com> (raw)
In-Reply-To: <b895f2e4-513f-0c0d-91ac-52af055f332c@LeoWandersleb.de>

[-- Attachment #1: Type: text/plain, Size: 4937 bytes --]

Something to consider adding to this proposal is to keep the idea of
pruning - i.e. retain a sequentially uninterrupted number of the most
recent blocks.

Many users do not run a node for entirely altruistic reasons - they do so,
at least in part, because it allows them to use their wallets privately.
Without this ability, I think the number of users willing to run their node
in this configuration might be reduced.

Another related thought is to have a decreasing density over blocks over
time as you go backwards towards genesis, in order for the data density of
the storage to match the actual usage of the network, in which (I would
imagine) more recent blocks are more heavily requested than early ones.

Craig

On Sun, Feb 28, 2021 at 10:18 AM Leo Wandersleb via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:

> Only headers need to be downloaded sequentially so downloading relevant
> blocks
> from one node is totally possible with gaps in between.
>
> On 2/27/21 4:10 AM, Igor Cota via bitcoin-dev wrote:
> > Hi Keagan,
> >
> > I had a very similar idea. The only difference being for the node to
> decide on
> > a range of blocks to keep beforehand, rather than making the decision
> > block-by-block like you suggest.
> >
> > I felt the other nodes would be better served by ranges due to the
> sequential
> > nature of IBD. Perhaps this would be computationally lighter as well.
> >
> > I also encourage you to read Ryosuke Abe's paper [1] that proposes a DHT
> > scheme to solve this same problem.
> >
> > Cheers,
> > Igor
> >
> > [1] https://arxiv.org/abs/1902.02174
> >
> > On Fri, 26 Feb 2021 at 21:57, Keagan McClelland via bitcoin-dev
> > <bitcoin-dev@lists•linuxfoundation.org
> > <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote:
> >
> >     Hi all,
> >
> >     I've been thinking for quite some time about the problem of pruned
> nodes
> >     and ongoing storage costs for full nodes. One of the things that
> strikes
> >     me as odd is that we only really have two settings.
> >
> >     A. Prune everything except the most recent blocks, down to the cache
> size
> >     B. Keep everything since genesis
> >
> >     From my observations and conversations with various folks in the
> >     community, they would like to be able to run a "partially" pruned
> node to
> >     help bear the load of bootstrapping other nodes and helping with data
> >     redundancy in the network, but would prefer to not dedicate hundreds
> of
> >     Gigabytes of storage space to the cause.
> >
> >     This led me to the idea that a node could randomly prune some of the
> >     blocks from history if it passed some predicate. A rough sketch of
> this
> >     would look as follows.
> >
> >     1. At node startup, it would generate a random seed, this would be
> unique
> >     to the node but not necessary that it be cryptographically secure.
> >     2. In the node configuration it would also carry a "threshold"
> expressed
> >     as some percentage of blocks it wanted to keep.
> >     3. As IBD occurs, based off of the threshold, the block hash, and the
> >     node's unique seed, the node would either decide to prune the data
> or keep
> >     it. The uniqueness of the node's hash should ensure that no block is
> >     systematically overrepresented in the set of nodes choosing this
> storage
> >     scheme.
> >     4. Once the node's IBD is complete it would advertise this as a peer
> >     service, advertising its seed and threshold, so that nodes could
> >     deterministically deduce which of its peers had which blocks.
> >
> >     The goals are to increase data redundancy in a way that more
> uniformly
> >     shares the load across nodes, alleviating some of the pressure of
> full
> >     archive nodes on the IBD problem. I am working on a draft BIP for
> this
> >     proposal but figured I would submit it as a high level idea in case
> anyone
> >     had any feedback on the initial design before I go into specification
> >     levels of detail.
> >
> >     If you have thoughts on
> >
> >     A. The protocol design itself
> >     B. The barriers to put this kind of functionality into Core
> >
> >     I would love to hear from you,
> >
> >     Cheers,
> >     Keagan
> >     _______________________________________________
> >     bitcoin-dev mailing list
> >     bitcoin-dev@lists•linuxfoundation.org
> >     <mailto:bitcoin-dev@lists•linuxfoundation.org>
> >     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
> >
> >
> > --
> > *Igor Cota*
> > Codex Apertus d.o.o.
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists•linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 6860 bytes --]