* [bitcoin-dev] Storing the Merkle Tree in a compact way
@ 2021-09-11 3:00 shymaa arafat
[not found] ` <f6RaXDTOElzKjWTLDhUPFKXs1AeJnaxdC2HGbaO8NponuccadSaInyzyBNIoH-Wa3_cyv4SKOoWNx2gTj1jdhyGLwFExyGF4q1d9UJ68skI=@protonmail.com>
0 siblings, 1 reply; 2+ messages in thread
From: shymaa arafat @ 2021-09-11 3:00 UTC (permalink / raw)
To: Bitcoin Protocol Discussion, lightning-dev
[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]
Allow me to introduce this simple idea that could be useful ...
-The Intuition was some discussion on Utreexo project about storage saving
and some traversing issues in handling the UTXOS Merkle Tree/ forest; that
is N internal nodes need to be stored along with 2N pointers (left&right),
+ maybe 1 more pointer in the leaves special nodes to handle different
traversing options (insert, delete, & differently proof fetch that traverse
aunt or niece node according to your implementation
https://github.com/mit-dci/utreexo/discussions/316)
.
Then, I thought of a simple idea that gets rid of all the pointers;
specially appealing when we have all trees are full (complete) in the
forest, but can work for any Merkle Tree:
- 2D array with variable row size; R[j] is of length (N/2^j)
-For example when N=8 nodes
R[0]=0,1,2,...,7
R[1]=8,9,10,11
R[2]=12,13
R[3]=14
.
-We can see that total storage is just 2N-1 nodes,
no need for pointers, and traversing could be neat in any direction with
the right formula:
-Pseudo code to fetch proof[i] ...
//direction to know + or -
If ((i mod 2)==0) drct=1;
else drct=-1;
// first, the sibling node
proof[i]=R[0,i+drct]
//add the rest thru loop
For(j=1; j≤logN; j++)
{ index= i/(2^j)+drct;
proof[i]=Add(R[j,index]);
}
-In fact it's just the simple primitive approach of transforming a
recursion to an iteration, and even if Utreexo team solved their problem
differently I thought it is worth telling as it can work for any Merkle Tree
.
Thanks for your time,
Shymaa M Arafat
[-- Attachment #2: Type: text/html, Size: 2424 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [bitcoin-dev] [Lightning-dev] Storing the Merkle Tree in a compact way
[not found] ` <f6RaXDTOElzKjWTLDhUPFKXs1AeJnaxdC2HGbaO8NponuccadSaInyzyBNIoH-Wa3_cyv4SKOoWNx2gTj1jdhyGLwFExyGF4q1d9UJ68skI=@protonmail.com>
@ 2021-09-16 15:05 ` shymaa arafat
0 siblings, 0 replies; 2+ messages in thread
From: shymaa arafat @ 2021-09-16 15:05 UTC (permalink / raw)
To: Vincent; +Cc: Bitcoin Protocol Discussion, lightning-dev
[-- Attachment #1.1: Type: text/plain, Size: 2953 bytes --]
It could be viewed as the simple complete tree to 1D array with no
pointers described in lecture 8 here
https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-006-introduction-to-algorithms-spring-2020/lecture-notes/index.htm
starting from min 15 in this video
https://youtu.be/Xnpo1atN-Iw
Since all trees in Utreexo forest are full binary trees, this is perfect to
use, and we can save *76*10⁶*2*size of pointer(probably4bytes)*
*~600MB *with almost no effort.
However, I suggest to put it in a 2D array to make it more easy to handle
(the indexing math) as we, different than the lecture, traverse in many
ways ( normally to delete or insert, and the parent siblings for the proofs)
I wrote more details here
https://bitcointalk.org/index.php?topic=5360009.0
On Thu, Sep 16, 2021, 14:37 Vincent <vincent.palazzo@protonmail•com> wrote:
> Hi.
>
> Thanks for the reference, but I missed where you want save space with this
> compression on the Merkle Tree.
>
> Regards.
>
> Vincent.
> vincenzo.palazzo@protonmail•com
> https://github.com/vincenzopalazzo
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, September 16th, 2021 at 5:15 AM, shymaa arafat <
> shymaa.arafat@gmail•com> wrote:
>
> Allow me to introduce this simple idea that could be useful ...
>
> -The Intuition was some discussion on Utreexo project about storage saving
> and some traversing issues in handling the UTXOS Merkle Tree/ forest; that
> is N internal nodes need to be stored along with 2N pointers (left&right),
> + maybe 1 more pointer in the leaves special nodes to handle different
> traversing options (insert, delete, & differently proof fetch that traverse
> aunt or niece node according to your implementation
> https://github.com/mit-dci/utreexo/discussions/316)
> .
> Then, I thought of a simple idea that gets rid of all the pointers;
> specially appealing when we have all trees are full (complete) in the
> forest, but can work for any Merkle Tree:
>
> - 2D array with variable row size; R[j] is of length (N/2^j)
> -For example when N=8 nodes
> R[0]=0,1,2,...,7
> R[1]=8,9,10,11
> R[2]=12,13
> R[3]=14
> .
> -We can see that total storage is just 2N-1 nodes,
> no need for pointers, and traversing could be neat in any direction with
> the right formula:
>
> -Pseudo code to fetch proof[i] ...
>
> //direction to know + or -
> If ((i mod 2)==0) drct=1;
> else drct=-1;
> // first, the sibling node
> proof[i]=R[0,i+drct]
>
> //add the rest thru loop
> For(j=1; j≤logN; j++)
> { index= i/(2^j)+drct;
> proof[i]=Add(R[j,index]);
> }
>
> -In fact it's just the simple primitive approach of transforming a
> recursion to an iteration, and even if Utreexo team solved their problem
> differently I thought it is worth telling as it can work for any Merkle Tree
> .
> Thanks for your time,
> Shymaa M Arafat
>
>
>
[-- Attachment #1.2: Type: text/html, Size: 5377 bytes --]
[-- Attachment #2: IMG_20210913_194256.jpg --]
[-- Type: image/jpeg, Size: 279763 bytes --]
[-- Attachment #3: IMG_20210913_193322.jpg --]
[-- Type: image/jpeg, Size: 164592 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-09-16 15:05 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-11 3:00 [bitcoin-dev] Storing the Merkle Tree in a compact way shymaa arafat
[not found] ` <f6RaXDTOElzKjWTLDhUPFKXs1AeJnaxdC2HGbaO8NponuccadSaInyzyBNIoH-Wa3_cyv4SKOoWNx2gTj1jdhyGLwFExyGF4q1d9UJ68skI=@protonmail.com>
2021-09-16 15:05 ` [bitcoin-dev] [Lightning-dev] " shymaa arafat
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox