public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: shymaa arafat <shymaa.arafat@gmail.com>
To: Vincent <vincent.palazzo@protonmail.com>
Cc: Bitcoin Protocol Discussion
	<bitcoin-dev@lists.linuxfoundation.org>,
	lightning-dev <lightning-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] [Lightning-dev] Storing the Merkle Tree in a compact way
Date: Thu, 16 Sep 2021 17:05:24 +0200	[thread overview]
Message-ID: <CAM98U8==V1igTn9UuM2w4SHOJ+G6McO-nEkAiV-93-tqf5+bKw@mail.gmail.com> (raw)
In-Reply-To: <f6RaXDTOElzKjWTLDhUPFKXs1AeJnaxdC2HGbaO8NponuccadSaInyzyBNIoH-Wa3_cyv4SKOoWNx2gTj1jdhyGLwFExyGF4q1d9UJ68skI=@protonmail.com>


[-- Attachment #1.1: Type: text/plain, Size: 2953 bytes --]

It could be viewed as the simple complete tree to 1D array  with no
pointers described in lecture 8 here
https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-006-introduction-to-algorithms-spring-2020/lecture-notes/index.htm
starting from min 15 in this video
https://youtu.be/Xnpo1atN-Iw

Since all trees in Utreexo forest are full binary trees, this is perfect to
use, and we can save *76*10⁶*2*size of pointer(probably4bytes)*
*~600MB *with almost no effort.

However, I suggest to put it in a 2D array to make it more easy to handle
(the indexing math) as we, different than the lecture, traverse in many
ways ( normally to delete or insert, and the parent siblings for the proofs)

I wrote more details here
https://bitcointalk.org/index.php?topic=5360009.0

On Thu, Sep 16, 2021, 14:37 Vincent <vincent.palazzo@protonmail.com> wrote:

> Hi.
>
> Thanks for the reference, but I missed where you want save space with this
> compression on the Merkle Tree.
>
> Regards.
>
> Vincent.
> vincenzo.palazzo@protonmail.com
> https://github.com/vincenzopalazzo
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, September 16th, 2021 at 5:15 AM, shymaa arafat <
> shymaa.arafat@gmail.com> wrote:
>
> Allow me to introduce this simple idea that could be useful ...
>
> -The Intuition was some discussion on Utreexo project about storage saving
> and some traversing issues in handling the UTXOS Merkle Tree/ forest; that
> is  N internal nodes need to be stored along with 2N pointers (left&right),
> + maybe 1 more pointer in the leaves special nodes to handle different
> traversing options (insert, delete, & differently proof fetch that traverse
> aunt or niece node according to your implementation
> https://github.com/mit-dci/utreexo/discussions/316)
> .
> Then, I thought of a simple idea that gets rid of all the pointers;
> specially appealing when we have all trees are full (complete) in the
> forest, but can work for any Merkle Tree:
>
> - 2D array with variable row size; R[j] is of length (N/2^j)
> -For example when N=8 nodes
> R[0]=0,1,2,...,7
> R[1]=8,9,10,11
> R[2]=12,13
> R[3]=14
> .
> -We can see that total storage is just 2N-1 nodes,
> no need for pointers, and traversing could be neat in any direction with
> the right formula:
>
> -Pseudo code to fetch proof[i] ...
>
> //direction to know + or -
> If ((i mod 2)==0) drct=1;
>             else drct=-1;
> // first, the sibling node
> proof[i]=R[0,i+drct]
>
> //add the rest thru loop
> For(j=1; j≤logN; j++)
>  { index= i/(2^j)+drct;
>     proof[i]=Add(R[j,index]);
>  }
>
> -In fact it's just the simple primitive approach of transforming a
> recursion to an iteration, and even if Utreexo team solved their problem
> differently I thought it is worth telling as it can work for any Merkle Tree
> .
> Thanks for your time,
> Shymaa M Arafat
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 5377 bytes --]

[-- Attachment #2: IMG_20210913_194256.jpg --]
[-- Type: image/jpeg, Size: 279763 bytes --]

[-- Attachment #3: IMG_20210913_193322.jpg --]
[-- Type: image/jpeg, Size: 164592 bytes --]

      parent reply	other threads:[~2021-09-16 15:05 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-11  3:00 [bitcoin-dev] Storing the Merkle Tree in a compact way shymaa arafat
     [not found] ` <f6RaXDTOElzKjWTLDhUPFKXs1AeJnaxdC2HGbaO8NponuccadSaInyzyBNIoH-Wa3_cyv4SKOoWNx2gTj1jdhyGLwFExyGF4q1d9UJ68skI=@protonmail.com>
2021-09-16 15:05   ` shymaa arafat [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAM98U8==V1igTn9UuM2w4SHOJ+G6McO-nEkAiV-93-tqf5+bKw@mail.gmail.com' \
    --to=shymaa.arafat@gmail.com \
    --cc=bitcoin-dev@lists.linuxfoundation.org \
    --cc=lightning-dev@lists.linuxfoundation.org \
    --cc=vincent.palazzo@protonmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox