From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BEF0A98A for ; Mon, 28 Aug 2017 16:27:10 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id CF9E8196 for ; Mon, 28 Aug 2017 16:27:09 +0000 (UTC) Received: by mail-wm0-f51.google.com with SMTP id t201so6820650wmt.1 for ; Mon, 28 Aug 2017 09:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=eYG8NpntpaUUts5p4vZq8Q8ewoZIOvWxwl8UkUPvjEk=; b=TbjUYjUZkDLXtVsqneEfuxnBrm0yfu1ynwpR8hn6EFk5Axyllkj0uItGXuLO2uHPeH uve+MBYUG3D7sairpYgqmiLe2i0qR+saRQ5NBoAKeibJjNB+qRXHiR4S+nc1YNUPesh/ yz5cDRq/YMBeOIe81PgOivXufue/ZO/qcWYPS/kUWyQNBATp/MRKLegHJHnNX/KUCcXV +J17gxmU7Z+alU/r2O/pnPIaF7Z+LOy+GxP59wwzseyakvmB7gAqWs8vcb/pAacFnxiG ThvjRudfWl3n11p1/GqfLkxk1RwTdlPQPHfV4dcg4OhiHC5IoQpKxX9el8ujTeK1918t JXOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=eYG8NpntpaUUts5p4vZq8Q8ewoZIOvWxwl8UkUPvjEk=; b=QdBQOfIsnAOVm/G6Id2rjTdWhd6QeSfopBV6L6/UAH7kPA0+D1BO892rjcmlaPBxVb F8twfT6Q2VZz/BjXD5nXghJm3ZJ8Xrus4IjGCGA10RzKzyyAPvy9/As9mI0l3TmOz7CY kFxGI8ddmM8YmkhnNyGLYYfIZW5f+rV1KVhKH15jyhNRilHEPxyodSjkfgggQ0TCq625 tNVYQQWb8WiY0XZKaXcXk82COCvlLdkV4tPI48V8lMxFQ43HXb9MlXiv5fEB1kUTBzHi Kk4B+Omr31mJQRpSN2eWekDPjVpAHI+vAEvALAqoZd1I5ZHWMNUEXopwn9Xy9dloqPfi qSfw== X-Gm-Message-State: AHYfb5ijzg/Cx08k8B6PFgP/RLz+JAwS7AudKeNeeVsWd0pk0888QFWY O7XwWN2dbZHc2572SslWHWWaQzcoHQ== X-Received: by 10.80.184.34 with SMTP id j31mr978569ede.160.1503937628467; Mon, 28 Aug 2017 09:27:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.129.163 with HTTP; Mon, 28 Aug 2017 09:26:48 -0700 (PDT) In-Reply-To: References: From: Greg Sanders Date: Mon, 28 Aug 2017 12:26:48 -0400 Message-ID: To: Riccardo Casatta Content-Type: multipart/alternative; boundary="f403045c0d841bf6870557d2c41b" X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Aug 2017 16:27:10 -0000 --f403045c0d841bf6870557d2c41b Content-Type: text/plain; charset="UTF-8" Well, if anything my question may bolster your use-case. If there's a heavier chain that is invalid, I kind of doubt it matters for timestamping reasons. /digression On Mon, Aug 28, 2017 at 12:25 PM, Riccardo Casatta < riccardo.casatta@gmail.com> wrote: > > 2017-08-28 18:13 GMT+02:00 Greg Sanders : > >> Is there any reason to believe that you need Bitcoin "full security" at >> all for timestamping? >> > > This is a little bit out of the main topic of the email which is the > savings in bandwidth in transmitting headers, any comment about that? > > > P.S. As a personal experience timestamping is nowadays used to prove date > and integrity of private databases containing a lot of value, so yes, in > that cases I will go with Bitcoin "full security" > > >> >> On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> >>> Hi everyone, >>> >>> the Bitcoin headers are probably the most condensed and important piece >>> of data in the world, their demand is expected to grow. >>> >>> When sending a stream of continuous block headers, a common case in IBD >>> and in disconnected clients, I think there is a possible optimization of >>> the transmitted data: >>> The headers after the first could avoid transmitting the previous hash >>> cause the receiver could compute it by double hashing the previous header >>> (an operation he needs to do anyway to verify PoW). >>> In a long stream, for example 2016 headers, the savings in bandwidth are >>> about 32/80 ~= 40% >>> without compressed headers 2016*80=161280 bytes >>> with compressed headers 80+2015*48=96800 bytes >>> >>> What do you think? >>> >>> >>> In OpenTimestamps calendars we are going to use this compression to give >>> lite-client a reasonable secure proofs (a full node give higher security >>> but isn't feasible in all situations, for example for in-browser >>> verification) >>> To speed up sync of a new client Electrum starts with the download of a >>> file ~36MB containing >>> the first 477637 headers. >>> For this kind of clients could be useful a common http API with fixed >>> position chunks to leverage http caching. For example /headers/2016/0 >>> returns the headers from the genesis to the 2015 header included while >>> /headers/2016/1 gives the headers from the 2016th to the 4031. >>> Other endpoints could have chunks of 20160 blocks or 201600 such that >>> with about 10 http requests a client could fast sync the headers >>> >>> >>> -- >>> Riccardo Casatta - @RCasatta >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >>> >> > > > -- > Riccardo Casatta - @RCasatta > --f403045c0d841bf6870557d2c41b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Well, if anything my question may bolster your use-case. I= f there's a heavier chain that is invalid, I kind of doubt it matters f= or timestamping reasons.

/digression=C2=A0

On Mon, Aug 28, 201= 7 at 12:25 PM, Riccardo Casatta <riccardo.casatta@gmail.com&g= t; wrote:

2017-08= -28 18:13 GMT+02:00 Greg Sanders <gsanders87@gmail.com>:<= br>
Is th= ere any reason to believe that you need Bitcoin "full security" a= t all for timestamping?

This i= s a little bit out of the main topic of the email which is the savings in b= andwidth in transmitting headers, any comment about that?


P.S. As a personal experience timestamping is nowada= ys used to prove date and integrity of private databases containing a lot o= f value, so yes, in that cases I will go with Bitcoin "full security&q= uot;
=C2=A0

On Mon, Aug 28, 2017 at= 11:50 AM, Riccardo Casatta via bitcoin-dev <bitcoin= -dev@lists.linuxfoundation.org> wrote:
Hi everyone,

the Bitcoin headers are probably the most condensed and important p= iece of data in the world, their demand is expected to grow.

=
When sending a stream of continuous block headers, a common case= in IBD and in disconnected clients, I think there is a possible optimizati= on of the transmitted data:
The headers after the first could avo= id transmitting the previous hash cause the receiver could compute it by do= uble hashing the previous header (an operation he needs to do anyway to ver= ify PoW).
In a long stream, for example 2016 headers, the savings= in bandwidth are about 32/80 ~=3D 40%
without compressed headers= 2016*80=3D161280 bytes
with compressed headers 80+2015*48=3D9680= 0 bytes

What do you think?


In OpenTimestamps calendars we are going to use this compr= ession to give lite-client a reasonable secure proofs (a full node give hig= her security but isn't feasible in all situations, for example for in-b= rowser verification)
To speed up sync of a new client Electrum st= arts with the download of a file=C2=A0~36MB containing the first 477= 637 headers.=C2=A0
For this kind of clients could be useful a com= mon http API with fixed position chunks to leverage http caching. For examp= le /headers/2016/0 returns the headers from the genesis to the 2015 header = included while /headers/2016/1 gives the headers from the 2016th to the 403= 1.
Other endpoints could have chunks of 20160 blocks or 201600 su= ch that with about 10 http requests a client could fast sync the headers<= font color=3D"#888888">

--
Riccardo Casatta - @RCasatta

_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev





<= /div>--
Riccardo Casatta - @RCasatta

--f403045c0d841bf6870557d2c41b--