From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 53F9F1048 for ; Thu, 17 May 2018 17:16:59 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ot0-f172.google.com (mail-ot0-f172.google.com [74.125.82.172]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id AA351A3 for ; Thu, 17 May 2018 17:16:58 +0000 (UTC) Received: by mail-ot0-f172.google.com with SMTP id n1-v6so5903206otf.7 for ; Thu, 17 May 2018 10:16:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=coryfields-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=T3aUL2ij42B7cfQFv47CINhK+e2M0yQQq4tVzaWixL4=; b=T3NCTYnX1dJxJTWkdbE2oDiWIiR6w+FZHbBn4zAiA+/NGaglPOQjA70d50Ws+5Gz35 rcJrK8qhCTK3WDWtgQ6s6zVzNyB/uaUF/TfDsp1IjOzuD/EtT63PYAGIvsIatnmoIaoB znZF6NkcsLxC2zhYf9t4pgHCl9qxsqGjh7/ZXTSXH4KmZ72EzXiVKCW7WXWZRDAceZAs 4gDpLmepKuExuQPnYsrxl4v5SHIqDut15NlBiGdK5Y8VZo6GCKrR7wcBBBvS6NtaOQ9U htxqbHiPd2FIt1geyxFL5nxLno0oXVt//QzEd3gRa46UPMBP/12PNcg41UH3Fi3Oq87Y Qtuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=T3aUL2ij42B7cfQFv47CINhK+e2M0yQQq4tVzaWixL4=; b=SBae3dPUszoDN4ONHaRSh0WawLXiNsleZsWNJs1ug6qy/mdlrnk3/UXlTC3hHHhd5/ HdIEIRfio9EnYwX+18FzVt41mUQkPhZ9Rj3kKj+c2E0A5x0GEJf6+lu+FP7Q5Mua7ng9 T73xxFOKQKEk2QdU6HszGe3uGB6NEHLnKU+7tzO/KiwZL+wBwkkn4+yscrID8ieOXsBe MGJA+cDW1+nHdbVDnaRruGem8qkjveBw1O20+xDliLeLgfjhmThUD34Z35NAPnDd8F1l Azlj5xt5OSQfZvSdHv4rYoBcJkB6l1ncxOzlmqmdJ/HvURzObyQLNG/4cAqAqWjQUTwz lehw== X-Gm-Message-State: ALKqPwe+92zOIRNJBR4XwRc9ccgAQZj5aG2YPtHYPmXIbudFfaHnHKKM +8dCKnPqxHMEX6lVle3O3JOjSAoGKTlh52ZMJ+l+jQ== X-Google-Smtp-Source: AB8JxZrLWVGozlFvVDKNB/GOlTGI3SCyusBKb7VzfyF0LUJgjJRKj7hVlCAwhsWVbZlhKOMzm2SnoI2Ix8qYZkW+Biw= X-Received: by 2002:a9d:48c6:: with SMTP id a6-v6mr4003653otj.359.1526577417832; Thu, 17 May 2018 10:16:57 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: Reply-To: lists@coryfields.com From: Cory Fields Date: Thu, 17 May 2018 13:16:46 -0400 Message-ID: To: Gregory Maxwell Content-Type: multipart/alternative; boundary="000000000000b66287056c6a0038" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] UHS: Full-node security without maintaining a full UTXO set X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 17:16:59 -0000 --000000000000b66287056c6a0038 Content-Type: text/plain; charset="UTF-8" Matt: That's a good point. I'll do up a chart comparing utxo size at each block, as well as comparing over-the-wire size for each block. I think the period of coalescing earlier this year should be a good example of what you're describing. Greg: heh, I was wondering how long it would take for someone to point out that I'm cheating. I avoided using the word "compression", mostly to side-step having the discussion turning into reworking the wire serialization. But you're absolutely right, the de-duping benefits are independent of the UHS use-case. Cory On Thu, May 17, 2018, 12:56 PM Gregory Maxwell wrote: > On Wed, May 16, 2018 at 4:36 PM, Cory Fields via bitcoin-dev > wrote: > > Tl;dr: Rather than storing all unspent outputs, store their hashes. > > My initial thoughts are it's not _completely_ obvious to me that a 5% > ongoing bandwidth increase is actually a win to get something like a > 40% reduction in the size of a pruned node (and less than a 1% > reduction in an archive node) primarily because I've not seen size of > a pruned node cited as a usage limiting factor basically anywhere. I > would assume it is a win but wouldn't be shocked to see a careful > analysis that concluded it wasn't. > > But perhaps more interestingly, I think the overhead is not really 5%, > but it's 5% measured in the context of the phenomenally inefficient tx > mechanisms ( https://bitcointalk.org/index.php?topic=1377345.0 ). > Napkin math on the size of a txn alone tells me it's more like a 25% > increase if you just consider size of tx vs size of > tx+scriptpubkeys,amounts. If I'm not missing something there, I think > that would get in into a very clear not-win range. > > On the positive side is that it doesn't change the blockchain > datastructure, so it's something implementations could do without > marrying the network to it forever. > --000000000000b66287056c6a0038 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Matt: That's a good point. I'll do up a chart com= paring utxo size at each block, as well as comparing over-the-wire size for= each block. I think the period of coalescing earlier this year should be a= good example of what you're describing.

Greg: heh, I was wondering how long it would take for someon= e to point out that I'm cheating. I avoided using the word "compre= ssion", mostly to side-step having the discussion turning into reworki= ng the wire serialization. But you're absolutely right, the de-duping b= enefits are independent of the UHS use-case.

Cory

On Thu, May 17, 2018, 12:56 PM Gregory Maxwell <greg@xiph.org&g= t; wrote:
On Wed, May 16, 2018 at 4= :36 PM, Cory Fields via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org= > wrote:
> Tl;dr: Rather than storing all unspent outputs, store their hashes.
My initial thoughts are it's not _completely_ obvious to me that a 5% ongoing bandwidth increase is actually a win to get something like a
40% reduction in the size of a pruned node (and less than a 1%
reduction in an archive node) primarily because I've not seen size of a pruned node cited as a usage limiting factor basically anywhere. I
would assume it is a win but wouldn't be shocked to see a careful
analysis that concluded it wasn't.

But perhaps more interestingly, I think the overhead is not really 5%,
but it's 5% measured in the context of the phenomenally inefficient tx<= br> mechanisms ( https://bitcoi= ntalk.org/index.php?topic=3D1377345.0 ).
Napkin math on the size of a txn alone tells me it's more like a 25% increase if you just consider size of tx vs size of
tx+scriptpubkeys,amounts.=C2=A0 If I'm not missing something there, I t= hink
that would get in into a very clear not-win range.

On the positive side is that it doesn't change the blockchain
datastructure, so it's something implementations could do without
marrying the network to it forever.
--000000000000b66287056c6a0038--