From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0D4283EE for ; Tue, 21 Feb 2017 22:00:25 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-it0-f41.google.com (mail-it0-f41.google.com [209.85.214.41]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8A6601B5 for ; Tue, 21 Feb 2017 22:00:24 +0000 (UTC) Received: by mail-it0-f41.google.com with SMTP id d9so36552928itc.0 for ; Tue, 21 Feb 2017 14:00:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bittorrent-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=DTYiRPoLkurICjMF+UofCZy8f0YDWagRiw0LZaN2Gvg=; b=UXamKRaLooK1f3ln58f7bo8Sgj7aCZ9Yfp8oNyjElWxAp6yLruoJb+pfPBKIOT5JbT WvRa862tUF1IDaugZ2CGODW8dhPQzGTppTMXnNwXpd9ozoUlYPvBiHaWojvkPLiMA7wE WjWJVdo+xq2bCMZGdVBxaDmyzh5PMGOZ2gsMhH+cDw4hqpMlTrttG8j0AsiWL+BftN7z +RvXz78W3NzYZT1Cg85lPU8k2rt19OZ/tGzJ7rL2nte6KlbEzOIFsQdPJlY5AEonjDhM +5LS08pV9ffWJxYyfkuOW14bZOklMUG3O9aETzR6RUAAUqREsxA6669WYCMCCEin98k2 ct+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=DTYiRPoLkurICjMF+UofCZy8f0YDWagRiw0LZaN2Gvg=; b=Ju5BBOKiaxYrcnS12LCnHjyDwEEKtTZhyyKPxOAfhW8hU6koGV+iUUTC+zFTQBW45u KdxgX4t/skZH5d/MoEhix+EdozMNjIz+gRUZAbrM1oKonXp14AnULmq7RLwnZB3a3bdF e8CQMRLIQXdiLGEWpeOucNbzIx7ns8/UBoG5DfBhXhMQNXrqm9Os+KjtMIxdlVGfNIvb BdbxX+GmpXh0zEoWL5msCSBrPVz/ePpZ0WXE/zw52pweJIEV+svY4IOyCjGAkvSI0Squ rLPoKpwCy1ikTXAV2d32j3zSucc1jetodqqMPe3Mht3D5yNFIPZjqZ+McF2i2BSBGfzj J18A== X-Gm-Message-State: AMke39nLYf1awAYNNVLH1JmTU16PiJs9FDDz1O1wy991gKvGpHiV20OKuB10z3KdQk6U4jXyyRQhNT63vt1Be72V X-Received: by 10.36.25.83 with SMTP id b80mr17180887itb.98.1487714423773; Tue, 21 Feb 2017 14:00:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.73.150 with HTTP; Tue, 21 Feb 2017 14:00:23 -0800 (PST) From: Bram Cohen Date: Tue, 21 Feb 2017 14:00:23 -0800 Message-ID: To: Bitcoin Protocol Discussion Content-Type: multipart/alternative; boundary=001a11440182c1b425054911818d X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [bitcoin-dev] Proposal for utxo commitment format X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 22:00:25 -0000 --001a11440182c1b425054911818d Content-Type: text/plain; charset=UTF-8 Here is a Merkle set data structure, whose format may be useful for utxo commitments in Bitcoin blocks. It may also be useful for any other distributed computation which wants an audit trail: https://github.com/bramcohen/MerkleSet This is a fairly straightforward Patricia Trie, with a simple format and a simple reference implementation plus a performance optimized non-reference implementation which is much more cache coherent. It will need to be ported to C and be properly turned before the potential performance gains can be realized though. The clever things which affect the format spec are: It uses blake2s as the internal hash function. This is the fastest hash function to use on 512 bit inputs because blake2b uses a 1024 bit block size. It might make sense to use a hypothetical variant of blake which is optimized for 64 bits with a 512 bit block size, but that hasn't been specified. Sha256 would take advantage of hardware acceleration, but that isn't available everywhere. Two bits of security are sacrificed to include metadata inline which halves the CPU cost of hashing. When one side of a node is empty and the other contains exactly two things the secure hash of the child is adopted verbatim rather than rehashing it. This roughly halves the amount of hashing done, and makes it more resistant to malicious data, and cleans up some implementation details, at the cost of some extra complexity. --001a11440182c1b425054911818d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Here is a Merkle set data structure, whose format may be u= seful for utxo commitments in Bitcoin blocks. It may also be useful for any= other distributed computation which wants an audit trail:

https://github.com/br= amcohen/MerkleSet

This is a fairly straigh= tforward Patricia Trie, with a simple format and a simple reference impleme= ntation plus a performance optimized non-reference implementation which is = much more cache coherent. It will need to be ported to C and be properly tu= rned before the potential performance gains can be realized though.=C2=A0

The clever things which affect the format spec are:=

It uses blake2s as the internal hash function. Th= is is the fastest hash function to use on 512 bit inputs because blake2b us= es a 1024 bit block size. It might make sense to use a hypothetical variant= of blake which is optimized for 64 bits with a 512 bit block size, but tha= t hasn't been specified. Sha256 would take advantage of hardware accele= ration, but that isn't available everywhere.

T= wo bits of security are sacrificed to include metadata inline which halves = the CPU cost of hashing.

When one side of a node i= s empty and the other contains exactly two things the secure hash of the ch= ild is adopted verbatim rather than rehashing it. This roughly halves the a= mount of hashing done, and makes it more resistant to malicious data, and c= leans up some implementation details, at the cost of some extra complexity.=
--001a11440182c1b425054911818d--