From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4427DDC6 for ; Mon, 29 Feb 2016 10:29:12 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from server3 (server3.include7.ch [144.76.194.38]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 6D4CDCB for ; Mon, 29 Feb 2016 10:29:11 +0000 (UTC) Received: by server3 (Postfix, from userid 115) id 0F1E92D00754; Mon, 29 Feb 2016 11:29:10 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, FSL_HELO_NON_FQDN_1 autolearn=ham version=3.3.1 Received: from Jonass-MacBook-Pro.local (cable-static-140-182.teleport.ch [87.102.140.182]) by server3 (Postfix) with ESMTPSA id 4FD8D2D006E8 for ; Mon, 29 Feb 2016 11:29:09 +0100 (CET) To: Bitcoin development mailing list From: Jonas Schnelli X-Enigmail-Draft-Status: N1110 Message-ID: <56D41D71.80503@jonasschnelli.ch> Date: Mon, 29 Feb 2016 11:29:05 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: [bitcoin-dev] Fast bootstrapping with a pre-generated UTXO-set database X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Feb 2016 10:29:12 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi I’ve been thinking around a solution to reduce nodes bootstrap time (IBD) as well as a way to reduce the amount of bandwidth/network usage per node. Not sure if this idea was/is already discussed, haven’t found anything in a quick research. ==Title== Fast bootstrapping with a pre-generated UTXO-set database. ==Abstract== This documents describes a way how bitcoin nodes can bootstrap faster by loading a pre-generated UTXO-set datafile with moderate reduction of the security model. ==Specification== Bitcoin-core or any other full node client will need to provide a feature to "freeze" the UTXO-set at a specified height (will require a reindex). The frozen UTXO-set – at a specific height – will be deterministic linearized in a currently not specified data-serializing-format. Additionally, a serialized form of the current chain-index (chain containing all block-headers) up to the specified height will be appended to the pre-generated UTXO-set-datafile. The datafile will be hashed with a double SHA256. The corresponding hash will be produced/reproduced and signed (ECDSA) by a group of developers, ideally the same group of developers who are also signing deterministic builds (binary distribution). Full node client implementations that supports bootstrapping from a pre-generated UTXO-set, need to include... 1.) a set of pubkeys from trusted developers 2.) the hash (or hashes) of the pre-generated UTXO-set-datafile(s) 3.) n signatures of the hash(es) from 2) from a subset of developers defined in 1) To guarantee the integrity of developers pubkeys & signatures, methods like the current gitian build, used in bitcoin-core, must be used. New nodes could download a copy of the pre-generated UTXO-set, hash it, verify the hash against the allowed UTXO-sets, verify the ECDSA signatures from various developers, and continue bootstrapping from the specified height if the users accepts the amount of valid signatures . Sharing of the pre-generated UTXO-set can be done over CDNs, bit-torrent or any other file hosting solution. It would also be possible to extend the bitcoin p2p layer with features to distribute/share a such pre-generated UTXO-set, in chunks and with the according hashes to detect invalidity before downloading the whole content (but would probably end up in something very similar to bit-torrent). - ---------------------- -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJW1B1wAAoJECnUvLZBb1PsqzsP/iSdvyhUzy+BZVSZbKXNjk5P 2vrtirI6NvKQd8hHbrcFeLfyswzYc2JWRnX8sATlauIS0pYdr97JriwUGlvxvNrY iVTDdf8MIVu8zScLQtJbMatpMvsewtqQEidn/yxWIhiCg4G2T5DZmlBU6O4XIKR6 5aPHElGOKZ15EWGHBG7z4owj3MiOaxhD9q5erBbfLPpcm08o6XAv5miqmGnnn3zh gocg4Gxs6iDygh3b2dCJFwWIVPxF6UVJhyjv2kLZUmEHT2Y2QvdGcLIIewcWHDze kgoZYmOEowujCbmeJ+LBwgOI0c1N6L/ciomPBne7ILmK4LyUEzyMLJKNYf/sZ8vI sVlmwZwZZLfILC7mzMAM0pfj99IOW680WHch9v31lWFlxW/bLvLqAO7n3acQuD6s xCZN2nAhmWC8FnMFxqB3EUz0lX8giV3qRJZjbQMS+ZrngYkAmVv2bAsoLndqf6MO l9W8B+ICg1KZLGIOF2pUrInpkB6gUALDFnypV4CeIVdeqtk5l4LnCHK6c4++Hl5n Bv5HQ/wTgKKNFtHBEJpWyYWvAjfFtgUZUKblR+Bh+D7/Gte1ehiYd02KYD4ds9Y4 3gfO8YbAz/I14Yuh2bIlvVKPWnLQBwL5BBioBfvmhV/r6rEpzWvB7H6Fmi1c759l VlL0GiUV8ar2LlFhEmWk =lZSy -----END PGP SIGNATURE-----