From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4FCB92B00 for ; Sun, 21 Apr 2019 09:13:17 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8E4D685B for ; Sun, 21 Apr 2019 09:13:14 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id a3so2670269wrx.0 for ; Sun, 21 Apr 2019 02:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=SJJ0NWvCLHtwb0Rlto8tvWjjD32ugYg/01NnKKWDPpo=; b=tAkePUpbPrJGdiwM5MlepTvZJuzUt4TJZSZ0rFJlZUGgukL/Pig0cL9RprVr9QRmwS Lx5eyPZd1WFESZOT3Q5YLzSwiKsr8TdLMYV81Y+xqgpcbmbxgNkpHFWFqvH8GZkYwPt5 uUXKlFOLkFjXsoR5nO+DKdnRThas/idFcLmmwxQ1XNx3iwQJiYS3HzGdrEUa8T+SNZnm k4owJv/7OTqOqvwZACpZiZN3wf+jNz03KfdaEgjrRSErbxyUcBkLH0fq9U6y89tkffjZ L94P6UajR7Kwvqk6SSr5EjhePGL6AIu7NjEylVh3XGLjWUG9B4dMssDnsCpfZObq0ETF L0jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=SJJ0NWvCLHtwb0Rlto8tvWjjD32ugYg/01NnKKWDPpo=; b=nUzRivUa0IpaQqYsXHdUcEQ0Jiz/ILp9gdotMVu82R2NgGcYFOAqrUDK12ziCRy5NW F3rdv9w+bJ1e9XiZhH+CG6mqtMFNrilQJsFGQdypfM28MukKWu642KCWPNyVuXJgurYA QrHgzI7M83vXtl3WPLNp8MdOpBZPnxH+ye6mX1bDx7nKE6TZTVMP0YApfWgkGlmWPa/v TQ8pyKa1jUtPpCQJ5rDI/gUvn5nb7SM58vc/phBPy7iqo0Vb+W+7BRs/vrTTsrHdocYO MaFQrhJk89xoBcGMDTcIg+Y+Nhl1+zyrl5ABsh84Cx+M1edXOsbUMdEdGQUuNHXIzekM vAIw== X-Gm-Message-State: APjAAAUjryJ96IbzgT0W+sv3v6+2x+PcqfrxUq/GJVdHkAXBrYhtg8VI QXGPezMP+bk91p1v1C3GFB0RSlpip5CHjpbFuuY= X-Google-Smtp-Source: APXvYqx8k4V8Z5MDMfHVjXPppFriJTFuQZWHEx7qmi/BYaAU0CXwm98xxWWrfCwYWLEYFrpb2WHFMz0gqkuwdTPwUbE= X-Received: by 2002:adf:e407:: with SMTP id g7mr9043814wrm.47.1555837993097; Sun, 21 Apr 2019 02:13:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ruben Somsen Date: Sun, 21 Apr 2019 11:13:00 +0200 Message-ID: To: ZmnSCPxj Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Mon, 22 Apr 2019 13:33:20 +0000 Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] Improving SPV security with PoW fraud proofs X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2019 09:13:17 -0000 Hi ZmnSCPxj, Allow me to reply to your post in mixed order (fraud proofs first): >But peers can be set up to allow you to hear of all chains while denying y= ou proof of the invalidity of some UTXO. I don't believe this is fundamentally different. In either scenario you end up on the wrong chain if all your peers are lying to you. One happens by omission of a fraud proof, while the other happens by omission of a valid longest chain. >This is precisely the "data unavailability claim" that shot down the previ= ous fraud proofs The "data unavailability" issue I was referring to, and which I believe is the reason why fraud proofs were abandoned, is the following: - Alice downloads a block with her full node, but the block is incomplete (e.g. a transaction is missing). - Alice reports this to Bob's SPV fraud proof client, who verifies this by requesting the transaction from the network. - If Bob can't download it, he rejects the block. - If Bob can download it, either Alice was malicious, or a miner was temporarily withholding the data. - Since Bob can't be certain Alice was being malicious, Bob can't ban her, which results in a DoS vector where SPV fraud proof clients can be forced to download all blocks. We circumvent the data unavailability problem here completely, since we are only questioning the validity of blocks which are involved in a fork (expensive and/or rare), and we are simply always downloading them in full. If my arguments above hold up, we can use fraud proof commitments as described in segwit BIP141 [0] instead of UTXO set commitments, which seems like the more elegant way to achieve the desired outcome. >Perhaps in combination with BIP157/158 it may be possible, if the filters = contain UTXO spends and a BIP158 filter was committed to on-chain. Then a p= roof of absence could be done by revealing all the BIP158 filters from the = UTXO creation to the block being validated, as well as the blocks whose BIP= 158 filters matched the UTXO and revealing that no, they actually do not sp= end the UTXO. Yes, I mentioned something similar to Laolu, but it does seem computationally expensive to run every input in a block through the filter of every past block. The fact that BIP157/158 can function without commitments is also why I suspected we may not necessarily need UTXO set commitments. >UTXO sets can only be validated by actually running the entire blockchain,= i.e. fullnoding. It seems to me you can validate uncommitted UTXO sets by comparing them. Download and compare UTXO set hashes from multiple peers. If they disagree on a certain block, download that block and the relevant merkle path(s) from the previous block's UTXO set, and then verify who is right. Ban the peer who lied. Note that unlike fraud proofs, it is not possible to lie by omission, but it does assume one of your peers is honest. Of course this does nothing to dispute your earlier point that this may not be all that efficient (e.g. full nodes keeping merkle paths of all prior states). >What BIP157 does is summarize data that is within a block, thus validating= them can be done simply by downloading the block in question. >UTXO sets summarize data in the entire blockchain, hence proper validation= requires downloading the entire blockchain. Thus it cannot be a comparison point. It's still possible to lie by omission. Let's say a miner spends some coins in block N, and spends the exact same coins again in block N+1, making block N+1 invalid. If the filter for block N is maliciously constructed, you won't notice the spend in block N, causing you to think block N+1 is valid. In short, you're still relying on one of your peers to give you a correct filter. If all your peers lie, you can always be deceived. >Tangentially, we cannot just magically commit to anything on the blockchai= n. [...] if you are adding new information to be committed, that may increa= se the resource usage of fullnodes. [...] This is probably still better tha= n BIP37 but we should still be aware the additional load on fullnodes. I agree with all this. To summarize, this is my current understanding of our options for enabling light clients to verify a single block in isolation: 1. UTXO set commitments (complex, more resource usage to full nodes) 2. BIP157/158 commitments (expensive for clients to check all filters to get exclusion proofs) 3. BIP141 fraud proof commitments (assumes fraud proofs will be passed on to the SPV client) The debate is still open on whether the options above can be done without actually committing them into blocks via a soft fork. My current hunch is "yes" for 1 and 2, and "no" for 3, which would be unfortunate, because 3 currently seems to me like the more elegant solution. -- Ruben Somsen [0] https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Compact_= fraud_proof_for_SPV_nodes >UTXO sets can only be validated by actually running the entire blockchain,= i.e. fullnoding. >What BIP157 does is summarize data that is within a block, thus validating= them can be done simply by downloading the block in question. UTXO sets summarize data in the entire blockchain, hence proper validation requires downloading the entire blockchain. Thus it cannot be a comparison point. > > This makes no sense > > or you trust that every peer you have is not omitting the proof. > > It's the latter, you trust every peer you have is not omitting the > proof. It requires one honest peer. The reason this is acceptable is > because you're already making that assumption. If none of your peers > are honest, you have no guarantee of hearing about the chain with the > most PoW. But peers can be set up to allow you to hear of all chains while denying you proof of the invalidity of some UTXO. This is precisely the "data unavailability claim" that shot down the previous fraud proofs (i.e. absence of proof is not proof of absence, and proof of UTXO validity was defined by proof of absence of any intervening spend of the UTXO). Perhaps in combination with BIP157/158 it may be possible, if the filters contain UTXO spends and a BIP158 filter was committed to on-chain. Then a proof of absence could be done by revealing all the BIP158 filters from the UTXO creation to the block being validated, as well as the blocks whose BIP158 filters matched the UTXO and revealing that no, they actually do not spend the UTXO. -- Tangentially, we cannot just magically commit to anything on the blockchain= . Header blocks commit to block data and commit to some other header block. All those header blocks and the block data need to be stored and transmitted over the network somehow, even though they are "only" being committed to. Thus, if you are adding new information to be committed, that may increase the resource usage of fullnodes. So if UTXO set commitments, or utreexo commitments, or BIP158 filter digests, etc. are committed to in the coinbase, they have to be stored somehow in fullnodes the entire UUTXO set, or the actual utreexo structure, or the actual BIP158 filter, etc. at each block. Otherwise it would be pointless to store those commitments since it would not be possible to somehow acquire the data being committed to after-the-fact. This is probably still better than BIP37 but we should still be aware the additional load on fullnodes. Regards, ZmnSCPxj On Sat, Apr 20, 2019 at 6:45 AM ZmnSCPxj wrote: > > Good morning Ruben, > > > Hi ZmnSCPxj, > > > > > There is no safe way to use UTXO sets without identifying who is tell= ing you those sets are valid, or making it expensive to lie > > > The first option requires trust and is weaker than SPV, the second re= quires committing to a proof-of-work > > > > Olaoluwa Osuntokun's BIP157 manages to function without a commitment: > > "If the client receives conflicting filter headers from different > > peers for any block and filter type, it SHOULD interrogate them to > > determine which is faulty." > > > > I am wondering if the same logic can be applied to UTXO sets or the > > fraud proofs I just described. > > UTXO sets can only be validated by actually running the entire blockchain= , i.e. fullnoding. > > What BIP157 does is summarize data that is within a block, thus validatin= g them can be done simply by downloading the block in question. > > UTXO sets summarize data in the entire blockchain, hence proper validatio= n requires downloading the entire blockchain. > Thus it cannot be a comparison point. > > > > > This makes no sense > > > or you trust that every peer you have is not omitting the proof. > > > > It's the latter, you trust every peer you have is not omitting the > > proof. It requires one honest peer. The reason this is acceptable is > > because you're already making that assumption. If none of your peers > > are honest, you have no guarantee of hearing about the chain with the > > most PoW. > > But peers can be set up to allow you to hear of all chains while denying = you proof of the invalidity of some UTXO. > This is precisely the "data unavailability claim" that shot down the prev= ious fraud proofs (i.e. absence of proof is not proof of absence, and proof= of UTXO validity was defined by proof of absence of any intervening spend = of the UTXO). > > Perhaps in combination with BIP157/158 it may be possible, if the filters= contain UTXO spends and a BIP158 filter was committed to on-chain. > Then a proof of absence could be done by revealing all the BIP158 filters= from the UTXO creation to the block being validated, as well as the blocks= whose BIP158 filters matched the UTXO and revealing that no, they actually= do not spend the UTXO. > > -- > > Tangentially, we cannot just magically commit to anything on the blockcha= in. > Header blocks commit to block data and commit to some other header block. > All those header blocks and the block data need to be stored and transmit= ted over the network somehow, even though they are "only" being committed t= o. > Thus, if you are adding new information to be committed, that may increas= e the resource usage of fullnodes. > > So if UTXO set commitments, or utreexo commitments, or BIP158 filter dige= sts, etc. are committed to in the coinbase, they have to be stored somehow = in fullnodes the entire UUTXO set, or the actual utreexo structure, or the = actual BIP158 filter, etc. at each block. > Otherwise it would be pointless to store those commitments since it would= not be possible to somehow acquire the data being committed to after-the-f= act. > > This is probably still better than BIP37 but we should still be aware the= additional load on fullnodes. > > Regards, > ZmnSCPxj