From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D763AC13 for ; Thu, 17 May 2018 20:45:35 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f171.google.com (mail-ua0-f171.google.com [209.85.217.171]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 4A1D8196 for ; Thu, 17 May 2018 20:45:35 +0000 (UTC) Received: by mail-ua0-f171.google.com with SMTP id e8-v6so3881595uam.13 for ; Thu, 17 May 2018 13:45:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=24UiwFUbM7XMVy8WjuvOwpxR7/xfJAJ0sxnLt0kD8VU=; b=i50mxnUejnWmPM15vLIFkKOaOsrf3nCoqyNrANQsv8Bnpay3xKGRnh5K34axjmSrkL LqPkRFwDLoYVfobU1VoUa8CHFgkhJ10pfA0BeHBigqJu9Ppk4ZhAenFxmdvQOqEXu5bN YPzBW+tnux6rI9Iu3Kc5elds3wzcAXO0k8J83ip0wHJN6cuWHcJnaHKF05JNXCL4vE/Z CmDQ4CLhKh4fIHu2u3BhNS01KOYacGN1LxY6YWI/7EsWyu7//gTyGlwXmiN55GBfGw23 6ck8FmWTlaMjj4masNQsdVN2cCqSQGEX5sGfed8WY6D9gdJwKcVAxeDG+iwpn+/Kj5BV 1qpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=24UiwFUbM7XMVy8WjuvOwpxR7/xfJAJ0sxnLt0kD8VU=; b=C5NH3KlGCse/4X8YV/zNjGK5vATzKjyldOG6nT2fcxZCHt72whbzx65YLn1JAHRbHA zrii+yJ6g2iODvJfkX1yrgP0VNXNkLKvIwW5sHXlSG6VoWUsMB7Iq+dnDrMXpdRv8RdI Hr51q02U7pnKSg1ruIa8yb3y4AZEijO5ZMjD4SBYadtZr0Z+UrCPB/bgVzNBC6ZAr9Iy LvVejOQqB1P7gmSYTCr8c3sO4A8qNtSibCjOFm5FDvsfdJX0MwbXtoZAxvCWDXA6ye0G 14yFCMthv0vdbi50UI/70YQVyItJyiV0V0AIZl7O2nHNtNuh1RbCbKoORDE6wc5qSy+b CjwQ== X-Gm-Message-State: ALKqPwd6Oryi9FRQMLvkRN7iOP1GOvhWpI8p0k46uSFGnTCl28+KJkr4 0CnQSq9t4j850/g8v4JIW+wWWCfLHiX4jlUVtaw= X-Google-Smtp-Source: AB8JxZqd/ZLZs62NPb12Wu87OnCArQvv/mjOdaBR9QM8V13ce//nF63nuRmk5wYV5RpqGAo2t6pD6eJ5pxb8HVVCyTs= X-Received: by 2002:ab0:1b6c:: with SMTP id n44-v6mr5432827uai.194.1526589934124; Thu, 17 May 2018 13:45:34 -0700 (PDT) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 10.103.81.132 with HTTP; Thu, 17 May 2018 13:45:33 -0700 (PDT) In-Reply-To: References: <22d375c7-a032-8691-98dc-0e6ee87a4b08@mattcorallo.com> From: Gregory Maxwell Date: Thu, 17 May 2018 20:45:33 +0000 X-Google-Sender-Auth: gXB2-geo9df7a8mKli11RtJ0XPs Message-ID: To: Jim Posen Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 20:45:36 -0000 On Thu, May 17, 2018 at 8:19 PM, Jim Posen wrote: > In my opinion, it's overly pessimistic to design the protocol in an insecure > way because some light clients historically have taken shortcuts. Any non-commited form is inherently insecure. A nearby network attacker (or eclipse attacker) or whatnot can moot whatever kind of comparisons you make, and non-comparison based validation doesn't seem like it would be useful without mooting all the bandwidth improvements unless I'm missing something. It isn't a question of 'some lite clients' -- I am aware of no implementation of these kinds of measures in any cryptocurrency ever. The same kind of comparison to the block could have been done with BIP37 filtering, but no one has implemented that. (similarly, the whitepaper suggests doing that for all network rules when a disagreement has been seen, though that isn't practical for all network rules it could be done for many of them-- but again no implementation or AFAIK any interest in implementing that) > If the > protocol can provide clients the option of getting additional security, it > should. Sure, but at what cost? And "additional" while nice doesn't necessarily translate into a meaningful increase in delivered security for any particular application. I think we might be speaking too generally here. What I'm suggesting would still allow a lite client to verify that multiple parties are offering the same map for a given block (by asking them for the map hash). It would still allow a future commitment so that lite client could verify that the hashpower they're hearing from agrees that the map they got is the correct corresponding map for the block. It would still allow downloading a block and verifying that all the outpoints in the block were included. So still a lot better than BIP37. What it would not permit is for a lite client to download a whole block and completely verify the filter (they could only tell if the filter at least told them about all the outputs in the block, but if extra bits were set or inputs were omitted, they couldn't tell). But in exchange the filters for a given FP rate would be probably about half the current size (actual measurements would be needed because the figure depends on much scriptpubkey reuse there is, it probably could be anywhere between 1/3 and 2/3rd). In some applications it would likely have better anonymity properties as well, because a client that always filters for both an output and and input as distinct items (and then leaks matches by fetching blocks) is more distinguishable. I think this trade-off is at leat worth considering because if you always verify by downloading you wash out the bandwidth gains, strong verification will eventually need a commitment in any case. A client can still partially verify, and can still multi-party comparison verify. ... and a big reduction in filter bandwidth Monitoring inputs by scriptPubkey vs input-txid also has a massive advantage for parallel filtering: You can usually known your pubkeys well in advance, but if you have to change what you're watching block N+1 for based on the txids that paid you in N you can't filter them in parallel. > On the general topic, Peter makes a good point that in many cases filtering > by txid of spending transaction may be preferable to filtering by outpoint > spend, which has the nice benefit that there are obviously fewer txs in a > block than txins. This wouldn't work for malleable transactions though. I think Peter missed Matt's point that you can monitor for a specific transaction's confirmation by monitoring for any of the outpoints that transaction contains. Because the txid commits to the outpoints there shouldn't be any case where the txid is knowable but (an) outpoint is not. Removal of the txid and monitoring for any one of the outputs should be a strict reduction in the false positive rate for a given filter size (the filter will contain strictly fewer elements and the client will match for the same (or usually, fewer) number). I _think_ dropping txids as matt suggests is an obvious win that costs nothing. Replacing inputs with scripts as I suggested has some trade-offs.