From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id DDD98F12 for ; Sun, 3 Jun 2018 00:28:35 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f176.google.com (mail-ua0-f176.google.com [209.85.217.176]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 76D02705 for ; Sun, 3 Jun 2018 00:28:35 +0000 (UTC) Received: by mail-ua0-f176.google.com with SMTP id i3-v6so19806851uad.4 for ; Sat, 02 Jun 2018 17:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=0fXtcGsoxR/gIbP3PDkFxq482pfFaOqgcq4MT7tY9Xc=; b=eJC8CuhrXni/6yNQ4CsUyKOKCjWKBa8pgRtaqH8Dp4pY8xD+ODEN+VDAfNkW4jNSLH RM2QklBSr+XGgHqnchWMbZMeUDqEdnSd6RQvOGvdvozcYHHT2OFPO3M07gGqyWet1JDr Desm3L/GWePzugm8BoZSIBXgmm+Oquivh5RkMuJ8ILzPVDKGyfNvhJ36KiGUCX9ft5Az cybvl4tS670cHqE6/wneUcbMR3dckRyHQyydJ2QrZEAg/BVZ57ZrJsW/ZYPOQWuIZiNA Bjq9RA4I5tSIq9zNukWmLsnN2IQ9Pz3hPnfIg8eMYpX8rgcBrAoVIbnT2RSIRXQsCQws HYJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=0fXtcGsoxR/gIbP3PDkFxq482pfFaOqgcq4MT7tY9Xc=; b=lGSTe+S8gQOCdbM6As4xJ5zA1ieEoGDsohFYuTs9mALwREgLA2t+CUrpplAUkCgIv0 9nTOcx4eerJdL8GZ2yg0gkfJXdNeXMa46ueoukEmsCpLvdcrknSMHzPXgVClHiGl7p50 gJ9yRzWWj7B0Fd0PJCWwdETFTSVl8MSKbYBIL2xy7leVudJqLcpNU07sKkPR616+lL1K URYxzQ7MKPhHIq4inBIpDI1TJ4M+NIoe1N6uKG2vIn1bp3kDUIJxZvD+wO//NxmyB2TW x91gIcX2ATWj/J+cs15rEoxP+qGSZOO5lC9rZq82Wg1fZy5pdbkye9dbdMvXSiAMnOrF pg3g== X-Gm-Message-State: ALKqPwcjw1TUHIxY6K3WAr0TPt65E7ubqgK1USAcB52P9GlzNirAGRFQ w2qDozOPPNI4Z2Ema0A9x2m1UMiLqtKxavit0GQ= X-Google-Smtp-Source: ADUXVKKKy2IxMlYT3dgPsiWa3Jy7ARafTA6hWSxNSdIG+TC5V9JOHCaxgStri7GSxNmRFmgQ9bsdn7T7ez3wu3EGkiQ= X-Received: by 2002:a9f:3b06:: with SMTP id i6-v6mr10727001uah.75.1527985714583; Sat, 02 Jun 2018 17:28:34 -0700 (PDT) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 2002:a67:5193:0:0:0:0:0 with HTTP; Sat, 2 Jun 2018 17:28:33 -0700 (PDT) In-Reply-To: <343A3542-3103-42E9-95B7-640DFE958FFA@gmail.com> References: <7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com> <20180602124157.744x7j4u7dqtaa43@email> <343A3542-3103-42E9-95B7-640DFE958FFA@gmail.com> From: Gregory Maxwell Date: Sun, 3 Jun 2018 00:28:33 +0000 X-Google-Sender-Auth: zzB72PFWoM05r4Z6GElRZpJVfjA Message-ID: To: Tamas Blummer , Bitcoin Protocol Discussion Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2018 00:28:36 -0000 On Sat, Jun 2, 2018 at 10:02 PM, Tamas Blummer via bitcoin-dev wrote: > Years of experience implementing wallets with BIP 37 pretty much us that all these filter things are a total waste of time. BIP37 use is nearly dead on the network-- monitor your own nodes to see the actual use of the filters: it's very low. I see under average of 1 peer per day using it. Moreover the primary complaint from users about BIP37 vs the alternatives they're choosing over it (electrum and web wallets) is that the sync time is too long-- something BIP158 doesn't improve. So if we were going to go based on history we wouldn't bother with on P2P at all. But I think the history's lesson here may mostly be an accident, and that the the non-use of BIP37 is due more to the low quality and/or abandoned status of most BIP37 implementing software, rather than a fundamental lack of utility. Though maybe we do find out that once someone bothers implementing a PIR based scanning mechanism (as electrum has talked about on and off for a while now) we'll lose another advantage. BIP37 also got a number of things wrong-- what went into the filters was a big element in that (causing massive pollution of matches due to useless data), along with privacy etc. This kind of approach will have the best chances if it doesn't repeat the mistakes... but also it'll have the best chances if it has good security, and getting SPV- equivalent security will require committing the filters, but committing them is a big step because then the behaviour becomes consensus normative-- it's worth spending a few months of extra iteration getting the design as good as possible before doing that (which is what we've been seeing lately).