From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <tamas.blummer@gmail.com> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 742B8707 for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 28 May 2018 18:28:20 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A76ED6F0 for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 28 May 2018 18:28:19 +0000 (UTC) Received: by mail-wr0-f195.google.com with SMTP id w3-v6so21558826wrl.12 for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 28 May 2018 11:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/c4KXSV0T2lgjnd0ezERRaMgvRHJliSvbKzb7TMEWGs=; b=fAp4wWUGA8Iuiop0jXlACpzeJXdKGFuTzrasCJcJxH7kiDm/NbUNoS1N+VkFkK2pjV 8ya6freM1eRYy/77rEhJ1GgEu9Qm6zQ4Y6IuMvJq5hBMxGvXX4UpywTqZb6JjLfmc4zg DYBTmphkRz3VkPR0mJo2ifW0IXUg5QLOP6zVoZj/OJG5IdyXLQ+5Q5bZ1+E5MPBiMHwK PXKpSfs8k6TNhgI7cOEfIBSIBTPMH8s8dYV1KxUeY9R82jwZoXbEtB67NHnNePApC3Sx CB7jSg2EKuZ8xMt6vLpzF59QalKF0ENumpTjtZ1YWgOqu8Spd+Bn1oGr9kRmpeDFXQNL rONg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/c4KXSV0T2lgjnd0ezERRaMgvRHJliSvbKzb7TMEWGs=; b=RyHqubm8wrx7PYtK/b1Jr/NkKUR0brWrCnoWgDLUmAhsBuAm1Dr0fvUEH3rdzRXRTH 0vJj7IS7exaqcBk02ee7Yn2OAaRh1SowxF6OKNgJgZISoNw5oM97QPk3dONqrfhUIKJB Myq8MoX8RnbCsDmsMuQVcvmWJSOgqZwrFydqJEMntpRto5pHg2vsbtQrB1zWsjoDpyLH gspIqMslE5l7CYbl6mz5qrQ8snGiANsJnWZOiAY/p/LjZHEUluGJp0k2fKeoBPcs00mU LuGFu76L3c/391pX9CKLYs19j+oSg53NyV/3h3gCr2qzmN5UgM7l/etysJS9ixv45stx 3BKg== X-Gm-Message-State: ALKqPweuMVpS5kvZJaYUByjhEicumOqJRJghcQiaEcgqS6fVLe0B3V+E xhTi3t26fN0pE6qNZfPnndA= X-Google-Smtp-Source: AB8JxZom+Zp16nidqtBwbOzdY0BCb6/PglzRCUtDc/KTLnqqbLzJ0F21ZZGLwL8eLWcojMsRU/ZZVQ== X-Received: by 2002:adf:9cc2:: with SMTP id h2-v6mr10417833wre.11.1527532098286; Mon, 28 May 2018 11:28:18 -0700 (PDT) Received: from p200300e38f2e73049c129df00e03e475.dip0.t-ipconnect.de (p200300E38F2E73049C129DF00E03E475.dip0.t-ipconnect.de. [2003:e3:8f2e:7304:9c12:9df0:e03:e475]) by smtp.gmail.com with ESMTPSA id k5-v6sm31375473wrd.13.2018.05.28.11.28.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 May 2018 11:28:17 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Tamas Blummer <tamas.blummer@gmail.com> In-Reply-To: <7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com> Date: Mon, 28 May 2018 20:28:16 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <F87D7069-0FDC-4572-B02B-398A2A455935@gmail.com> References: <d43c6082-1b2c-c95b-5144-99ad0021ea6c@mattcorallo.com> <CAAS2fgRF-MhOvpFY6c_qAPzNMo3GQ28RExdSbOV6Q6Oy2iWn1A@mail.gmail.com> <22d375c7-a032-8691-98dc-0e6ee87a4b08@mattcorallo.com> <CAAS2fgR3QRHeHEjjOS1ckEkL-h7=Na56G12hYW9Bmy9WEMduvg@mail.gmail.com> <CADZtCShLmH_k-UssNWahUNHgHvWQQ1y638LwaOfnJEipwjbiYg@mail.gmail.com> <CAAS2fgQLCN_cuZ-3QPjCLfYOtHfEk=SenTn5=y9LfGzJxLPR3Q@mail.gmail.com> <CADZtCSjYr6VMBVQ=rx44SgRWcFSXhVXUZJB=rHMh4X78Z2eY1A@mail.gmail.com> <CAO3Pvs9K3n=OzVQ06XGQvzNC+Aqp9S60kWM9VRPA8hWTJ3u9BQ@mail.gmail.com> <c23a5346-9f99-44f0-abbf-d7e7979bf1d8@gmail.com> <CAO3Pvs_MA4TtgCCu1NgCBjK2bZRN+rKnGQJN6m4yTrViBXRiPA@mail.gmail.com> <CAD3i26BibcaMdbQv-j+Egz_1y0GuhzepBp5ATNpj=Qv8hi1TVA@mail.gmail.com> <CADZtCShAYpbN=4qNoX5c8yd1j08+mEZzG8gZwcHrj2suY0mb9w@mail.gmail.com> <CADZtCShYnM3A949H18V2+BArA-K9J+cDkd=rX8xRn0+0js5CwA@mail.gmail.com> <CAAS2fgTXS5Tains7dfe_Rc9JxR6M=NuFW9UtieRELm+6N2uNog@mail.gmail.com> <CAFfwr8F+ghYb2HYEgC7Lh7Z-ytNE7EABr6cxiVXYhWLk-TPO7A@mail.gmail.com> <CADZtCShDzPK_jqeOrK4XBoB2uriU9c9T8Dm7By-8ew3XOoAeQg@mail.gmail.com> <7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com> To: Jim Posen <jim.posen@gmail.com>, Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> X-Mailer: Apple Mail (2.3273) X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Mon, 28 May 2018 18:53:39 +0000 Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Mon, 28 May 2018 18:28:20 -0000 Forgot to mention: The link I sent is to a branch that is patched to = produce the filter stats.=20 This is the main project and the BIP158 implementation: = https://github.com/rust-bitcoin/rust-bitcoin-spv/blob/master/src/blockfilt= er.rs Tamas Blummer > On May 28, 2018, at 20:18, Tamas Blummer <tamas.blummer@gmail.com> = wrote: >=20 > Hi Jim, >=20 > A =E2=80=9Cbasic=E2=80=9D combined filter would mean up to 0.5 GB = filter data per month (with 100% segwith use). Considering that 1 GB is = the usual data quota for an entry level mobile phone contract, this = could be a too high barrier for adoption. >=20 > I repeated your calculations and produced a slightly different graph = that shows the fraction of cummulative filter size to cummulative = blockchain size. This is less noisy but otherwise confirms your = measurement. >=20 > I think that the data supports separation of filters as a combined = filter does not seem to come with significant savings. (basic size ~=3D = txid + input points + output scripts sizes) > =20 > My calculations are repeatable with: >=20 > = https://github.com/tamasblummer/rust-bitcoin-spv/blob/blockfilterstats/src= /bin/blockfilterstats.rs >=20 > that is using a Rust implementation of an SPV client built on top of = other libraries we work on in the rust-bitcoin GitHub community = (https://github.com/rust-bitcoin). Yes, this is a shameles plug for the = project hoping to attract more developer. >=20 > Tamas Blummer >=20 >=20 > <filters.png> >=20 >> On May 24, 2018, at 05:48, Jim Posen via bitcoin-dev = <bitcoin-dev@lists.linuxfoundation.org> wrote: >>=20 >> Greg, I've attached a graph including the input scripts. >>=20 >> In the top graph, we can see how the input script filter compares to = the input outpoint filter. It is definitely smaller as a result of = address reuse. The bottom graph shows the ratio over time of combining = the input prev script and output script filters vs keeping them = separate. In more recent blocks, it appears that there are decreasing = savings. >>=20 >=20