From: "Johan Torås Halseth" <johanth@gmail.com>
To: Olaoluwa Osuntokun <laolu32@gmail.com>
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size
Date: Mon, 21 May 2018 10:35:28 +0200 [thread overview]
Message-ID: <c23a5346-9f99-44f0-abbf-d7e7979bf1d8@gmail.com> (raw)
In-Reply-To: <CAO3Pvs9K3n=OzVQ06XGQvzNC+Aqp9S60kWM9VRPA8hWTJ3u9BQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3249 bytes --]
Hi all,
Most light wallets will want to download the minimum amount of data required to operate, which means they would ideally download the smallest possible filters containing the subset of elements they need.
What if instead of trying to decide up front which subset of elements will be most useful to include in the filters, and the size tradeoff, we let the full-node decide which subsets of elements it serves filters for?
For instance, a full node would advertise that it could serve filters for the subsets 110 (txid+script+outpoint), 100 (txid only), 011 ( script+outpoint) etc. A light client could then choose to download the minimal filter type covering its needs.
The obvious benefit of this would be minimal bandwidth usage for the light client, but there are also some less obvious ones. We wouldn’t have to decide up front what each filter type should contain, only the possible elements a filter can contain (more can be added later without breaking existing clients). This, I think, would let the most served filter types grow organically, with full-node implementations coming with sane defaults for served filter types (maybe even all possible types as long as the number of elements is small), letting their operator add/remove types at will.
The main disadvantage of this as I see it, is that there’s an exponential blowup in the number of possible filter types in the number of element types. However, this would let us start out small with only the elements we need, and in the worst case the node operators just choose to serve the subsets corresponding to what now is called “regular” + “extended” filters anyway, requiring no more resources.
This would also give us some data on what is the most widely used filter types, which could be useful in making the decision on what should be part of filters to eventually commit to in blocks.
- Johan On Sat, May 19, 2018 at 5:12, Olaoluwa Osuntokun via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
On Thu, May 17, 2018 at 2:44 PM Jim Posen via bitcoin-dev <bitcoin- Monitoring inputs by scriptPubkey vs input-txid also has a massive
advantage for parallel filtering: You can usually known your pubkeys
well in advance, but if you have to change what you're watching block
N+1 for based on the txids that paid you in N you can't filter them
in parallel.
Yes, I'll grant that this is a benefit of your suggestion.
Yeah parallel filtering would be pretty nice. We've implemented a serial filtering for btcwallet [1] for the use-case of rescanning after a seed phrase import. Parallel filtering would help here, but also we don't yet take advantage of batch querying for the filters themselves. This would speed up the scanning by quite a bit.
I really like the filtering model though, it really simplifies the code, and we can leverage identical logic for btcd (which has RPCs to fetch the filters) as well.
[1]: https://github.com/Roasbeef/btcwallet/blob/master/chain/neutrino.go#L180 [https://github.com/Roasbeef/btcwallet/blob/master/chain/neutrino.go#L180]
_______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[-- Attachment #2: Type: text/html, Size: 4734 bytes --]
next prev parent reply other threads:[~2018-05-21 8:35 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-17 15:25 [bitcoin-dev] BIP 158 Flexibility and Filter Size Matt Corallo
2018-05-17 15:43 ` Peter Todd
2018-05-17 15:46 ` Matt Corallo
2018-05-17 16:36 ` Gregory Maxwell
2018-05-17 16:59 ` Matt Corallo
2018-05-17 18:34 ` Gregory Maxwell
2018-05-17 20:19 ` Jim Posen
2018-05-17 20:45 ` Gregory Maxwell
2018-05-17 21:27 ` Jim Posen
2018-05-19 3:12 ` Olaoluwa Osuntokun
2018-05-21 8:35 ` Johan Torås Halseth [this message]
2018-05-22 1:16 ` Olaoluwa Osuntokun
2018-05-22 9:23 ` Johan Torås Halseth
2018-05-23 0:42 ` Jim Posen
2018-05-23 7:38 ` Jim Posen
2018-05-23 8:16 ` Johan Torås Halseth
2018-05-23 17:28 ` Gregory Maxwell
2018-05-24 1:04 ` Conner Fromknecht
2018-05-24 3:48 ` Jim Posen
2018-05-28 18:18 ` Tamas Blummer
2018-05-28 18:28 ` Tamas Blummer
2018-05-28 19:24 ` Gregory Maxwell
2018-05-29 2:42 ` Jim Posen
2018-05-29 3:24 ` Gregory Maxwell
2018-05-29 4:01 ` Olaoluwa Osuntokun
2018-05-31 14:27 ` Tamas Blummer
2018-06-01 2:52 ` Olaoluwa Osuntokun
2018-06-01 4:15 ` Gregory Maxwell
[not found] ` <CAAS2fgSyVi0d_ixp-auRPPzPfFeffN=hsWhWT5=EzDO3O+Ue1g@mail.gmail.com>
2018-06-02 0:01 ` Olaoluwa Osuntokun
2018-06-02 0:22 ` Gregory Maxwell
2018-06-02 2:02 ` Jim Posen
2018-06-02 12:41 ` David A. Harding
2018-06-02 22:02 ` Tamas Blummer
2018-06-03 0:28 ` Gregory Maxwell
2018-06-03 5:14 ` Tamas Blummer
2018-06-03 6:11 ` Pieter Wuille
2018-06-03 16:44 ` Tamas Blummer
2018-06-03 16:50 ` Tamas Blummer
2018-06-08 5:03 ` Olaoluwa Osuntokun
2018-06-08 16:14 ` Gregory Maxwell
2018-06-08 23:35 ` Olaoluwa Osuntokun
2018-06-09 10:34 ` David A. Harding
2018-06-12 23:51 ` Olaoluwa Osuntokun
2018-06-09 15:45 ` Gregory Maxwell
2018-06-12 23:58 ` Olaoluwa Osuntokun
2018-05-17 18:34 ` Gregory Maxwell
2018-05-18 8:46 ` Riccardo Casatta
2018-05-19 3:08 ` Olaoluwa Osuntokun
2018-05-19 2:57 ` Olaoluwa Osuntokun
2018-05-19 3:06 ` Pieter Wuille
2018-05-22 1:15 ` Olaoluwa Osuntokun
2018-05-18 6:28 ` Karl-Johan Alm
2018-06-04 8:42 ` Riccardo Casatta
2018-06-05 1:08 ` Jim Posen
2018-06-05 4:33 ` Karl-Johan Alm
2018-06-05 17:22 ` Jim Posen
2018-06-05 17:52 ` Gregory Maxwell
2018-06-06 1:12 ` Olaoluwa Osuntokun
2018-06-06 15:14 ` Riccardo Casatta
2018-05-19 2:51 ` Olaoluwa Osuntokun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c23a5346-9f99-44f0-abbf-d7e7979bf1d8@gmail.com \
--to=johanth@gmail.com \
--cc=bitcoin-dev@lists.linuxfoundation.org \
--cc=laolu32@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox