[bitcoin-dev] Moving away from BIP37, unsetting NODE_BLOOM

[bitcoin-dev] Moving away from BIP37, unsetting NODE_BLOOM

[Caius Cosades]

As previously discussed[0][1][2] on the mailing list, github issue commentary, and IRC channels, there's substantial reason to disable BIP37 in network nodes which are getting stronger as the size of the chain increases. BIP37 has significant denial of service issues which are unsolvable in the design, it introduces undue load on the bitcoin network  by default, and doesn't provide an acceptable amount of security and reliability to "lightweight wallets" as originally intended. 

BIP37 allows "lightweight wallets" to connect to nodes in the network, and request that they load, deseralize, and expensively apply an arbitrary bloom filter to their block files and mempool. This should never have been the role of nodes in the network, rather it should have been opt-in, or performed by a different piece of software entirely. The inability of the nodes to cache the responses or meaningfully rate limit them makes it detrimental to serve these requests. 

BIP37 was intended to have stronger privacy than it does in reality[3][4], where effectively any node that can capture `filterload` and `filteradd` responses can trivially de-anonymize an entire wallet that has connected irrespective of the amount of noise they add to their filters. The connected node lying by omission is undetectable by any wallet software, where they will be lead to believe that there are no matching responses; this is counter-able by further destroying privacy and loading down the network by having multiple peers simultaneously return filter results and hoping that at least one isn't lying. 

NODE_BLOOM has been implemented already which allows nodes to signal in their service message that they do, or do not support filtering. I suggest that in the next major release this is defaulted to 0, and any software relying on BIP37 move to using other filtering options, or another piece of dedicated software to serve the requests. Future releases of the reference software should remove BIP37 commands entirely. 


[0]: https://www.reddit.com/r/Bitcoin/comments/3hjak7/the_hard_work_of_core_devs_not_xt_makes_bitcoin/cu9xntf/?context=3
[1]: https://github.com/bitcoin/bitcoin/issues/6578
[2]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010535.html
[3]: https://jonasnick.github.io/slides/2016-zurich-meetup.pdf
[4]: https://eprint.iacr.org/2014/763.pdf

Re: [bitcoin-dev] Moving away from BIP37, unsetting NODE_BLOOM

[Jonas Schnelli]

[-- Attachment #1: Type: text/plain, Size: 4332 bytes --]

Hi Caius

Thanks for brining this up.
As far as it looks, one of the major SPV library does not yet respect the NODE_BLOOM service flag [1]. Unsure sure about others.

It think disabling NODE_BLOOM services by default in full node implementations makes sense as soon as BIP158 [2] (compact block filters) has been implemented and therefore NODE_COMPACT_FILTERS is signalled broadly. Unsure if it would make sense to even wait for block filter commitment softfork (probably no).

Then, the question is, if there are alternatives for mempool filtering (display unconfirmed transactions) or if the protocol recommendation are to disable that by default or recommend to use centralised filtering technique via wallet provider infrastructure (privacy?!).

I guess everyone here agrees that there are major privacy and load-distribution issues with BIP37, and, that it should be disabled in the long run.
But, due to the lack of alternatives, there is the risk of breaking existing SPV client models and thus pushing users to complete centralised validation-solutions (and even towards centralised key-storage), which, may result in making privacy and security even more worse.

I personally miss a long term concept how to keep non expert users (or say light clients) close to the p2p protocol in a decentralized fashion. Unclear how decentralized fee estimations and „incoming transactions“ (which is something users want even if it's a broken concept) are handled in the future.

—
/jonas

[1] https://github.com/bitcoinj/bitcoinj/pull/1212
[2] https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki

> Am 16.05.2018 um 23:22 schrieb Caius Cosades via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>:
> 
> As previously discussed[0][1][2] on the mailing list, github issue commentary, and IRC channels, there's substantial reason to disable BIP37 in network nodes which are getting stronger as the size of the chain increases. BIP37 has significant denial of service issues which are unsolvable in the design, it introduces undue load on the bitcoin network  by default, and doesn't provide an acceptable amount of security and reliability to "lightweight wallets" as originally intended.
> 
> BIP37 allows "lightweight wallets" to connect to nodes in the network, and request that they load, deseralize, and expensively apply an arbitrary bloom filter to their block files and mempool. This should never have been the role of nodes in the network, rather it should have been opt-in, or performed by a different piece of software entirely. The inability of the nodes to cache the responses or meaningfully rate limit them makes it detrimental to serve these requests.
> 
> BIP37 was intended to have stronger privacy than it does in reality[3][4], where effectively any node that can capture `filterload` and `filteradd` responses can trivially de-anonymize an entire wallet that has connected irrespective of the amount of noise they add to their filters. The connected node lying by omission is undetectable by any wallet software, where they will be lead to believe that there are no matching responses; this is counter-able by further destroying privacy and loading down the network by having multiple peers simultaneously return filter results and hoping that at least one isn't lying.
> 
> NODE_BLOOM has been implemented already which allows nodes to signal in their service message that they do, or do not support filtering. I suggest that in the next major release this is defaulted to 0, and any software relying on BIP37 move to using other filtering options, or another piece of dedicated software to serve the requests. Future releases of the reference software should remove BIP37 commands entirely.
> 
> 
> [0]: https://www.reddit.com/r/Bitcoin/comments/3hjak7/the_hard_work_of_core_devs_not_xt_makes_bitcoin/cu9xntf/?context=3
> [1]: https://github.com/bitcoin/bitcoin/issues/6578
> [2]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010535.html
> [3]: https://jonasnick.github.io/slides/2016-zurich-meetup.pdf
> [4]: https://eprint.iacr.org/2014/763.pdf
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]