[bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Jonas Schnelli]

[-- Attachment #1.1: Type: text/plain, Size: 318 bytes --]

Hi

Currently, pruned peers have no way how to signal their (valuable) service.
A BIP proposal to improve this (draft):
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT <https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT>

Feedback is highly welcome.

</jonas>

[-- Attachment #1.2: Type: text/html, Size: 739 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Gregory Maxwell]

It probably should be stated in terms of what you're promising to do--
 288 and 1152 blocks, not what we hope it will accomplish. Then advise
clients to use peers with headroom because their estimates could be
wrong and reorgs.

Reorgs aren't the only concerns that drive larger numbers:  The peak
at syncing is at ~24 hours, but sometimes there are quite a few more
than 144 blocks in 24 hours. Also, new blocks show up in the chain:
you think you're 144 behind but by the time you connect you find
you're 146 behind from that peer's perspective.

I think it's a bit ambiguous what it's saying about the headers,
especially because it goes into detail about address relay. I believe
nodes with any of these settings should be willing to serve headers
for their entire best chain.  Perhaps you could say that this is
equivalent to NODE_NETWORK except that they aren't necessarily willing
to server historical blocks.

I'm unsure about the third depth level. Perhaps that should be left
undefined for sending right now and treated as least 1152 blocks by
receivers-- I don't have any reason to think 7056 is a particularly
useful choice, and we'll need another (longer) level for UTXO based
sync.   You could probably go further and say that nodes shouldn't
send it now, but if sent it means they intend to keep 2016*2 blocks.
(Not sending because the requirement for sending it may be that the
node is able to send you a UTXO data feed.)

> consider to switch a low percentage

That isn't grammatical, you want "switching".  But I think it would be
better to say that when a node believe it is in sync enough to use
NODE_NETWORK_LIMITED_X it should just treat them identically to
NODE_NETWORK in peer selection.   We don't really need any more
topology distortion than that.   In particular, I don't want to be in
a case where NODE_NETWORK peers suddenly find themselves far less well
connected.

In terms of making room, a node network peer could choose to
disconnect the least useful peers that aren't syncing from them to
make more room for ones that are.  This lets them decide what
connections they want, based on how full they are and what is useful
to them, rather than finding themselves all lonely because nodes
decided to avoid them to be "helpful", and we already use
disconnections to manage fullness.

On Thu, May 11, 2017 at 3:13 PM, Jonas Schnelli via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi
>
> Currently, pruned peers have no way how to signal their (valuable) service.
> A BIP proposal to improve this (draft):
> https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT
>
> Feedback is highly welcome.
>
> </jonas>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Luke Dashjr]

> A peer signaling NODE_NETWORK_LIMITED_LOW & NODE_NETWORK_LIMITED_HIGH MUST
> be capable of serving at least the last 7'056 blocks (~49 days)
> (NODE_NETWORK_LIMITED_HIGH's value ^2).

Is 49 days particularly useful? Would it be a problem to instead leave both-
bits undefined? I'm thinking this might be better as a way to indicate "7 
days, plus a deterministically chosen set of historical blocks"...

> Current Bitcoin-Core pruned full nodes guarantees a minimum of 288 blocks,
> thus allowing to signal NODE_NETWORK_LIMITED_LOW with the current minimum
> configuration.

This is technically true right now, but as soon as segwit activates, it will 
no longer be... Therefore, I suggest striking it from the BIP, expounding on 
it in greater detail, or making it true for the longer term.

> Peers following this BIP SHOULD connect a limited amount of their available
> outbound connections to peers signaling one or both of the
> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
> than the signaled number.

This isn't entirely clear whether it refers to peers downloading blocks, or 
peers serving them. (I assume the former, but it should be clarified.)

> Light clients (and such) who are not checking the nServiceFlags (service
> bits) from a relayed addr-message may unwillingly connect to a pruned peer
> and ask for (filtered) blocks at a depth below their pruned depth.

Wouldn't this already be a problem, without the BIP?

Luke

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Jonas Schnelli]

[-- Attachment #1: Type: text/plain, Size: 1955 bytes --]


> Is 49 days particularly useful? Would it be a problem to instead leave both-
> bits undefined? I'm thinking this might be better as a way to indicate "7
> days, plus a deterministically chosen set of historical blocks"…

I though two service bits allow three states and we should define all three combinations.
But I guess an adequate „definition“ would be to reserve it for future „definitions“.
Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.

49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.

> 
> This is technically true right now, but as soon as segwit activates, it will
> no longer be... Therefore, I suggest striking it from the BIP, expounding on
> it in greater detail, or making it true for the longer term.

AFAIK Core does also guaranteed the 288 blocks post segwit activation:
https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
But maybe I’m confused.

> 
>> Peers following this BIP SHOULD connect a limited amount of their available
>> outbound connections to peers signaling one or both of the
>> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
>> than the signaled number.
> 
> This isn't entirely clear whether it refers to peers downloading blocks, or
> peers serving them. (I assume the former, but it should be clarified.)

Indeed. I’ll try to make that more clear.

> 
>> Light clients (and such) who are not checking the nServiceFlags (service
>> bits) from a relayed addr-message may unwillingly connect to a pruned peer
>> and ask for (filtered) blocks at a depth below their pruned depth.
> 
> Wouldn't this already be a problem, without the BIP?

AFAIK, Core does currently only relay NODE_NETWORK addresses.
But yes, It may be a problem already.

</jonas>


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Aymeric Vitte]

[-- Attachment #1: Type: text/plain, Size: 3181 bytes --]

Sorry again, is this discussion really serious?

In this thread, previous ones or others, the level of approximation is
obvious, often shadowed by useless maths/papers and strange calculations
that are not helping, at the end nobody knows what would happen "if",
how far the chain can roll back, etc

Then don't make pruning the default if it's the current concern, pruning
is of no use

Again, the priority should be to scale the full nodes


Le 11/05/2017 à 22:10, Jonas Schnelli via bitcoin-dev a écrit :
>> Is 49 days particularly useful? Would it be a problem to instead leave both-
>> bits undefined? I'm thinking this might be better as a way to indicate "7
>> days, plus a deterministically chosen set of historical blocks"…
> I though two service bits allow three states and we should define all three combinations.
> But I guess an adequate „definition“ would be to reserve it for future „definitions“.
> Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.
>
> 49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.
>
>> This is technically true right now, but as soon as segwit activates, it will
>> no longer be... Therefore, I suggest striking it from the BIP, expounding on
>> it in greater detail, or making it true for the longer term.
> AFAIK Core does also guaranteed the 288 blocks post segwit activation:
> https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
> But maybe I’m confused.
>
>>> Peers following this BIP SHOULD connect a limited amount of their available
>>> outbound connections to peers signaling one or both of the
>>> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
>>> than the signaled number.
>> This isn't entirely clear whether it refers to peers downloading blocks, or
>> peers serving them. (I assume the former, but it should be clarified.)
> Indeed. I’ll try to make that more clear.
>
>>> Light clients (and such) who are not checking the nServiceFlags (service
>>> bits) from a relayed addr-message may unwillingly connect to a pruned peer
>>> and ask for (filtered) blocks at a depth below their pruned depth.
>> Wouldn't this already be a problem, without the BIP?
> AFAIK, Core does currently only relay NODE_NETWORK addresses.
> But yes, It may be a problem already.
>
> </jonas>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms


[-- Attachment #2: Type: text/html, Size: 5338 bytes --]

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Eric Voskuil]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 05/11/2017 01:36 PM, Aymeric Vitte via bitcoin-dev wrote:
> Sorry again, is this discussion really serious?
> 
> In this thread, previous ones or others, the level of approximation
> is obvious, often shadowed by useless maths/papers and strange
> calculations that are not helping, at the end nobody knows what
> would happen "if", how far the chain can roll back, etc
> 
> Then don't make pruning the default if it's the current concern,
> pruning is of no use
> 
> Again, the priority should be to scale the full nodes

I agree. Every full node operator should (and likely would at some
point) simply never connect to, or relay the address of, any "peer"
advertising itself as diminished. Why on earth would a full node
operator want to encourage shrinking support for bootstrapping and
deep reorgs, resulting in greater load for himself. That's a race to
the bottom.

We are literally talking about $7.50 for the *entire chain* with
breathing room. If someone wants to save a few dollars they are better
off attempting to hide their pruning.

e

> Le 11/05/2017 à 22:10, Jonas Schnelli via bitcoin-dev a écrit :
>>> Is 49 days particularly useful? Would it be a problem to
>>> instead leave both- bits undefined? I'm thinking this might be
>>> better as a way to indicate "7 days, plus a deterministically
>>> chosen set of historical blocks"…
>> I though two service bits allow three states and we should define
>> all three combinations. But I guess an adequate „definition“
>> would be to reserve it for future „definitions“. Or use Gregory's
>> proposal of min 2016*2 blocks & keep it „undefined“ for now.
>> 
>> 49 days was chosen to allow SPV peers to be „offline“ for a month
>> and still be capable to catch-up with a peer pruned to a datadir
>> of ~10GB.
>> 
>>> This is technically true right now, but as soon as segwit
>>> activates, it will no longer be... Therefore, I suggest
>>> striking it from the BIP, expounding on it in greater detail,
>>> or making it true for the longer term.
>> AFAIK Core does also guaranteed the 288 blocks post segwit
>> activation: 
>> https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa624008
93f4358c5ae/src/validation.h#L204
>>
>> 
But maybe I’m confused.
>> 
>>>> Peers following this BIP SHOULD connect a limited amount of
>>>> their available outbound connections to peers signaling one
>>>> or both of the NODE_NETWORK_LIMITED_* service bits if they
>>>> expect to request less blocks than the signaled number.
>>> This isn't entirely clear whether it refers to peers
>>> downloading blocks, or peers serving them. (I assume the
>>> former, but it should be clarified.)
>> Indeed. I’ll try to make that more clear.
>> 
>>>> Light clients (and such) who are not checking the
>>>> nServiceFlags (service bits) from a relayed addr-message may
>>>> unwillingly connect to a pruned peer and ask for (filtered)
>>>> blocks at a depth below their pruned depth.
>>> Wouldn't this already be a problem, without the BIP?
>> AFAIK, Core does currently only relay NODE_NETWORK addresses. But
>> yes, It may be a problem already.
>> 
>> </jonas>
>> 
>> 
>> 
>> _______________________________________________ bitcoin-dev
>> mailing list bitcoin-dev@lists.linuxfoundation.org 
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
> -- Zcash wallets made simple:
> https://github.com/Ayms/zcash-wallets Bitcoin wallets made simple:
> https://github.com/Ayms/bitcoin-wallets Get the torrent dynamic
> blocklist: http://peersm.com/getblocklist Check the 10 M passwords
> list: http://peersm.com/findmyass Anti-spies and private torrents,
> dynamic blocklist: http://torrent-live.org Peersm :
> http://www.peersm.com torrent-live:
> https://github.com/Ayms/torrent-live node-Tor :
> https://www.github.com/Ayms/node-Tor GitHub :
> https://www.github.com/Ayms
> 
> 
> 
> _______________________________________________ bitcoin-dev mailing
> list bitcoin-dev@lists.linuxfoundation.org 
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBCAAGBQJZFNHtAAoJEDzYwH8LXOFOYRwH/0By+TNSgnV6m4c7g1ZrjboG
8fZSeGaz7FXmAUZ69XMdQ1H+wlP0e4OAz9eRCcVqcn3K9OZJn++hbzI2K+zijyxZ
cpQjg/2dcTc4B0Z3PZdnuZx5mnHzavr/1vPlgOYla7AbYqcKB5dkq/HqgD6tFsJP
HXPClbEkVRF6UFP/7sDfzW8FMJycMSVcbEpuWAhcx2d+NusywWhbkuc5NiT9J1Ug
/3OFhHVJtd+rDl2B4iRHXHOhysUGffvmmLANZpPMcOgplM6Xwv7nMT34FV4HCdgs
Gyxc9GSFsD6xsOshBRPICtEZe+IDDb0cnOLjDdAnUnKeolUljFY52djSa300Fp0=
=REyc
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits

[Gregory Maxwell]

On Thu, May 11, 2017 at 3:13 PM, Jonas Schnelli via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi
>
> Currently, pruned peers have no way how to signal their (valuable) service.
> A BIP proposal to improve this (draft):
> https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT

The instructions for relay addresses should not instruct you to relay
these addresses but rather that you should relay addresses you would
connect to, under the generalized assumption that if it is useful to
you it will be useful to others.

This avoids instructing someone who might not consider
non-node-network peers useful from being directed by the BIP to relay
things that they don't find useful. (In particular, it means that the
obvious implementation of just throwing out the 'useless' information
works fine.)  I think would better reflect what people are likely to
actually do.