[bitcoin-dev] Bitcoin Node Speed Test

[bitcoin-dev] Bitcoin Node Speed Test

[slurms]

On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.


The testing methodology is as follows:

 * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.

 * A random selection of blocks was downloaded from each peer.

 * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.

 * The connecting node was in Amsterdam with a 1GB NIC. 

 
Results:

 * 37% of connected nodes failed to upload blocks faster than 1MB/s.

 * 16% of connected nodes uploaded blocks faster than 10MB/s.

 * Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ


This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 2306 bytes --]

Ahh, data... a breath of fresh air...

Can you re-analyze for 8MB blocks?  There is no current proposal for 20MB
blocks.

Also, most hashing power is now using Matt Corallo's fast block propagation
network; slow 'block' propagation to merchants/end-users doesn't really
matter (as long as it doesn't get anywhere near the 10-minute block time).

On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On this day, the Bitcoin network was crawled and reachable nodes surveyed
> to find their maximum throughput in order to determine if it can safely
> support a faster block rate. Specifically this is an attempt to prove or
> disprove the common statement that 1MB blocks were only suitable slower
> internet connections in 2009 when Bitcoin launched, and that connection
> speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
>  * Nodes were randomly selected from a peers.dat, 5% of the reachable
> nodes in the network were contacted.
>
>  * A random selection of blocks was downloaded from each peer.
>
>  * There is some bias towards higher connection speeds, very slow
> connections (<30KB/s) timed out in order to run the test at a reasonable
> rate.
>
>  * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
>  * Raw data, one line per connected node, kilobytes per second
> http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available
> bandwidth for increased block sizes, as in its current state 37% of nodes
> would fail to upload a 20MB block to a single peer in under 20 seconds
> (referencing a number quoted by Gavin). If the bar for suitability is
> placed at taking only 1% of the block time (6 seconds) to upload one block
> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
> only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>



-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 3092 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Jameson Lopp]

[-- Attachment #1: Type: text/plain, Size: 2012 bytes --]

Are you willing to share the code that you used to run the test?

- Jameson

On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On this day, the Bitcoin network was crawled and reachable nodes surveyed
> to find their maximum throughput in order to determine if it can safely
> support a faster block rate. Specifically this is an attempt to prove or
> disprove the common statement that 1MB blocks were only suitable slower
> internet connections in 2009 when Bitcoin launched, and that connection
> speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
>  * Nodes were randomly selected from a peers.dat, 5% of the reachable
> nodes in the network were contacted.
>
>  * A random selection of blocks was downloaded from each peer.
>
>  * There is some bias towards higher connection speeds, very slow
> connections (<30KB/s) timed out in order to run the test at a reasonable
> rate.
>
>  * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
>  * Raw data, one line per connected node, kilobytes per second
> http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available
> bandwidth for increased block sizes, as in its current state 37% of nodes
> would fail to upload a 20MB block to a single peer in under 20 seconds
> (referencing a number quoted by Gavin). If the bar for suitability is
> placed at taking only 1% of the block time (6 seconds) to upload one block
> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
> only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2679 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Peter Todd]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256



On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>This does not support the theory that the network has the available
>bandwidth for increased block sizes, as in its current state 37% of
>nodes would fail to upload a 20MB block to a single peer in under 20
>seconds (referencing a number quoted by Gavin). If the bar for
>suitability is placed at taking only 1% of the block time (6 seconds)
>to upload one block to one peer, then 69% of the network fails for 20MB
>blocks. For comparison, only 10% fail this metric for 1MB blocks.

Note how due to bandwidth being generally asymetric your findings are probably optimistic - you've measured download capacity. On top of that upload is further reduced by the fact that multiple peers at once need to be sent blocks for reliability.

Secondly you're measuring a network that isn't under attack - we need significant additional margin to resist attack as performance is consensus-critical.

-----BEGIN PGP SIGNATURE-----

iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVsRCj
AAoJEMCF8hzn9Lnc47AIAIQbznavjd2Rbqxeq5a3GLqeYoI4BZIQYqfWky+6OQtq
yGRKaqPtGuES5y9L0k7efivT385mOl87PWnWMy61xxZ9FJgoS+YHkEx8K4tfgfA2
yLOKzeFSar2ROCcjHYyPWa2XXjRbNmiLzfNuQyIBArg/Ch9//iXUUM+GG0mChF5k
nUxLstXgXDNh5H8xkHeLi4lEbt9HFiwcZnT1Tzeo2dvVTujrtyNb/zEhNZScMXDc
UOlT8rBLxzHlytKdXt1GNKIq0feTRJNbreBh7/EB4nYTT54CItaaVXul0LdHd5/2
kgKtdbUdeyaRUKrKcvxiuIwclyoOuRQp0DZThsB262o=
=tBUM
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Leo Wandersleb]

[-- Attachment #1: Type: text/plain, Size: 2314 bytes --]

Thank you a lot for doing this test!

Two questions:

1) A node is typically connected to many nodes that would all in parallel
download said block. In your test you measured how fast new blocks that
presumably are being uploaded in parallel to all those other nodes are being
uploaded? Or did you download blocks while those nodes were basically idle?

2) What is your percentage of the very slow connections?

On 07/23/2015 11:19 AM, slurms--- via bitcoin-dev wrote:
> On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
>  * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.
>
>  * A random selection of blocks was downloaded from each peer.
>
>  * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.
>
>  * The connecting node was in Amsterdam with a 1GB NIC. 
>
>  
> Results:
>
>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
>  * Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev





[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Slurms MacKenzie]

[-- Attachment #1: Type: text/html, Size: 4926 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Slurms MacKenzie]

[-- Attachment #1: Type: text/html, Size: 3544 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Pindar Wong]

[-- Attachment #1: Type: text/plain, Size: 3097 bytes --]

This looks like the beginnings of some great analysis.

Per Peter's remarks, I think it would be productive to run the test(s) on a
simulated network with worst case network failure(s) so that we can
determine the safety margin needed.

I have potential access to h/w resources that would be available for
running such tests at the necessary scales.

Cheers,

p.


On Fri, Jul 24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> The library used isn't open source, so unfortunately not. It shouldn't be
> too hard to replicate in python-bitcoinlib or bitcoinj though.
>
> *Sent:* Thursday, July 23, 2015 at 6:55 PM
> *From:* "Jameson Lopp" <jameson.lopp@gmail.com>
> *To:* slurms@gmx.us
> *Cc:* bitcoin-dev@lists.linuxfoundation.org
> *Subject:* Re: [bitcoin-dev] Bitcoin Node Speed Test
>  Are you willing to share the code that you used to run the test?
>
> - Jameson
>
> On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> On this day, the Bitcoin network was crawled and reachable nodes surveyed
>> to find their maximum throughput in order to determine if it can safely
>> support a faster block rate. Specifically this is an attempt to prove or
>> disprove the common statement that 1MB blocks were only suitable slower
>> internet connections in 2009 when Bitcoin launched, and that connection
>> speeds have improved to the point of obviously supporting larger blocks.
>>
>>
>> The testing methodology is as follows:
>>
>>  * Nodes were randomly selected from a peers.dat, 5% of the reachable
>> nodes in the network were contacted.
>>
>>  * A random selection of blocks was downloaded from each peer.
>>
>>  * There is some bias towards higher connection speeds, very slow
>> connections (<30KB/s) timed out in order to run the test at a reasonable
>> rate.
>>
>>  * The connecting node was in Amsterdam with a 1GB NIC.
>>
>>
>> Results:
>>
>>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>>
>>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
>>
>>  * Raw data, one line per connected node, kilobytes per second
>> http://pastebin.com/raw.php?i=6b4NuiVQ
>>
>>
>> This does not support the theory that the network has the available
>> bandwidth for increased block sizes, as in its current state 37% of nodes
>> would fail to upload a 20MB block to a single peer in under 20 seconds
>> (referencing a number quoted by Gavin). If the bar for suitability is
>> placed at taking only 1% of the block time (6 seconds) to upload one block
>> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
>> only 10% fail this metric for 1MB blocks.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 4913 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Marcel Jamin]

[-- Attachment #1: Type: text/plain, Size: 1992 bytes --]

He measured the upload capacity of the peers by downloading from them, or
am I being dumb? :)


2015-07-23 18:05 GMT+02:00 Peter Todd via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
>
> On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
> >This does not support the theory that the network has the available
> >bandwidth for increased block sizes, as in its current state 37% of
> >nodes would fail to upload a 20MB block to a single peer in under 20
> >seconds (referencing a number quoted by Gavin). If the bar for
> >suitability is placed at taking only 1% of the block time (6 seconds)
> >to upload one block to one peer, then 69% of the network fails for 20MB
> >blocks. For comparison, only 10% fail this metric for 1MB blocks.
>
> Note how due to bandwidth being generally asymetric your findings are
> probably optimistic - you've measured download capacity. On top of that
> upload is further reduced by the fact that multiple peers at once need to
> be sent blocks for reliability.
>
> Secondly you're measuring a network that isn't under attack - we need
> significant additional margin to resist attack as performance is
> consensus-critical.
>
> -----BEGIN PGP SIGNATURE-----
>
> iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVsRCj
> AAoJEMCF8hzn9Lnc47AIAIQbznavjd2Rbqxeq5a3GLqeYoI4BZIQYqfWky+6OQtq
> yGRKaqPtGuES5y9L0k7efivT385mOl87PWnWMy61xxZ9FJgoS+YHkEx8K4tfgfA2
> yLOKzeFSar2ROCcjHYyPWa2XXjRbNmiLzfNuQyIBArg/Ch9//iXUUM+GG0mChF5k
> nUxLstXgXDNh5H8xkHeLi4lEbt9HFiwcZnT1Tzeo2dvVTujrtyNb/zEhNZScMXDc
> UOlT8rBLxzHlytKdXt1GNKIq0feTRJNbreBh7/EB4nYTT54CItaaVXul0LdHd5/2
> kgKtdbUdeyaRUKrKcvxiuIwclyoOuRQp0DZThsB262o=
> =tBUM
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2749 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Joseph Gleason ⑈]

[-- Attachment #1: Type: text/plain, Size: 2374 bytes --]

That is how I read it as well.


On Thu, Jul 23, 2015 at 12:56 PM Marcel Jamin via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> He measured the upload capacity of the peers by downloading from them, or
> am I being dumb? :)
>
>
> 2015-07-23 18:05 GMT+02:00 Peter Todd via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>>
>>
>> On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>> >This does not support the theory that the network has the available
>> >bandwidth for increased block sizes, as in its current state 37% of
>> >nodes would fail to upload a 20MB block to a single peer in under 20
>> >seconds (referencing a number quoted by Gavin). If the bar for
>> >suitability is placed at taking only 1% of the block time (6 seconds)
>> >to upload one block to one peer, then 69% of the network fails for 20MB
>> >blocks. For comparison, only 10% fail this metric for 1MB blocks.
>>
>> Note how due to bandwidth being generally asymetric your findings are
>> probably optimistic - you've measured download capacity. On top of that
>> upload is further reduced by the fact that multiple peers at once need to
>> be sent blocks for reliability.
>>
>> Secondly you're measuring a network that isn't under attack - we need
>> significant additional margin to resist attack as performance is
>> consensus-critical.
>>
>> -----BEGIN PGP SIGNATURE-----
>>
>> iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVsRCj
>> AAoJEMCF8hzn9Lnc47AIAIQbznavjd2Rbqxeq5a3GLqeYoI4BZIQYqfWky+6OQtq
>> yGRKaqPtGuES5y9L0k7efivT385mOl87PWnWMy61xxZ9FJgoS+YHkEx8K4tfgfA2
>> yLOKzeFSar2ROCcjHYyPWa2XXjRbNmiLzfNuQyIBArg/Ch9//iXUUM+GG0mChF5k
>> nUxLstXgXDNh5H8xkHeLi4lEbt9HFiwcZnT1Tzeo2dvVTujrtyNb/zEhNZScMXDc
>> UOlT8rBLxzHlytKdXt1GNKIq0feTRJNbreBh7/EB4nYTT54CItaaVXul0LdHd5/2
>> kgKtdbUdeyaRUKrKcvxiuIwclyoOuRQp0DZThsB262o=
>> =tBUM
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 3562 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Matt Corallo]

You may see much better throughput if you run a few servers around the
globe and test based on closest-by-geoip. TCP throughput is rather
significantly effected by latency, though I'm not really sure what you
should be testing here, ideally.

On 07/23/15 14:19, slurms--- via bitcoin-dev wrote:
> On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.
> 
> 
> The testing methodology is as follows:
> 
>  * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.
> 
>  * A random selection of blocks was downloaded from each peer.
> 
>  * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.
> 
>  * The connecting node was in Amsterdam with a 1GB NIC. 
> 
>  
> Results:
> 
>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
> 
>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
> 
>  * Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ
> 
> 
> This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 295 bytes --]

On Thu, Jul 23, 2015 at 09:56:36PM +0200, Marcel Jamin wrote:
> He measured the upload capacity of the peers by downloading from them, or
> am I being dumb? :)

Lol, no, I'm being dumb. :)

-- 
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Bitcoin Node Speed Test

[Slurms MacKenzie]

Yes that is completely doable for the next crawl, however I am not sure how much that reflects the behavior bitcoind would see when making connections. Nodes do not make any attempt to sync with close peers, which is an undesirable property if you are attempting to be sybil resistant. With some quick playing around it seems that you do get the expected speedup with close proximity, but it's not a particularly huge difference at present. I'll keep working on it and see where I get. 


> Sent: Friday, July 24, 2015 at 4:48 AM
> From: "Matt Corallo via bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org>
> To: bitcoin-dev@lists.linuxfoundation.org
> Subject: Re: [bitcoin-dev] Bitcoin Node Speed Test
>
> You may see much better throughput if you run a few servers around the
> globe and test based on closest-by-geoip. TCP throughput is rather
> significantly effected by latency, though I'm not really sure what you
> should be testing here, ideally.

Re: [bitcoin-dev] Bitcoin Node Speed Test

[grarpamp]

On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>  * A random selection of blocks was downloaded from each peer.

If the selection is different for each peer the results will be broken.