From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pindar.wong@gmail.com> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id C39C65B1 for <bitcoin-dev@lists.linuxfoundation.org>; Thu, 23 Jul 2015 17:41:36 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-la0-f45.google.com (mail-la0-f45.google.com [209.85.215.45]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B29BA112 for <bitcoin-dev@lists.linuxfoundation.org>; Thu, 23 Jul 2015 17:41:35 +0000 (UTC) Received: by lagw2 with SMTP id w2so162128966lag.3 for <bitcoin-dev@lists.linuxfoundation.org>; Thu, 23 Jul 2015 10:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ki4MbdR+P2LW2x92Sj8perItZuuLGAbUe5mGIMzOmME=; b=i8A1DMT0eRFtpQpVj8cQHm5BVJY1wM5LGfuWA1+WdkU+MnHjPAjNvePIF0xKruKMyn ElzVFQ0CSdPVSCzyYSY4rdY4zfdF2ZKEeZduHRognqqNyfoKn0qEE2MYseYCeQeAO8LD o2+4SHDEK7uOK1Plv+LMH1OnuPgYpIOUFId+7G44of+VuD0hlsIi9B6Nm3Vty5YU6u60 r6zb0YV/f3eCYCW3PdOAToQVgweI1mC4OTHXwJtny1RQpWaq/5212WIlOVQSBxc6zZB1 2G34bwCQnVCY/JonVQsJIfCGIkp6+GLYJIEipDFlBysbOu6KI8ROoRcd7kkhc+SzSYSh Wuog== MIME-Version: 1.0 X-Received: by 10.112.168.102 with SMTP id zv6mr9333296lbb.45.1437673294253; Thu, 23 Jul 2015 10:41:34 -0700 (PDT) Received: by 10.152.133.84 with HTTP; Thu, 23 Jul 2015 10:41:33 -0700 (PDT) In-Reply-To: <trinity-650f6539-8135-4f95-a54f-9dd0744df911-1437671671241@3capp-mailcom-bs04> References: <trinity-c97bc41b-a953-4580-b2d2-ebdda9eb96b2-1437661199263@3capp-mailcom-bs02> <CADL_X_dmeyjR2PJN8oLn8EutVCu8Pn_qsP9ATRCYadx3dh4Erg@mail.gmail.com> <trinity-650f6539-8135-4f95-a54f-9dd0744df911-1437671671241@3capp-mailcom-bs04> Date: Fri, 24 Jul 2015 01:41:33 +0800 Message-ID: <CAM7BtUrtfCM+P6DTGdgTTSpfq-Ot=YdTzn40dbwa_KkyoN7F7w@mail.gmail.com> From: Pindar Wong <pindar.wong@gmail.com> To: Slurms MacKenzie <slurms@gmx.us> Content-Type: multipart/alternative; boundary=001a11c33d3201d4a8051b8e66eb X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE, RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] Bitcoin Node Speed Test X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Thu, 23 Jul 2015 17:41:36 -0000 --001a11c33d3201d4a8051b8e66eb Content-Type: text/plain; charset=UTF-8 This looks like the beginnings of some great analysis. Per Peter's remarks, I think it would be productive to run the test(s) on a simulated network with worst case network failure(s) so that we can determine the safety margin needed. I have potential access to h/w resources that would be available for running such tests at the necessary scales. Cheers, p. On Fri, Jul 24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > The library used isn't open source, so unfortunately not. It shouldn't be > too hard to replicate in python-bitcoinlib or bitcoinj though. > > *Sent:* Thursday, July 23, 2015 at 6:55 PM > *From:* "Jameson Lopp" <jameson.lopp@gmail.com> > *To:* slurms@gmx.us > *Cc:* bitcoin-dev@lists.linuxfoundation.org > *Subject:* Re: [bitcoin-dev] Bitcoin Node Speed Test > Are you willing to share the code that you used to run the test? > > - Jameson > > On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: >> >> On this day, the Bitcoin network was crawled and reachable nodes surveyed >> to find their maximum throughput in order to determine if it can safely >> support a faster block rate. Specifically this is an attempt to prove or >> disprove the common statement that 1MB blocks were only suitable slower >> internet connections in 2009 when Bitcoin launched, and that connection >> speeds have improved to the point of obviously supporting larger blocks. >> >> >> The testing methodology is as follows: >> >> * Nodes were randomly selected from a peers.dat, 5% of the reachable >> nodes in the network were contacted. >> >> * A random selection of blocks was downloaded from each peer. >> >> * There is some bias towards higher connection speeds, very slow >> connections (<30KB/s) timed out in order to run the test at a reasonable >> rate. >> >> * The connecting node was in Amsterdam with a 1GB NIC. >> >> >> Results: >> >> * 37% of connected nodes failed to upload blocks faster than 1MB/s. >> >> * 16% of connected nodes uploaded blocks faster than 10MB/s. >> >> * Raw data, one line per connected node, kilobytes per second >> http://pastebin.com/raw.php?i=6b4NuiVQ >> >> >> This does not support the theory that the network has the available >> bandwidth for increased block sizes, as in its current state 37% of nodes >> would fail to upload a 20MB block to a single peer in under 20 seconds >> (referencing a number quoted by Gavin). If the bar for suitability is >> placed at taking only 1% of the block time (6 seconds) to upload one block >> to one peer, then 69% of the network fails for 20MB blocks. For comparison, >> only 10% fail this metric for 1MB blocks. >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > --001a11c33d3201d4a8051b8e66eb Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div><div>This looks like the beginnings of some grea= t analysis.<br><br>Per Peter's remarks, I think it would be productive = to run the test(s) on a simulated network with worst case network failure(s= ) so that we can determine the safety margin needed. <br><br></div>I have p= otential access to h/w resources that would be available for running such t= ests at the necessary scales.<br><br></div>Cheers,<br><br></div>p.<br><br><= /div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Fri, Jul = 24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev <span dir=3D"ltr"><= ;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank"= >bitcoin-dev@lists.linuxfoundation.org</a>></span> wrote:<br><blockquote= class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli= d;padding-left:1ex"><div><div style=3D"font-family:Verdana;font-size:12.0px= "><div> <div>The library used isn't open source, so unfortunately not. It shoul= dn't be too hard to replicate in python-bitcoinlib or bitcoinj though.<= /div> <div>=C2=A0 <div name=3D"quote" style=3D"margin:10px 5px 5px 10px;padding:10px 0 10px 1= 0px;border-left:2px solid #c3d9e5;word-wrap:break-word"> <div style=3D"margin:0 0 10px 0"><b>Sent:</b>=C2=A0Thursday, July 23, 2015 = at 6:55 PM<br> <b>From:</b>=C2=A0"Jameson Lopp" <<a href=3D"mailto:jameson.lo= pp@gmail.com" target=3D"_blank">jameson.lopp@gmail.com</a>><br> <b>To:</b>=C2=A0<a href=3D"mailto:slurms@gmx.us" target=3D"_blank">slurms@g= mx.us</a><br> <b>Cc:</b>=C2=A0<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" ta= rget=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><span class=3D""><= br> <b>Subject:</b>=C2=A0Re: [bitcoin-dev] Bitcoin Node Speed Test</span></div> <div name=3D"quoted-content"><span class=3D""> <div>Are you willing to share the code that you used to run the test? <div>=C2=A0</div> <div>- Jameson</div> </div> </span><div class=3D"gmail_extra">=C2=A0 <div class=3D"gmail_quote"><span class=3D"">On Thu, Jul 23, 2015 at 10:19 A= M, slurms--- via bitcoin-dev <span><<a href=3D"http://bitcoin-dev@lists.= linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.or= g</a>></span> wrote: </span><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 0.8ex;border= -left:1.0px rgb(204,204,204) solid;padding-left:1.0ex"><span class=3D"">On = this day, the Bitcoin network was crawled and reachable nodes surveyed to f= ind their maximum throughput in order to determine if it can safely support= a faster block rate. Specifically this is an attempt to prove or disprove = the common statement that 1MB blocks were only suitable slower internet con= nections in 2009 when Bitcoin launched, and that connection speeds have imp= roved to the point of obviously supporting larger blocks.<br> <br> <br> The testing methodology is as follows:<br> <br> =C2=A0* Nodes were randomly selected from a peers.dat, 5% of the reachable = nodes in the network were contacted.<br> <br> =C2=A0* A random selection of blocks was downloaded from each peer.<br> <br> =C2=A0* There is some bias towards higher connection speeds, very slow conn= ections (<30KB/s) timed out in order to run the test at a reasonable rat= e.<br> <br> =C2=A0* The connecting node was in Amsterdam with a 1GB NIC.<br> <br> =C2=A0<br> Results:<br> <br> =C2=A0* 37% of connected nodes failed to upload blocks faster than 1MB/s.<b= r> <br> =C2=A0* 16% of connected nodes uploaded blocks faster than 10MB/s.<br> <br> =C2=A0* Raw data, one line per connected node, kilobytes per second <a href= =3D"http://pastebin.com/raw.php?i=3D6b4NuiVQ" target=3D"_blank">http://past= ebin.com/raw.php?i=3D6b4NuiVQ</a><br> <br> <br> This does not support the theory that the network has the available bandwid= th for increased block sizes, as in its current state 37% of nodes would fa= il to upload a 20MB block to a single peer in under 20 seconds (referencing= a number quoted by Gavin). If the bar for suitability is placed at taking = only 1% of the block time (6 seconds) to upload one block to one peer, then= 69% of the network fails for 20MB blocks. For comparison, only 10% fail th= is metric for 1MB blocks.<br> _______________________________________________<br> bitcoin-dev mailing list<br> </span><a href=3D"http://bitcoin-dev@lists.linuxfoundation.org" target=3D"_= blank">bitcoin-dev@lists.linuxfoundation.org</a><br> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = target=3D"_blank">https://lists.linuxfoundation.org/mailman/listinfo/bitcoi= n-dev</a></blockquote> </div> </div> </div> </div> </div> </div></div></div> <br>_______________________________________________<br> bitcoin-dev mailing list<br> <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.= linuxfoundation.org</a><br> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev</a><br> <br></blockquote></div><br></div> --001a11c33d3201d4a8051b8e66eb--