From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B421183D for ; Thu, 2 Jul 2015 08:32:45 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from st11p02im-asmtp001.me.com (st11p02im-asmtp001.me.com [17.172.220.113]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id EB18F154 for ; Thu, 2 Jul 2015 08:32:44 +0000 (UTC) Received: from [10.0.1.7] (216-19-182-8.dyn.novuscom.net [216.19.182.8]) by st11p02im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Mar 31 2015)) with ESMTPSA id <0NQU00EI5QEHC230@st11p02im-asmtp001.me.com> for bitcoin-dev@lists.linuxfoundation.org; Thu, 02 Jul 2015 08:32:43 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-07-02_07:2015-06-30, 2015-07-02, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1507020146 MIME-version: 1.0 (Mac OS X Mail 8.2 \(2098\)) Content-type: multipart/signed; boundary="Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5 From: Jean-Paul Kogelman In-reply-to: <20150702071814.GA1904@banane.informatik.uni-ulm.de> Date: Thu, 02 Jul 2015 01:33:08 -0700 Message-id: <7A4D6052-CAA8-4E22-92E4-83828E589570@me.com> References: <20150702071814.GA1904@banane.informatik.uni-ulm.de> To: Henning Kopp X-Mailer: Apple Mail (2.2098) X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] Defining a min spec X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 08:32:45 -0000 --Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 In the case of Bitcoin Core, for a starting point, you basically have to = work backwards from what we have right now. We know some of the bounds = already. Block size already tells you a lot about your bandwidth = requirements, and Pieter=E2=80=99s simulations gives you even more = information when you take orphan rates into account. There=E2=80=99s = also a hard cap on the number of SigOps if I recall correctly, so = that=E2=80=99s probably a good starting point for a MIPS metric, etc. Memory is probably the hardest to pin down since some memory structures = like (from what I understand, correct me if I=E2=80=99m wrong) the UTXO = database live fully in memory and are basically unbounded. Perhaps this = can somehow be capped at a certain size and move all the really old = UTXO=E2=80=99s that are unlikely to move to disk and just take the CPU / = disk hit in the unlikely event that they are referenced by a new block. = Has the address database been capped yet? Mempool? I realize that it=E2=80= =99s probably debatable whether or not this behaviour should be = independent of available memory since any bugs here could affect = consensus (especially the UTXO db). Ultimately, what comes out of it is a list of numbers. A Mbit network = I/O, B MIPS, C MB memory, D Disk space, etc. At that point you can = debate whether or not such a machine can be considered an entrypoint = bitcoin full node. You round up the numbers that are not really = available anymore in off the shelf hardware (like disk space) and you = round down the numbers that seem too high. For all we know we=E2=80=99re = already over budget on some of the metrics that we decide to track as = min spec. Network I/O for example. At that point you can start focussed = research into bringing Bitcoin Core back into budget on those metrics. = Then the discussion moves from =E2=80=9Cit=E2=80=99s probably too = high=E2=80=9D to =E2=80=9Cwe=E2=80=99re X% over budget=E2=80=9D. The most valuable thing that could come out of this is to get some kind = of formulation how all the different levers in Bitcoin Core affect the = min spec and ideally have a benchmark tool. For example, we could settle = on a min spec that would exclude the Raspberry Pi 1 on MIPS, but when = secp256k1 is enabled for validation, the MIPS requirement could drop = significantly, allowing us to adjust the min spec downward to include = the Raspberry Pi 1 again (again, just an example). Ideally some people would have the actual min spec machine built and = running. The cost of that shouldn=E2=80=99t be too high (it=E2=80=99s = the min spec after all) and I=E2=80=99m sure people would be happy to = chip in a couple bits for this. Remember, the min spec should be able to handle Bitcoin Core running = under full load; that=E2=80=99s maxed out blocks with maxed out SigOps, = etc. jp > On Jul 2, 2015, at 12:18 AM, Henning Kopp = wrote: >=20 > Hi Jean-Paul, >=20 > that's a very interesting point of view and I have never thought about > it this way, since I have a totally different background. >=20 > How would you go on about defining a min spec? Is this done by testing > the software on different hardware configurations or are you looking > at the requirements a priori? >=20 > Best regards > Henning >=20 >=20 > On Wed, Jul 01, 2015 at 09:04:19PM -0700, Jean-Paul Kogelman wrote: >> Hi folks, >>=20 >> I=E2=80=99m a game developer. I write time critical code for a living = and have to deal with memory, CPU, GPU and I/O budgets on a daily basis. = These budgets are based on what we call a minimum specification (of = hardware); min spec for short. In most cases the min spec is based on = entry model machines that are available during launch, and will give the = user an enjoyable experience when playing our games. Obviously, we can = turn on a number of bells and whistles for people with faster machines, = but that=E2=80=99s not the point of this mail. >>=20 >> The point is, can we define a min spec for Bitcoin Core? The number = one reason for this is: if you know how your changes affect your = available budgets, then the risk of breaking something due to capacity = problems is reduced to practically zero. >>=20 >> One way of doing so is to work backwards from what we have right now: = Block size (network / disk I/O), SigOps/block (CPU), UTXO size (memory), = etc. Then there=E2=80=99s Pieter=E2=80=99s analysis of network = bottlenecks and how it affects orphan rates that could be used to set = some form of cap on what transfer time + verification time should be to = keep the orphan rate at an acceptable level. >>=20 >> So taking all of the above (and more) into account, what = configuration would be the bare minimum to comfortably run Bitcoin Core = at maximum load and can it be reasonably expected to still be out there = in the field running Bitcoin Core? Also, can the parameters that were = used to determine this min spec be codified in some way so that they can = later be used if Bitcoin Core is optimized (or extended with new = functionality) and see how it affects the min spec? Basically, with any = reasonably big change, one of the first questions could be: =E2=80=9CHow = does this change affect min spec?" >>=20 >> For example, currently OpenSSL is used to verify the signatures in = the transactions. The new secp256k1 implementation is several times = faster than (depending on CPU architecture, I=E2=80=99m sure) = OpenSSL=E2=80=99s implementation. So it would result in faster = verification time. This can then result in the following things; either = network I/O and CPU requirements are adjusted downward in the min spec = (you can run the new Bitcoin Core on a cheaper configuration), or other = parameters can be adjusted upwards (number of SigOps / transaction, = block size?), through proper rollout obviously. Since we know how min = spec is affected by these changes, they should be non-controversial by = default. Nobody running min spec is going to be affected by it, etc. >>=20 >> Every change that has a positive effect on min spec (do more on the = same hardware) basically pushes the need to start following any of the = curve laws (Nielsen, Moore) forward. No need for miners / node operators = to upgrade. >>=20 >> Once we hit what we call SOL (Speed Of Light, the fastest something = can go on a specific platform) it=E2=80=99s time to start looking at = periodically adjusting min spec upwards, or by that time maybe it=E2=80=99= s possible to use conservative plots of the curve laws as a basis. >>=20 >> Lastly, a benchmark test could be developed that can tell everyone = running Bitcoin Core how their setup compares to the min spec and how = long they can expect to run on this setup. >>=20 >> What do you guys think? >>=20 >>=20 >> jp >=20 >=20 >=20 >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >=20 >=20 > -- > Henning Kopp > Institute of Distributed Systems > Ulm University, Germany >=20 > Office: O27 - 3402 > Phone: +49 731 50-24138 > Web: http://www.uni-ulm.de/in/vs/~kopp --Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVlPdEAAoJEG93Vo4Z7tpFmYgQAMYaSUrped38/ASQnjME/jG7 ojcykAFIAArlHKh4qLhka7JnsZ47i3dYrzEZN0IOUGyOANnaa7zddsfDo29uT89O kk3t/LSE04Bdk/e2J18cwnqsp8yfdhQyJ+XyOLzwVtaSUuwm5SVZd0RwkScB+Crq bm8muzut/2IDD+RTCmuXnBBLSp1Hss1tnDuhv1sQh9w1UAfh278Z6XHtLu/Qgm6y 45DTWi8btvST7I68CGY5ftYOQOxNGGLz7ZLRSVwrxBDwGtU0NS4p4OUu/L5Vt/lZ 5f2lDFUOCsyuXVBfmm8eozMECTCm1aCneEwOKt9k0B4q6hv8YYj154qFzPn5mVYp d5SEHO7riSKNFYcqC/pWYu7rFInTxw2BRWrs1gjDA7AlwFHjd+GTTkX5H/fLMAe1 atyI+4xFVbCAE4HI9K0yAXi9oJ9uo/tC8qaXSSfpOTWhpXG+iACKpA6BHJJg7Q7s eeu5gNKNSF/AWBYykcuHwBlI9dVOiDtDifah8XpkF7jeqwMN0sFnZdBqQxuSGrFy tihcJ2yC3e8kciLJ6ls9Y1rZbobi/7RBTATKK7VwatIcR+Wv2FxULJumKwtENYcu kdZj5njc8gdMpcditWT0h/cpX6kHCbA+E4uqu+LVDCV2QLaIbalQo5ZHjOqC9EP5 D6Uzafx1EHzHdTxOW/9E =9JCZ -----END PGP SIGNATURE----- --Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9--