From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7CBCB726 for ; Thu, 23 Jul 2015 17:19:11 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f175.google.com (mail-wi0-f175.google.com [209.85.212.175]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8C1BD1A2 for ; Thu, 23 Jul 2015 17:19:10 +0000 (UTC) Received: by wibxm9 with SMTP id xm9so3163780wib.1 for ; Thu, 23 Jul 2015 10:19:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-type; bh=pNGcc3DKLD8XvwF9xU8IVny8NWAZgHAiGF0ECw2Xqqc=; b=yFTdz/KEIi51sCOroBdRJDPqI1Y9H1obS0l6MhAAGgNV4dLfj0iuSuMAwlWKul7CVa YlfcNn/DtsFuAUthHTEBX0LhOjmiekDpbymgk5k7j9rFBh+rPOr2dB+2fAxx65gxQy96 4v5qlkAYh6h+gqeZaSVgc6RwuQ2gxATzoDok40O/S8d583B/eEZEueGUOjwWGQemw0Ue rrr3vPriiFUlN4rDnHORd6PUv4bmSAejUBwh+nsbCfl3PmPOH0ALH9UIf7lv/UmZxv5/ DR/Fs+T3RDoJKkg2O14AuLKFo2vOeOtwrMNKzLQUirPtLGbYfjum6nL1xRyiHlw5O6X2 6PtQ== X-Received: by 10.180.186.34 with SMTP id fh2mr1634494wic.94.1437671949337; Thu, 23 Jul 2015 10:19:09 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?Sm9zZXBoIEdsZWFzb24g4pGI?= Date: Thu, 23 Jul 2015 17:18:59 +0000 Message-ID: To: Slurms MacKenzie , bitcoin-dev@lists.linuxfoundation.org Content-Type: multipart/alternative; boundary=001a11c2630ad80adc051b8e15b6 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Electrum Server Speed Test X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jul 2015 17:19:11 -0000 --001a11c2630ad80adc051b8e15b6 Content-Type: text/plain; charset=UTF-8 I have concerns about the performance of the Electrum server software as well. It seems to load data one block at a time (which normally makes sense) and I think it is even single threaded on transactions inside the block. To try to addresses these issues, I made my own implementation of the electrum server. It doesn't support UTXO (yet) but happily interacts with all the clients I've tested. It is heavily multithreaded, uses mongodb as a key value store and bitcoinj for block and transaction parsing. https://github.com/fireduck64/jelectrum You can hit a running instance at: b.1209k.com:50002:s or b.1209k.com:50001:t A synced node uses 347G of mongodb storage. Here are the recent blocks imported, with number of transactions and import time. http://pastebin.com/cfW3C2L6 These times are based on having mongodb on SSD. The CPU is 8 core Intel(R) Xeon(R) CPU E5430 @ 2.66GHz I'd be happy to help with anything you need to evaluate it. On Thu, Jul 23, 2015 at 9:01 AM Slurms MacKenzie via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Similar to the Bitcoin Node Speed Test, this is a quick quantitative look > at how the Electrum server software handles under load. The Electrum wallet > is extremely popular, and the distributed servers which power it are all > hosted by volunteers without budget. The server requires a fully indexed > Bitcoin Core daemon running, and produces sizable external index in order > to allow SPV clients to quickly retrieve their history. > > 3.9G electrum/utxo > 67M electrum/undo > 19G electrum/hist > 1.4G electrum/addr > 24G electrum/ > > Based on my own logs produced by the electrum-server console, it takes > this server (Xeon, lots of memory, 7200 RPM RAID) approximately 3.7 minutes > per megabyte of block to process into the index. This seems to hold true > through the 10 or so blocks I have in my scroll buffer, the contents of > blocks seem to be of approximately the same processing load. Continuing > this trend with the current inter-block time of 9.8 minutes, an > electrum-server instance running on modest-high end dedicated server is > able to support up to 2.64 MB block sizes before permanently falling behind > the chain. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --001a11c2630ad80adc051b8e15b6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I have concerns about the performance of the Electrum serv= er software as well.=C2=A0 It seems to load data one block at a time (which= normally makes sense) and I think it is even single threaded on transactio= ns inside the block.

To try to addresses these issues, I= made my own implementation of the electrum server.=C2=A0 It doesn't su= pport UTXO (yet) but happily interacts with all the clients I've tested= .=C2=A0 It is heavily multithreaded, uses mongodb as a key value store and = bitcoinj for block and transaction parsing.


You can hit a = running instance at:
b.1209k.com:50002:s
or
b.1209k.com:50001:t

A synced node uses 347G= of mongodb storage. =C2=A0

Here are the recent bl= ocks imported, with number of transactions and import time.
http://pastebin.com/cfW3C2L6
These times are based on having mongodb on SSD.
The CPU i= s 8 core=C2=A0Intel(R) Xeon(R) CPU E5430 =C2=A0@ 2.66GHz

I'd be happy to help with anything you need to evaluate it.


On Th= u, Jul 23, 2015 at 9:01 AM Slurms MacKenzie via bitcoin-dev <bitcoin-dev= @lists.linuxfoundation.org> wrote:
Similar to the Bitcoin Node Speed Test, this is a quick quantitative= look at how the Electrum server software handles under load. The Electrum = wallet is extremely popular, and the distributed servers which power it are= all hosted by volunteers without budget. The server requires a fully index= ed Bitcoin Core daemon running, and produces sizable external index in orde= r to allow SPV clients to quickly retrieve their history.

=C2=A0 =C2=A0 3.9G=C2=A0 =C2=A0 electrum/utxo
=C2=A0 =C2=A0 67M=C2=A0 =C2=A0 =C2=A0electrum/undo
=C2=A0 =C2=A0 19G=C2=A0 =C2=A0 =C2=A0electrum/hist
=C2=A0 =C2=A0 1.4G=C2=A0 =C2=A0 electrum/addr
=C2=A0 =C2=A0 24G=C2=A0 =C2=A0 =C2=A0electrum/

Based on my own logs produced by the electrum-server console, it takes this= server (Xeon, lots of memory, 7200 RPM RAID) approximately 3.7 minutes per= megabyte of block to process into the index. This seems to hold true throu= gh the 10 or so blocks I have in my scroll buffer, the contents of blocks s= eem to be of approximately the same processing load. Continuing this trend = with the current inter-block time of 9.8 minutes, an electrum-server instan= ce running on modest-high end dedicated server is able to support up to 2.6= 4 MB block sizes before permanently falling behind the chain.
_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev
--001a11c2630ad80adc051b8e15b6--