From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1VLt0z-0002Mn-1b for bitcoin-development@lists.sourceforge.net; Tue, 17 Sep 2013 11:00:17 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.53 as permitted sender) client-ip=209.85.214.53; envelope-from=mh.in.england@gmail.com; helo=mail-bk0-f53.google.com; Received: from mail-bk0-f53.google.com ([209.85.214.53]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1VLt0t-000071-SU for bitcoin-development@lists.sourceforge.net; Tue, 17 Sep 2013 11:00:17 +0000 Received: by mail-bk0-f53.google.com with SMTP id d7so2118389bkh.12 for ; Tue, 17 Sep 2013 04:00:05 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.205.10.132 with SMTP id pa4mr28399825bkb.15.1379415604951; Tue, 17 Sep 2013 04:00:04 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.204.72.69 with HTTP; Tue, 17 Sep 2013 04:00:04 -0700 (PDT) Date: Tue, 17 Sep 2013 13:00:04 +0200 X-Google-Sender-Auth: cEc01CnfJTfAOolO8fewsadxPZg Message-ID: From: Mike Hearn To: Bitcoin Dev Content-Type: multipart/alternative; boundary=20cf30223b1321abc604e6923924 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1VLt0t-000071-SU Subject: [Bitcoin-development] Faster databases than LevelDB X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Sep 2013 11:00:17 -0000 --20cf30223b1321abc604e6923924 Content-Type: text/plain; charset=UTF-8 LevelDB is fast - very fast if you give it enough CPU time and disk seeks. But it's not the last word in performance. HyperLevelDB is a forked LevelDB with some changes, mostly, finer grained locking and changes to how compaction works: http://hyperdex.org/performance/leveldb/ However, it comes with a caveat - one of the changes they made is to take away write throttling if compaction falls behind, the app itself is expected to do that. Sophia is a competitor to LevelDB. The website claims that in benchmarks it completely smokes LevelDB. I have not explored how it does this or tried to replicate their benchmarks myself: http://sphia.org/index.html http://sphia.org/benchmarks.html It's written in C and BSD licensed. As an example of the kind of speedup they claim to be capable of, they say LevelDB could do 167,476 random reads per second on their SSD based machine. Sophia could do 438,084 reads/sec. Random reads are of course the most interesting for us because that's what UTXO lookups involve. They also compare against HyperLevelDB, where the differences are much less pronounced and actually HyperLevelDB appears to be able to do random writes faster than Sophia. --20cf30223b1321abc604e6923924 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
LevelDB is fast - very fast if you give it enough CPU time= and disk seeks. But it's not the last word in performance.

HyperLevelDB is a forked LevelDB with some changes, mostly, finer g= rained locking and changes to how compaction works:


How= ever, it comes with a caveat - one of the changes they made is to take away= write throttling if compaction falls behind, the app itself is expected to= do that.

Sophia is a competitor to LevelDB. The website claims t= hat in benchmarks it completely smokes LevelDB. I have not explored how it = does this or tried to replicate their benchmarks myself:


It's written = in C and BSD licensed.

As an example of the kind of speedup they claim to be c= apable of, they say LevelDB could do 167,476 random reads per second on the= ir SSD based machine. Sophia could do 438,084 reads/sec. Random reads are o= f course the most interesting for us because that's what UTXO lookups i= nvolve.

They also compare against HyperLevelDB, where the diffe= rences are much less pronounced and actually HyperLevelDB appears to be abl= e to do random writes faster than Sophia.


--20cf30223b1321abc604e6923924--