From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Sh0x0-00070R-4D for bitcoin-development@lists.sourceforge.net; Tue, 19 Jun 2012 16:06:42 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 74.125.82.175 as permitted sender) client-ip=74.125.82.175; envelope-from=mh.in.england@gmail.com; helo=mail-we0-f175.google.com; Received: from mail-we0-f175.google.com ([74.125.82.175]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Sh0wu-0003UV-IW for bitcoin-development@lists.sourceforge.net; Tue, 19 Jun 2012 16:06:42 +0000 Received: by werg55 with SMTP id g55so5181254wer.34 for ; Tue, 19 Jun 2012 09:06:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.102.228 with SMTP id fr4mr4642932wib.6.1340121990386; Tue, 19 Jun 2012 09:06:30 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.216.254.232 with HTTP; Tue, 19 Jun 2012 09:06:30 -0700 (PDT) In-Reply-To: References: Date: Tue, 19 Jun 2012 18:06:30 +0200 X-Google-Sender-Auth: 44Bucve7j4PzuxdayNYbCI9NtA4 Message-ID: From: Mike Hearn To: Gavin Andresen Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Sh0wu-0003UV-IW Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] LevelDB benchmarking X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jun 2012 16:06:42 -0000 > What problem does it solve? Primarily that block verification and therefore propagation is too slow because it's very CPU and IO intensive. The CPU work can be multi-threaded. The IO work, not as much. As Bitcoin grows we need to scale the nodes. Eventually there may be multi-machine nodes, but for now we can buy more time by making the existing nodes faster. I don't see this as a replacement for moving users to SPV clients. Obviously, otherwise I would not be writing one ;) > If the problem it will solve is the "too easy to get a DB_RUNRECOVERY > error" because bdb is fragile when it comes to its environment... then > LevelDB looks very interesting. I have no experience with how robust LevelDB is. It has an API call to try and repair the database and I know from experience that BigTable is pretty solid. But that doesn't mean LevelDB is. > If the problem is bdb is creaky and old and has obscure semantics and > a hard-to-work-with API, then yes, lets switch (I'm easily seduced by > a pretty API and blazing fast performance). The code is a lot simpler for sure. > As long as it compiles and runs on mac/windows/linux that doesn't > really worry me. It was refactored out of BigTable and made standalone for usage in Chrome. Therefore it's as portable as Chrome is. Mac/Windows/Linux should all work. Solaris, I believe, may need 64 bit binaries to avoid low FD limits. > Lack of infrastructure because it is new does worry me; for example, > could I rework bitcointools to read the LevelDB blockchain? =C2=A0(are > there python bindings for LevelDB?) Yes: http://code.google.com/p/py-leveldb/ First look at the code is here, but it's not ready for a pull req yet, and I'll force push over it a few times to get it into shape. So don't branch: https://github.com/mikehearn/bitcoin/commit/2b601dd4a0093f834084241735d84d8= 4e484f183 It has misc other changes I made whilst profiling, isn't well commented enough, etc.