From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Sgzzk-000395-9z for bitcoin-development@lists.sourceforge.net; Tue, 19 Jun 2012 15:05:28 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.215.175 as permitted sender) client-ip=209.85.215.175; envelope-from=gavinandresen@gmail.com; helo=mail-ey0-f175.google.com; Received: from mail-ey0-f175.google.com ([209.85.215.175]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Sgzzj-0005sV-G9 for bitcoin-development@lists.sourceforge.net; Tue, 19 Jun 2012 15:05:28 +0000 Received: by eaal1 with SMTP id l1so2112934eaa.34 for ; Tue, 19 Jun 2012 08:05:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.14.189.3 with SMTP id b3mr4299574een.223.1340118321207; Tue, 19 Jun 2012 08:05:21 -0700 (PDT) Received: by 10.14.3.66 with HTTP; Tue, 19 Jun 2012 08:05:21 -0700 (PDT) In-Reply-To: References: Date: Tue, 19 Jun 2012 11:05:21 -0400 Message-ID: From: Gavin Andresen To: Mike Hearn Content-Type: text/plain; charset=ISO-8859-1 X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gavinandresen[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.0 AWL AWL: From: address is in the auto white-list X-Headers-End: 1Sgzzj-0005sV-G9 Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] LevelDB benchmarking X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jun 2012 15:05:28 -0000 > OK, to make progress on this work I need a few decisions (Gavin?) > > 1) Shall we do it? What problem does it solve? If the problem it will solve is "it will only take 4 hours to download the entire blockchain next year instead of taking 16 hours" then no, I don't think we should do it, both 4 and 16 hours to get fully up and running is too long. If the problem it will solve is the "too easy to get a DB_RUNRECOVERY error" because bdb is fragile when it comes to its environment... then LevelDB looks very interesting. If the problem is bdb is creaky and old and has obscure semantics and a hard-to-work-with API, then yes, lets switch (I'm easily seduced by a pretty API and blazing fast performance). > 2) LevelDB is obscure, new and has a very minimalist build system. It > supports "make" but not "make install", for example, and is unlikely > to be packaged. It's also not very large. I suggest we just check the > source into the main Bitcoin tree and link it statically rather than > complicate the build. As long as it compiles and runs on mac/windows/linux that doesn't really worry me. I just tried it, and it compiled quickly with no complaints on my mac. Lack of infrastructure because it is new does worry me; for example, could I rework bitcointools to read the LevelDB blockchain? (are there python bindings for LevelDB?) > 3) As the DB format would change and a slow migration period > necessary, any other tweaks to db format we could make at the same > time? Right now the key/values are the same as before, though using > satoshi serialization for everything is a bit odd. Satoshi rolled his own network serialization because he didn't trust existing serialization solutions to be 100% secure against remote exploits. Then it made sense to use the same solution for disk serialization; I don't see a compelling reason to switch to some other serialization scheme. Modifying the database schema during migration to better support applications like InstaWallet (tens of thousands of separate wallets) or something like Pieter's ultra-pruning makes sense. -- -- Gavin Andresen