From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TtfKs-0005K3-Bp for bitcoin-development@lists.sourceforge.net; Fri, 11 Jan 2013 14:11:54 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.178 as permitted sender) client-ip=209.85.214.178; envelope-from=mh.in.england@gmail.com; helo=mail-ob0-f178.google.com; Received: from mail-ob0-f178.google.com ([209.85.214.178]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TtfKo-0002Wn-Jp for bitcoin-development@lists.sourceforge.net; Fri, 11 Jan 2013 14:11:54 +0000 Received: by mail-ob0-f178.google.com with SMTP id eh20so1768684obb.9 for ; Fri, 11 Jan 2013 06:11:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.60.29.70 with SMTP id i6mr46385697oeh.38.1357913505299; Fri, 11 Jan 2013 06:11:45 -0800 (PST) Sender: mh.in.england@gmail.com Received: by 10.76.128.139 with HTTP; Fri, 11 Jan 2013 06:11:44 -0800 (PST) In-Reply-To: References: <20121121151534.GA5540@vps7135.xlshosting.net> <1353523117.1085.14.camel@localhost.localdomain> <20121127211019.GA22701@vps7135.xlshosting.net> <1357876751.1740.4.camel@localhost.localdomain> Date: Fri, 11 Jan 2013 15:11:44 +0100 X-Google-Sender-Auth: llF7-Of9-7B__9_YzOxfYH0H2_I Message-ID: From: Mike Hearn To: Jeff Garzik Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TtfKo-0002Wn-Jp Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 14:11:54 -0000 I did some very rough initial performance tests. Syncing from a local peer gives me about 50 blocks per second in the later parts of the chain (post SD), which is about a 10-20x speedup over what I could do before. This is on a MacBook Pro. But at those points it's clearly bottlenecked by bitcoind which has saturated its CPU core. This makes sense - the filtering is much more server than client intensive because every transaction in every block has to be loaded and checked. I think filtering can be fairly well parallelized on the server side. So the current 10-20x speedup could potentially be larger if the server becomes more efficient at scanning and filtering blocks. It's still a very nice win for now, especially bandwidth wise. And if Matt makes the mempool command filtered it solves a common usability problem as well. Once we get this code in, merged and rolled out I think what we need for bloom v2 is clear: - Multi-thread the filtering process in bitcoind so transactions can be checked in parallel. A 4-core server would then get 4x faster at filtering blocks and assuming it's not too busy doing other stuff we could maybe sync at more like 200 blocks per second, which is cool ... more than a days worth of history for each second of syncing. - Make the client smarter so the FP rate is adapted during the sync process. An FP rate that makes sense post-SD results in no false positives pre-SD, more or less. - Make the client shard its wallet keys over multiple peers, for better privacy. - Make the client suck down filtered blocks in parallel from multiple peers, for better speed. As it seems the bottleneck for chain sync is now CPU time, the latter point may be the most important from a practical perspective. On Fri, Jan 11, 2013 at 6:02 AM, Jeff Garzik wrote: > On Thu, Jan 10, 2013 at 10:59 PM, Matt Corallo wrote: >> Ive been missing lately, when is 0.8 targeted for freeze? > > 0.8rc1 will probably happen when the core ultraprune/leveldb stuff is stable. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti.com