From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TtfMI-0005U2-6G for bitcoin-development@lists.sourceforge.net; Fri, 11 Jan 2013 14:13:22 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.170 as permitted sender) client-ip=209.85.214.170; envelope-from=mh.in.england@gmail.com; helo=mail-ob0-f170.google.com; Received: from mail-ob0-f170.google.com ([209.85.214.170]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TtfMG-0002pR-Hz for bitcoin-development@lists.sourceforge.net; Fri, 11 Jan 2013 14:13:22 +0000 Received: by mail-ob0-f170.google.com with SMTP id wp18so1766315obc.15 for ; Fri, 11 Jan 2013 06:13:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.88.3 with SMTP id bc3mr53741737obb.8.1357913595216; Fri, 11 Jan 2013 06:13:15 -0800 (PST) Sender: mh.in.england@gmail.com Received: by 10.76.128.139 with HTTP; Fri, 11 Jan 2013 06:13:15 -0800 (PST) In-Reply-To: References: <20121121151534.GA5540@vps7135.xlshosting.net> <1353523117.1085.14.camel@localhost.localdomain> <20121127211019.GA22701@vps7135.xlshosting.net> <1357876751.1740.4.camel@localhost.localdomain> Date: Fri, 11 Jan 2013 15:13:15 +0100 X-Google-Sender-Auth: 0Nkh8VdhW7BfJ-TIVuDLzNkl-1Q Message-ID: From: Mike Hearn To: Jeff Garzik Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TtfMG-0002pR-Hz Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 14:13:22 -0000 Oh, one last stat - syncing the entire chain with a wallet containing two keys and a 0.0001 FP rate (one or two FPs every 5 blocks or so) resulted in a download of about 46mb of data. On Fri, Jan 11, 2013 at 3:11 PM, Mike Hearn wrote: > I did some very rough initial performance tests. > > Syncing from a local peer gives me about 50 blocks per second in the > later parts of the chain (post SD), which is about a 10-20x speedup > over what I could do before. This is on a MacBook Pro. But at those > points it's clearly bottlenecked by bitcoind which has saturated its > CPU core. This makes sense - the filtering is much more server than > client intensive because every transaction in every block has to be > loaded and checked. > > I think filtering can be fairly well parallelized on the server side. > So the current 10-20x speedup could potentially be larger if the > server becomes more efficient at scanning and filtering blocks. It's > still a very nice win for now, especially bandwidth wise. And if Matt > makes the mempool command filtered it solves a common usability > problem as well. > > Once we get this code in, merged and rolled out I think what we need > for bloom v2 is clear: > > - Multi-thread the filtering process in bitcoind so transactions can > be checked in parallel. A 4-core server would then get 4x faster at > filtering blocks and assuming it's not too busy doing other stuff we > could maybe sync at more like 200 blocks per second, which is cool ... > more than a days worth of history for each second of syncing. > > - Make the client smarter so the FP rate is adapted during the sync > process. An FP rate that makes sense post-SD results in no false > positives pre-SD, more or less. > > - Make the client shard its wallet keys over multiple peers, for > better privacy. > > - Make the client suck down filtered blocks in parallel from multiple > peers, for better speed. > > As it seems the bottleneck for chain sync is now CPU time, the latter > point may be the most important from a practical perspective. > > On Fri, Jan 11, 2013 at 6:02 AM, Jeff Garzik wrote: >> On Thu, Jan 10, 2013 at 10:59 PM, Matt Corallo wrote: >>> Ive been missing lately, when is 0.8 targeted for freeze? >> >> 0.8rc1 will probably happen when the core ultraprune/leveldb stuff is stable. >> >> -- >> Jeff Garzik >> exMULTI, Inc. >> jgarzik@exmulti.com