From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1StyA8-00011T-9a for bitcoin-development@lists.sourceforge.net; Wed, 25 Jul 2012 09:45:48 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of mac.com designates 17.172.81.1 as permitted sender) client-ip=17.172.81.1; envelope-from=gronager@mac.com; helo=st11p00mm-asmtpout002.mac.com; Received: from st11p00mm-asmtpout002.mac.com ([17.172.81.1]) by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1StyA2-0005O2-IJ for bitcoin-development@lists.sourceforge.net; Wed, 25 Jul 2012 09:45:48 +0000 MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Received: from [109.105.106.237] (unknown [109.105.106.237]) by st11p00mm-asmtp002.mac.com (Oracle Communications Messaging Exchange Server 7u4-22.01 64bit (built Apr 21 2011)) with ESMTPSA id <0M7P00DIDN3Y0A40@st11p00mm-asmtp002.mac.com> for bitcoin-development@lists.sourceforge.net; Wed, 25 Jul 2012 09:45:36 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.7.7855,1.0.260,0.0.0000 definitions=2012-07-25_04:2012-07-24, 2012-07-25, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1203120001 definitions=main-1207250042 From: =?iso-8859-1?Q?Michael_Gr=F8nager?= In-reply-to: <500EFE00.7060200@mistfpga.net> Date: Wed, 25 Jul 2012 11:45:33 +0200 Content-transfer-encoding: quoted-printable Message-id: References: <500D73B9.5030806@mistfpga.net> <500EFE00.7060200@mistfpga.net> To: steve X-Mailer: Apple Mail (2.1278) X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gronager[at]mac.com) -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 SPF_PASS SPF: sender matches SPF record 1.0 MALFORMED_FREEMAIL Bad headers on message from free email service X-Headers-End: 1StyA2-0005O2-IJ Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Scalability issues X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 09:45:48 -0000 Hi Steve, I see dramatic differences in performance on virtual machines vs running = directly on the iron. I am not an expert in virtual machines, but it = seems to me that they are weak when it comes to disk i/o. And berkeley = DB, as used by bitcoin is a sucker for disk i/o. In top I easily hit = >1/#processors in top wa, meaning that the cpu doing the blockchain = download is just waiting for the disk all the time. I would like to do a test keeping database log files in memory. It = should not matter for durability of the wallet, as it flushes at each = write anyway. As for the blockindex, it will remain consistent, but = might be lagging some blocks behind at startup, which shouldn't really = matter (except that the same block could end up appearing twice in the = block00X files, inelegant, but not really a problem). Otherwise the system you describe (raid0 over 6 disks) should perform = like crazy wrt disk i/o, at least on par with SSD. It is your = virtualization I am worried about. Have a safe trip to down under! /M On 24/07/2012, at 21:56, steve wrote: > Hi Michael, >=20 > from what I have noticed, bitcoin blockchain download/verfication all > happens in 1 thread. (so multicores doesnt really help) >=20 > That said, I have never tried on an ssd. >=20 > What I do have is 6 SATA 6gbs configed as RAID0 Drives. > 32gb of ram. ubuntu 64 (yeah I know), this runs upto 16 VM's > (I have 4 of these) >=20 > However I have not tried to download the blockchain on the master os, > just in virtulisation. However, the dedicated machines that I have = been > using for benchmarking the VM's against is a q6600 8gb ram sata2 hdd - > Win 7 (seems faster than slackware...) to me it has always felt like > network bandwidth was the issue. I might instrument the bitcoin-qt = exe > to only pick low ping nodes (has someone already done this?) >=20 > I guess it is time to start some benchmarking (like the gpu comparison = page) >=20 > hte verification for the 5 past 5 days was negliglable. I am off on a > flight to australia tomorrrow, so I will set some breakpoints and do > some timings in a debugger. >=20 > This will all happen on an e-450 (wonderful machine!) >=20 > Thanks very much for your response. it would seem that I am 'doing it > wrong' :/ >=20 > cheers mate, >=20 > steve >=20 > (this message isnt signed because I have forgotten my password.) >=20 > On 24/07/2012 09:25, Michael Gr=F8nager wrote: >> Hi Steve, >>=20 >> 45-90 minutes - note that its numbers from March/April, so a bit >> longer today, but far, far away from the 12 hours. >>=20 >> I am using libcoin and the bitcoind build based on this. Libcoin is >> based on the Satoshi client, but refactured to use an async >> concurrency model. I also did a minor tweeks to the db parameters. It >> has earlier been tested up against Satoshi bitcoin where on some >> OS'es it performs similarly (at least on some linuxes) and on some >> faster (e.g. mac). >>=20 >> What is your CPU load during a block download ? (both initially/up to >> the point where verification sets in and after). The initial download >> is typically disk I/O bound, the verification stage CPU bound, though >> I lean to believe that even there it is disk I/O bound (at least on >> my system ~50% CPU load). What should be better in libcoin is the >> concurrency model. The Satoshi client uses a pure reentrant mutexes >> model, that is not generally believed to motivate the best coding >> practice nor performance, you might end up without the concurrency >> you initially strived for *). As mentioned earlier libcoin uses a >> pure async concurrency model (and so does libbitcoin btw). >>=20 >> I would like to stress again that these numbers will depend largely >> on the system running the test - I would call my laptop a bit over >> the average today (MB Pro, 2.66Ghz i7 dual core, 8GBRAM, 512GB SSD). >> But again 12 hours - I only reach such numbers on some of my VPS'es >> (linode 1024) that are known for notoriously slow disk I/O. (here I >> have a few % CPU load during the verification indicating indeed that >> the disk i/o is the culprit). >>=20 >> Cheers, >>=20 >> Michael >>=20 >>=20 >> *) I like this Dave Butenhof quote: "The biggest of all the big >> problems with recursive mutexes is that they encourage you to >> completely lose track of your locking scheme and scope. This is >> deadly. Evil. It's the "thread eater". You hold locks for the >> absolutely shortest possible time. Period. Always. If you're calling >> something with a lock held simply because you don't know it's held, >> or because you don't know whether the callee needs the mutex, then >> you're holding it too long. You're aiming a shotgun at your >> application and pulling the trigger. You presumably started using >> threads to get concurrency; but you've just PREVENTED concurrency." >>=20 >>=20 >>=20 >>=20 >> On 23/07/2012, at 17:54, steve wrote: >>=20 >> Hi Michael, >>=20 >> On 23/07/2012 10:00, Michael Gr=F8nager wrote: >>>>> I get a full blockchain from scratch in 45 minutes on my >>>>> laptop, /M >>>>>=20 >> Hang on a sec, in 45 minutes you can download the entire chain from=20= >> the genesis block? >>=20 >> I have been doing extensive testing in this area and would love to=20 >> know what is special about your setup (I have never had the entire=20 >> chain in under 12 hours, infact it is normally closerto 24.) I have >> an extensive setup of test machines, everything from e4300 to >> phenom2x6 to i5's. >>=20 >> as an example on an amd e-450 with 4gb ram, and approx 3gb/s >> internet connection it took 2 hours to sync the last 5 days. >>=20 >> Maybe i am missing something important... >>=20 >> Any additional information that you could provide to help me with=20 >> testing would be really appreciated. >>=20 >> cheers, >>=20 >> steve >>=20 >>>=20 >>> = --------------------------------------------------------------------------= ---- >>>=20 >>>=20 > Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and=20 >>> threat landscape has changed and how IT managers can respond. >>> Discussions will include endpoint security, mobile security and the >>> latest in malware threats. >>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/=20 >>> _______________________________________________ Bitcoin-development >>> mailing list Bitcoin-development@lists.sourceforge.net=20 >>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >>=20 >>=20 >>=20 >>=20 >>=20 >=20 >=20 > = --------------------------------------------------------------------------= ---- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and=20 > threat landscape has changed and how IT managers can respond. = Discussions=20 > will include endpoint security, mobile security and the latest in = malware=20 > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development