From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1QZroq-0005uZ-4V for bitcoin-development@lists.sourceforge.net; Thu, 23 Jun 2011 21:52:12 +0000 X-ACL-Warn: Received: from vm136.rz.uni-osnabrueck.de ([131.173.16.11] helo=smtp-auth.serv.Uni-Osnabrueck.DE) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1QZrom-0008BA-7J for bitcoin-development@lists.sourceforge.net; Thu, 23 Jun 2011 21:52:12 +0000 Received: from msmtp-using-host (0xbcb2ab7c.ronqu1.dynamic.dsl.tele.dk [188.178.171.124] (may be forged)) (authenticated bits=0) by smtp-auth.serv.Uni-Osnabrueck.DE (8.13.8/8.13.8) with ESMTP id p5NLpx8B031954 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 23 Jun 2011 23:52:00 +0200 From: jan@uos.de Date: Thu, 23 Jun 2011 23:51:43 +0200 To: Bitcoin Dev Message-ID: <20110623215143.GA3351@dax.lan.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.6.23.213914 (Univ. Osnabrueck) X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report= BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1900_1999 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, FROM_MISSING 0, NO_URI_FOUND 0, __CD 0, __CT 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __PHISH_PHRASE1_B 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __TO_MALFORMED_2 0, __USER_AGENT 0 X-PMX-Spam-Level: IIIIIIII X-Spam-Score: 0.3 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.3 MAY_BE_FORGED Relay IP's reverse DNS does not resolve to IP X-Headers-End: 1QZrom-0008BA-7J Subject: [Bitcoin-development] Speeding up "getbalance " calls X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2011 21:52:12 -0000 Hi there! Instawallet has enjoyed steady growth and I'm running into a bottleneck now with "getbalance " taking quite some time to complete. My understanding is, that this is because bitcoind runs through all relevant transactions each time anew to compute the balance. I was hoping the list could give me some pointers/ideas on how I can improve this. I might start to do account handling completely in my application at some point, but for now I would like to continue letting bitcoind handle this, so I don't have to worry about thinks like block chain reorgs. Basically I don't have enough familiarity with the source code to feel confident about doing fairly invasive changes. So right now I'm planning to implement a very simple cache (directly in bitcoind), which just caches calls to getbalance and simply invalidates the whole cache as soon as a new block or a transaction that affects the wallet comes in. I'm hoping this will help a little bit, although with blocks arriving every 10 minutes on average it's not really the perfect solution. Maybe one step better would be to only invalidate cache entries that are affected by a new transaction or by transactions arriving in a block. This would need to take block chain reorgs into account though, which seems fairly complicated. Maybe I could simply invalidate the whole cache on reorgs, which should be seldom enough that it's not a big problem. Where would be a good place in the source code to hook into to notice block chain reorgs? The perfect solution would be, of course, if bitcoind could keep running balances of all accounts and update them as new information becomes available. But as I said, I don't feel that I have a good overview of all the corner cases to make such an improvement. I guess an extensive test suite testing all sorts of esoteric block chain reorgs would really be a nice thing to have. :-) Regards! Jan