From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UOAmR-00080S-Vp for bitcoin-development@lists.sourceforge.net; Fri, 05 Apr 2013 17:50:28 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of bluematt.me designates 173.246.101.161 as permitted sender) client-ip=173.246.101.161; envelope-from=bitcoin-list@bluematt.me; helo=mail.bluematt.me; Received: from vps.bluematt.me ([173.246.101.161] helo=mail.bluematt.me) by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1UOAmQ-0004z4-LD for bitcoin-development@lists.sourceforge.net; Fri, 05 Apr 2013 17:50:27 +0000 Received: from [192.168.1.6] (dhcp00757.north-resnet.unc.edu [152.23.202.249]) by mail.bluematt.me (Postfix) with ESMTPSA id DF8345DF1; Fri, 5 Apr 2013 17:33:43 +0000 (UTC) Message-ID: <1365183223.2757.16.camel@localhost.localdomain> From: Matt Corallo To: Adam Ritter Date: Fri, 05 Apr 2013 13:33:43 -0400 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.6.4 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Score: -3.9 (---) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record -2.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain X-Headers-End: 1UOAmQ-0004z4-LD Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Integration testing for BitCoin X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2013 17:50:28 -0000 These tests are run on each pull requests and on each new commit to master. They arent very complete, but they do test a lot of block acceptance rules. https://github.com/TheBlueMatt/test-scripts Matt On Fri, 2013-04-05 at 12:24 -0500, Adam Ritter wrote: > Hey guys, > > I just bought some BitCoins after being lazy to do it for the last few > years, but also looked at the client code and the messages that are > going on this mailing list. > I saw that there are quite some unit tests, but I didn't find > integration test for BitCoin, and I believe that it's quite important > for the future of BitCoin (making the current code more stable, > testing attack scenarios, refactoring and extending code). > > I have wrote some integration tests before at other projects, and they > usually turned out useful, but I have 0 experience with the BitCoin > development and codebase. > I wrote a short document of what I think would be the safest way to do > the testing (but not yet the tests themselves, as I don't have enough > experience..I'd like to have something like testing that the wallets > are empty, and after somebody mines she'll have more money..after she > sends money to the other person, the other person will see it...things > like this, just to get to know the code base). > > What do you guys think? > The plan is here: > https://github.com/xiphias/bitcoin/blob/master/src/test/integration/README.md > Please feel free to comment/fork, I'll try to write all your replies > in the document as well. > > Also here's the text to make it easier to comment: > > Integration testing for bitcoin > ================================ > > Tests that simulate multiple bitcoin users and can verify that the > whole network of bitcoin clients work together > to achieve the goals of Bitcoin. Also maybe [System > testing](http://en.wikipedia.org/wiki/System_testing) > would be a better name for the tests, but I'm not sure. > > Goals > --------- > - Make the bitcoin code easier to refactor while increasing the guarantee > that it doesn't break the overall behaviour of the client. > - Make it easier to have multiple experimental coins (for example > LightCoin or PPCoin) in the codebase, while guaranteeing that the > original BitCoin protocol doesn't break > - Make it easy to test attack scenarios (like DOS, > releasing an incompatible BitCoin client), monitoring > - Have tests without (or at least minimal number of) unreadable > constants and unreadable fake data to make them easier to verify. > > Proposed implementation > ------------------------------------ > The first implementation should use the JSON-RPC interface and build > up as much verification > of BitCoin network as possible. It should be in C++, which makes it > easier to move to the second implementation. > The JSON-RPC calls should be hidden by a C++ interface. > > The second implementation should use the same interface that was used > for JSON-RPC, > but using the BitCoin code directly (while not breaking the JSON-RPC tests). > For this the BitCoin client has to be refactored as a library, > getting rid of all global variables (and having them in a data structure), > so that multiple BitCoin clients can be run in the same process. > > The improvement of the second implementation should have dependency injection > for the time and for finding/verifying a mined block, > so that the tests don't need to use real CPU power for mining, > and they can run faster and test more complex scenarios. > > ------------------------------------------------------------------------------ > Minimize network downtime and maximize team effectiveness. > Reduce network management and security costs.Learn how to hire > the most talented Cisco Certified professionals. Visit the > Employer Resources Portal > http://www.cisco.com/web/learning/employer_resources/index.html > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development