public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Adam Ritter <aritter@gmail.com>
To: bitcoin-development@lists.sourceforge.net
Subject: [Bitcoin-development] Integration testing for BitCoin
Date: Fri, 5 Apr 2013 12:24:55 -0500	[thread overview]
Message-ID: <CAKuKjyUFBuPMPRV6R-u0iTa=8DWMN9vqdOxnr8o8kxg9rJtVBA@mail.gmail.com> (raw)

Hey guys,

I just bought some BitCoins after being lazy to do it for the last few
years, but also looked at the client code and the messages that are
going on this mailing list.
I saw that there are quite some unit tests, but I didn't find
integration test for BitCoin, and I believe that it's quite important
for the future of BitCoin (making the current code more stable,
testing attack scenarios, refactoring and extending code).

I have wrote some integration tests before at other projects, and they
usually turned out useful, but I have 0 experience with the BitCoin
development and codebase.
I wrote a short document of what I think would be the safest way to do
the testing (but not yet the tests themselves, as I don't have enough
experience..I'd like to have something like testing that the wallets
are empty, and after somebody mines she'll have more money..after she
sends money to the other person, the other person will see it...things
like this, just to get to know the code base).

What do you guys think?
The plan is here:
https://github.com/xiphias/bitcoin/blob/master/src/test/integration/README.md
Please feel free to comment/fork, I'll try to write all your replies
in the document as well.

Also here's the text to make it easier to comment:

Integration testing for bitcoin
================================

Tests that simulate multiple bitcoin users and can verify that the
whole network of bitcoin clients work together
to achieve the goals of Bitcoin. Also maybe [System
testing](http://en.wikipedia.org/wiki/System_testing)
would be a better name for the tests, but I'm not sure.

Goals
---------
- Make the bitcoin code easier to refactor while increasing the guarantee
 that it doesn't break the overall behaviour of the client.
- Make it easier to have multiple experimental coins (for example
LightCoin or PPCoin) in the codebase, while guaranteeing that the
original BitCoin protocol doesn't break
- Make it easy to test attack scenarios (like DOS,
 releasing an incompatible BitCoin client), monitoring
- Have tests without (or at least minimal number of) unreadable
constants and unreadable fake data to make them easier to verify.

Proposed implementation
------------------------------------
The first implementation should use the JSON-RPC interface and build
up as much verification
of BitCoin network as possible. It should be in C++, which makes it
easier to move to the second implementation.
The JSON-RPC calls should be hidden by a C++ interface.

The second implementation should use the same interface that was used
for JSON-RPC,
but using the BitCoin code directly (while not breaking the JSON-RPC tests).
For this the BitCoin client has to be refactored as a library,
getting rid of all global variables (and having them in a data structure),
so that multiple BitCoin clients can be run in the same process.

The improvement of the second implementation should have dependency injection
for the time and for finding/verifying a mined block,
so that the tests don't need to use real CPU power for mining,
and they can run faster and test more complex scenarios.



             reply	other threads:[~2013-04-05 17:25 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-05 17:24 Adam Ritter [this message]
2013-04-05 17:33 ` [Bitcoin-development] Integration testing for BitCoin Matt Corallo
2013-04-05 17:42 ` Gregory Maxwell
2013-04-05 19:29   ` Adam Ritter
2013-04-06 12:21     ` Mike Hearn
2013-04-07 13:50       ` Adam Ritter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAKuKjyUFBuPMPRV6R-u0iTa=8DWMN9vqdOxnr8o8kxg9rJtVBA@mail.gmail.com' \
    --to=aritter@gmail.com \
    --cc=bitcoin-development@lists.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox