public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Jeff Garzik <jgarzik@exmulti.com>
To: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: [Bitcoin-development] On bitcoin testing
Date: Tue, 9 Oct 2012 19:12:04 -0400	[thread overview]
Message-ID: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com> (raw)

Copying from a response posted to "Bitcoin software testing effort"
https://bitcointalk.org/index.php?topic=117487.0 as it is relevant to
a recent thread here...

Any level of testing is useful and appreciated.  Various types of
testing that are helpful:

* "it works" testing:  Simply run the latest Release Candidate (or
latest version, if released).  Make sure all the basics work (for
whatever definition of "basics" you desire).  This is the level most
accessible to casual users.
* Major features testing:  Develop a short checklist of must-work
features, and organize volunteers to work together and go through that
checklist, item by item.  Test each major feature on each major
platform.
* Stress and fuzz testing:  Attempt to "stress" the system somehow, or
randomly corrupt bits of data.  See what breaks.
* Regression testing:  Record bugs fixed, and develop automated test
cases that successfully reproduce the bugs on older versions, and
verify newer versions remain fixed.
* Unit function testing:  Rigorously exercise each C++ class to ensure
it behaves as expected at a micro level.
* Full peer automated testing:  Automated testing of RPC and P2P
functions is non-existent, because of the difficulty in doing so.
Find a solution to this problem.
* Data-driven tests: If possible, write software-neutral, data-driven
tests.  This enables clients other than the reference one (Satoshi
client) to be tested.  Embed tests in testnet3 chain, if possible.


The community at large can be a big help simply by doing the first
item:  download and run the Release Candidates and the latest version,
and report any problems.  Even reporting success is fine by me, for
example: "Version 0.7.1 works for me on Windows 7/32-bit" posted on a
forum thread.

It is always very difficult to organize any sort of testing regime
with open source volunteers that come and go.  Each volunteer chooses
their level of involvement.  Any amount of testing and test-case
writing, large or small, is helpful to bitcoin.

-- 
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com



             reply	other threads:[~2012-10-09 23:12 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-10-09 23:12 Jeff Garzik [this message]
2012-10-09 23:42 ` [Bitcoin-development] On bitcoin testing Arklan Uth Oslin
2012-10-10  0:03 ` Gregory Maxwell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com \
    --to=jgarzik@exmulti.com \
    --cc=bitcoin-development@lists.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox