From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TI1ks-0006gG-46 for bitcoin-development@lists.sourceforge.net; Sat, 29 Sep 2012 18:27:10 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of mistfpga.net designates 208.91.199.213 as permitted sender) client-ip=208.91.199.213; envelope-from=steve@mistfpga.net; helo=us2.outbound.mailhostbox.com; Received: from us2.outbound.mailhostbox.com ([208.91.199.213]) by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1TI1kr-0000yP-0J for bitcoin-development@lists.sourceforge.net; Sat, 29 Sep 2012 18:27:10 +0000 Received: from [10.10.10.55] (5ad2e75a.bb.sky.com [90.210.231.90]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: steve@mistfpga.net) by us2.outbound.mailhostbox.com (Postfix) with ESMTPSA id DA6A71E981D5; Sat, 29 Sep 2012 18:27:00 +0000 (GMT) Message-ID: <50673D69.5040105@mistfpga.net> Date: Sat, 29 Sep 2012 19:26:49 +0100 From: steve User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Gavin Andresen References: <5061F8CC.9070906@mistfpga.net> <1348605677.2284.2.camel@localhost.localdomain> <5062F4F8.6040504@mistfpga.net> <506301AC.90101@mistfpga.net> <50633F02.6030807@mistfpga.net> In-Reply-To: X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-CTCH-Spam: Unknown X-CTCH-VOD: Unknown X-CTCH-RefID: str=0001.0A02020A.50673D76.0078, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-Scanned-By: MIMEDefang 2.72 on 208.91.199.211 X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TI1kr-0000yP-0J Cc: Bitcoin Development List Subject: Re: [Bitcoin-development] Bitcoin Testing Project X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 18:27:10 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Gavin, Sorry for the delayed response, I wanted to take a couple of days to reflect on your email. On 26/09/2012 19:09, Gavin Andresen wrote: And their are other methods too. The GUI::Test package for perl will allow this to be greatly automated. (I have done this before on the localisation of photoshop.) this why we need detailed testscripts and plans. so we know what has and hasnt been done. The more boring the task the more work that needs to go into testcase development. This is the area I see as my greatest failing last time. I have a large number of virtual machines and should have at least this work. But we need very detailed testcases. with decent testplans just downloading the software, syncing the block chain, syncing an existing wallet, rescanning the blockchain and verifying the balance would cover a large number of tests. The idea behind having lots of very specific testcases is you get to see what tests have not been run. I understand your concern, however I have taken a couple of days to reflect on this and I still strongly feel that in order to make sure that this sticks, and is still useful in 1 years time we need to lay proper foundations. Those foundations are not word documents, spreadsheets, etc. they are selecting the right tools for the job. We can gain so much benefit from using 3rd party software. (bettermeans would rock if it wasnt rotting) I am sure you could do your coding work just using vi, but an sdk makes it much easier and allows you to work in a more productive manner. I have had a couple of off list emails with some testers and they also feel that it is very important to make sure we have a sound foundation (mantis is so much more than just a bug reporting tool, I see the bug reporting functionality as secondary to the main test run functionality - but it doesnt have to be mantis based, we do need workflow and testcase software though - and proper software for this is much better than just a massive google doc.) however I am checking out some other software that has been recommended. It will be very hard to change 'the process' once we have something we are used too (just look at the current resistance) I promise nothing will change for the dev team. But test does need other tools, and processes. If you feel that strongly that I am going about this the wrong way, I am happy to step back and let someone else sort it out (I will still do all the testing I possibly can). I would feel that this would be a real shame and we have the chance to setup requirements to functionality to tests all with traceability. why not do it right from the start? I will open up my vps' somepoint over the next few days and you can see what I mean. I will setup a fake git project, and sort out the interactions. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQEcBAEBAgAGBQJQZz1pAAoJEFvEB9dQFvtQRLkIAJtPCkW1R9vmMPY9u4o+ET1t w4pV/+W2PXo2p86HnljCIPLV/cua/1I/EJp7XR7s145Nj4KZUbzHGhvUUmwDOHW2 TGvJs+HO1bjsJfh4pWEb6PXcW3TguZxZSt5/rBAAI/5BeomSuRcZOdoV87D1xnK8 TSlgaseWrJcpKLO30/FQA3QnH/bjJ4OBmtHp8WaOtSnfww9Zbb5VYca37O15c2U4 2d0RUunDg1w2kRbkKjztxr3YasSOX+07Uvj4d5Lw7zgA0U93krNWVT1Ypo94dNJ7 6SyKi30UuqDdJ9XxZrMB/LBVNGOLlIBNWL++ocu5GFnOn9pnw57ZMBZM5g6YDpo= =ekQ/ -----END PGP SIGNATURE-----