[Bitcoin-development] Fwd: Re: Bitcoin Testing Project

[steve <steve@mistfpga.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I think he had a typo in the CC.  here is a forward of the email.
You will have to work out the indentations yourselves :)

- -------- Original Message --------
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
Date: Tue, 2 Oct 2012 22:01:19 -0700
From: Peter Vessenes <peter@coinlab.com>
To: steve <steve@mistfpga.net>

On Tue, Oct 2, 2012 at 6:15 PM, steve <steve@mistfpga.net> wrote:

> On 01/10/2012 17:52, Peter Vessenes wrote:
>> I'm a big proponent of a testing project.
> 
> I am very happy to hear this, however, your actual words are
> slightly evasive. I do not expect you to be up to speed on this.
> Gavin started a project called 'the bitcoin testing project' This
> project solicited donations, about 80 coins last time I checked.
> However these 80 odd coins were donated to 'the bitcoin testing
> project' This would seem to be an official bitcoin (both protocol
> and client) testing project.  I signed up to work on this, and
> organise as much as i could of this. for various reasons I did not
> manage to do the testing I wanted to on 0.7 i over committed
> myself.
> 
> Are the donations solicited for the 'bitcoin testing project)
> funds going to be given to 'the foundation'?
> 
> 
> Not as far as I know; sounds like they should go toward testing.
> 
> 
> Does the foundation support 'the bitcoin testing project'? does
> the foundation have any involvement with 'the bitcoin testing
> project'?
> 
> I personally support the idea of a testing project. I would like
> the Foundation to fund it if it can't crowdsource funding from the
> forums; sounds like so far the support hasn't been enough to get
> all the work done. The Foundation has no formal role with the
> bitcoin testing project that I'm aware of.
> 
> 
> 
> 
>> I think if one could self organize that Gavin and team wanted to 
>> bless we could put up some BTC as bounties or funding. We won't 
>> have our heads around the foundation budget for a few more
>> weeks, but self-organization is often slower than budgeting. :)
> 
> Im ready to go, more or less.  Please check out the links in my 
> previous emails. I have over 400 testcases (8 platforms * 50
> release tests) - Also I am not sure what you mean by bless, I take
> it that is a euphemism for pay?
> 
> Wow, that's awesome! I use bless to mean "Gavin saying that it
> sounds good."
> 
> 
> I have tried my hardest to get bettermeans to work, but it doesnt.
> It does show quite a lot of work that I have done though. If you
> were to say to me, 'steve, by monday we need end to end,
> requirements based testing' It would be done. (I have already spent
> over 4 months on this)  Leaderless leadership is something I am
> having a hard time with, bettermeans is excellent at this.  But I
> have found very little in regards to voting and polling that
> integrates with the project in an effortless way like bettermeans.
> 
> I understand that the budget from the foundation is something that 
> needs to be worked on and organised.  I offer my services in this
> area (qa only).  I would be happy to submit my cv and refs for
> this, if required.
> 
> I am now feeling frustrated and useless.  has my last 4 months of
> work been for nothing? it feels like it.  I know I bang on about
> processes but they are sorted, you can only attract talent like
> Arklan if he has a process to follow. i feel like a broken record.
> 
> I'm a little late to this conversation, so I don't know what to say
> in response. I will answer your questions below, though.
> 
> 
> tl;dr version 1 - Will donations to the 'bitcoin testing project'
> as started by gavin going to be given to the foundation?
> 
> 
> I don't expect so, although we'd take them if whoever is in charge
> of the testing project wants to do so. I'd expect that if the
> testing project is good and community approved and supported by the
> dev team the funding flow would go the other way, but we'll need to
> wait for budgets to get finished.
> 
> 
> 2 - Is the work bill hees and myself going to be binned?
> 
> 
> I have no idea whatsoever, I would guess that's up to you and bill
> hees and the dev team.
> 
> 
> 3 - I feel like I have the knowledge and drive to push this, but I 
> cant do it on my own.
> 
> 
> Totally understand the feeling!
> 
> 
> 4 - Is bill or I entitled to any of the cash raised for 'the
> bitcoin testing project'
> 
> 
> I have no idea what the bitcoin testing project finance situation
> is.
> 
> 
> 5 - Do I have to join the foundation to have a say in how the
> project (testing) is done?
> 
> 
> Nope
> 
> 
> 6 - sorry for being so mercenary, but am I going to receive any
> coin for work I have done?
> 
> 
> Ditto to above
> 
> 
> 7 - It really probably is the time for a bitcoin-test list to
> appear. Is there anything I can do to make this happen?
> 
> I don't have much to do with this.
> 
> 
>> This is just my opinion, but I would like very, very much to
>> move the current specification into unit tests so that anyone
>> could validate their alternate bitcoin implementation. This is a
>> lot of work, some of which has been done, much of which hasn't.
> 
> have a look at the stuff in bettermeans.  I personally think we can
> go a step further and publish guidelines (similar to RFC's and all
> the tests that we would do against a ref client)
> 
> 
> Cool, I will check it out. I agree with the RFC oriented approach,
> or perhaps a canonical test suite for validation and verification.
> 
> 
> But I dont want to waste any more time on stuff that is going to
> be ignored, life is short.
> 
> 
>> So, my two cents, plus an offer to bring it up at our next 
>> budgeting meeting.
> 
> I accept that offer. and I really appreciate it.  I have some more 
> questions I would like you to ask in regards to QA. (Gavin and I 
> skyped about this a while ago and we didnt really come to a 
> resolution, weworked out the problems though ;) )
> 
> I have an exceptionally detailed qa process (based off the game 
> certification process) - but I have gone on about this at length
> in previous messages.
> 
> I thank you for your email and your involvement with this, but do
> you think we are closer to getting stuff tested? call my bluff...
> Not one person has asked for login details to my proposals - and i
> even have a bugzilla version now.
> 
> I need to sleep.  sorry if i rambled.
> 
> nite nite,
> 
> steve
> 
> 
> Thanks for all the details. I'm trying to go into extra detail in
> my response to you because people are still figuring out the
> Foundation's situation with all things Bitcoin. I can't offer you
> any promises for your future, but I'm personally really psyched to
> hear about these tests. I'll let other more knowledgeable folks
> jump in on the technology side of things.
> 
> 
> 
>> Peter
> 
> 
>> On Mon, Oct 1, 2012 at 7:28 AM, steve <steve@mistfpga.net>
>> wrote:
> 
>> On 01/10/2012 14:52, Arklan Uth Oslin wrote:
>>>>> Hi guys.
>>>>> 
>>>>> So, as I mentioned on the bitcointalk.org forums thread
>>>>> about the foundation, I want to get involved in the QA side
>>>>> of bitcoin development. I've done functional testing in
>>>>> the video game industry for years.
> 
>> Nice one, I worked in games for quite a few years. (before
>> getting into finance then pentesting) there are about 6 keen
>> testers now. maybe we should get a bitcoin-test mailing list,
>> where we can discuss stuff without disturbing the dev team.
> 
>>>>> I've read all the messages in this thread, but I'm left 
>>>>> unclear how I can most effectively and quickly being
>>>>> helping out. Could I get a bit of a directional nudge?
> 
>> Great question... for me I feel structure is the most important 
>> thing to sort out first.  However we desperately need detailed 
>> testcases for the release of a new version. - Not too much on
>> the change log stuff, more on the noddy stuff (as gavin points
>> out below), downloading and making sure it works on a non dev
>> machine, make sure the wallet isnt overwritten, etc.) doing games
>> qa I imagine this would be an ideal place for you to start.  I
>> have a MSDN and TechNet licence so if you need some reference ms
>> virtual machines I can help you out.
> 
>> However we need some testcase software.  Please check out what
>> was done on bettermeans for the stuff I was planning out...
> 
>> It details everything from recompense and testcases. bettermeans 
>> kinda died a death though...
> 
>> check out:
> 
>> Bitcoin over all- 
>> https://secure.bettermeans.com/projects/4180/wiki/Page_index 
>> discussion https://secure.bettermeans.com/projects/4180/boards
> 
>> 0.7 https://secure.bettermeans.com/projects/4256/boards and 
>> https://secure.bettermeans.com/projects/4256/wiki
> 
>> I still have the testcases, but until we get some proper
>> testcase software I am loathed to publish them in a half arsed
>> format. (they worked well on bettermeans, then just vanished one
>> day...) what testcase software are you familiar with?
> 
>> apart from that, what do you feel you can do for the project?
>> how long have you been involved in bitcoin?  It may well be
>> worth reading up all the dev stuff on the wiki so you can get you
>> head around how the bitcoin protocol is different from the daemon
>> and qt client. What do you think you can and will enjoy doing?
>> What is your skill set in regard to networking, crypto and
>> operating systems. (not that you need any, in any we still want
>> and need you. :)
> 
>> there really is room for you to do whatever role you want, and
>> as little or as much as you want - however funding is now a very 
>> tricky issue. so much so that I am not sure I want anything to
>> do with it(distribution of coin based on work.). - I just paid
>> for some logo spec work out of my own pocket (for example).  I
>> have some testers i know irl who are willing to work for coin.
> 
>> NOTE: This response has nothing to do with the bitcoin
>> foundation. I am not a member of the foundation. I do not speak
>> for them or even probably with them. I am still trying to work
>> out how much qa the foundation should be responsible for, and/how
>> it is supposed to work. I think the games cert process would be
>> ideal for this.  This however this a discussion that probably
>> wont have my involvement. (personally I believe that the
>> foundation should publish requirements with example code and
>> testcases for each aspect of the reference client. (on reference
>> platforms - I do not expect many to agree with this though)
> 
>> As a side note, what happens to the donations to the bitcoin 
>> testing project? do they get moved over to the foundation? this 
>> question is bigger than this email. as far as I know they are
>> all on an address Gavin holds. Actually I would like to be
>> involved in any discussions that would impact QA, does this mean
>> I need to join the foundation or just go lone wolf?
> 
>> tbh I dont really understand foundations.  I always thought they 
>> were just a tax dodge.
> 
>> Sorry for the long message. :)
> 
>>>>> 
>>>>> Arklan
>>>>> 
>>>>> ---------- As long as there is light, the darkness holds
>>>>> no fear. And yet, even in the deepest black, there is life.
>>>>> - Arklan Uth Oslin
>>>>> 
>>>>> I want to leave this world the same way I came into it: 
>>>>> backwards and on fire. - Arklan Uth Oslin
>>>>> 
>>>>> 
>>>>> 
>>>>> On Sat, Sep 29, 2012 at 12:26 PM, steve
>>>>> <steve@mistfpga.net> wrote:
>>>>> 
>>>>> Hi Gavin,
>>>>> 
>>>>> Sorry for the delayed response, I wanted to take a couple
>>>>> of days to reflect on your email.
>>>>> 
>>>>> On 26/09/2012 19:09, Gavin Andresen wrote:
>>>>> 
>>>>> 
>>>>> And their are other methods too.
>>>>> 
>>>>> 
>>>>> 
>>>>> The GUI::Test package for perl will allow this to be
>>>>> greatly automated. (I have done this before on the
>>>>> localisation of photoshop.)
>>>>> 
>>>>> 
>>>>> 
>>>>> this why we need detailed testscripts and plans.  so we
>>>>> know what has and hasnt been done. The more boring the task
>>>>> the more work that needs to go into testcase development.
>>>>> This is the area I see as my greatest failing last time.  I
>>>>> have a large number of virtual machines and should have at
>>>>> least this work.  But we need very detailed testcases.
>>>>> with decent testplans just downloading the software,
>>>>> syncing the block chain, syncing an existing wallet,
>>>>> rescanning the blockchain and verifying the balance would
>>>>> cover a large number of tests.  The idea behind having lots
>>>>> of very specific testcases is you get to see what tests
>>>>> have not been run.
>>>>> 
>>>>> 
>>>>> 
>>>>> I understand your concern, however I have taken a couple
>>>>> of days to reflect on this and I still strongly feel that
>>>>> in order to make sure that this sticks, and is still useful
>>>>> in 1 years time we need to lay proper foundations. Those 
>>>>> foundations are not word documents, spreadsheets, etc.
>>>>> they are selecting the right tools for the job.
>>>>> 
>>>>> We can gain so much benefit from using 3rd party software. 
>>>>> (bettermeans would rock if it wasnt rotting)
>>>>> 
>>>>> I am sure you could do your coding work just using vi, but
>>>>> an sdk makes it much easier and allows you to work in a
>>>>> more productive manner.
>>>>> 
>>>>> I have had a couple of off list emails with some testers
>>>>> and they also feel that it is very important to make sure
>>>>> we have a sound foundation (mantis is so much more than
>>>>> just a bug reporting tool, I see the bug reporting
>>>>> functionality as secondary to the main test run
>>>>> functionality - but it doesnt have to be mantis based, we
>>>>> do need workflow and testcase software though - and proper
>>>>> software for this is much better than just a massive google
>>>>> doc.) however I am checking out some other software that
>>>>> has been recommended.  It will be very hard to change 'the
>>>>> process' once we have something we are used too (just look
>>>>> at the current resistance) I promise nothing will change
>>>>> for the dev team.  But test does need other tools, and
>>>>> processes.
>>>>> 
>>>>> If you feel that strongly that I am going about this the 
>>>>> wrong way, I am happy to step back and let someone else
>>>>> sort it out (I will still do all the testing I possibly
>>>>> can). I would feel that this would be a real shame and we
>>>>> have the chance to setup requirements to functionality to
>>>>> tests all with traceability. why not do it right from the
>>>>> start?
>>>>> 
>>>>> I will open up my vps' somepoint over the next few days
>>>>> and you can see what I mean. I will setup a fake git
>>>>> project, and sort out the interactions.
>>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
> 
> ------------------------------------------------------------------------------
>>>>>>
>>>>>>
>
> 
> 
> How fast is your code?
>>>>>> 3 out of 4 devs don\\\'t know how their code performs in 
>>>>>> production. Find out how slow your code is with 
>>>>>> AppDynamics Lite. 
>>>>>> http://ad.doubleclick.net/clk;262219672;13503038;z? 
>>>>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>>>>>
>>>>>>
>
>>>>>> 
_______________________________________________
>>>>>> Bitcoin-development mailing list 
>>>>>> Bitcoin-development@lists.sourceforge.net 
>>>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>
>>>>>> 
>>>>>> 
> -
> 
> ------------------------------------------------------------------------------
>>>>>
>>>>>
>>
> 
Got visibility?
>>>>> Most devs has no idea what their production app looks
>>>>> like. Find out how fast your code is with AppDynamics
>>>>> Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? 
>>>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 
_______________________________________________
>>>>> Bitcoin-development mailing list 
>>>>> Bitcoin-development@lists.sourceforge.net 
>>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>>
>
>>>>> 
>>>>> 
>>> 
>>> 
>>> 
> ------------------------------------------------------------------------------
>>>
>>>
>
> 
Got visibility?
>>> Most devs has no idea what their production app looks like.
>>> Find out how fast your code is with AppDynamics Lite. 
>>> http://ad.doubleclick.net/clk;262219671;13503038;y? 
>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html 
>>> _______________________________________________ 
>>> Bitcoin-development mailing list 
>>> Bitcoin-development@lists.sourceforge.net 
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>
>>> 
> 
> 
> 
> 

- -- 
- ------------------------------

[image: CoinLab Logo]PETER VESSENES
CEO

*peter@coinlab.com * /  206.486.6856  / SKYPE: vessenes
811 FIRST AVENUE  /  SUITE 480  /  SEATTLE, WA 98104



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQEcBAEBAgAGBQJQbGSIAAoJEFvEB9dQFvtQw1YH+wW3QDMHHV7kf7ZQXvAswv5T
S2htfdgp1DBYBatoJB9fUU3hgHTEz1UhENH509SjS1/dxHuIxguJkbgmuSdoeB/4
rDuuYzQVcmym/G1YXvmAkCDk5/fUY4RKlCDGLg9MVMpVFhLIr6Jiwxd0NiIqWm9K
aE2bXijiTIRp1sNFxJh5SZTh9/rz1zF8TU3LBWyCegISwyBH2XeIIbVgORoMDBlm
n8yKGwPSg5PaH/eWhXR17OUMPbX6xGlMqTmBo7BZDCwPpZU470rkDsPywsn55mAf
W6EQhFjxpSCnmZwRhrkdM/Le/0ySSUvM4spPu7MlQ6ekGP2gBzzUPbKnTGHzNgQ=
=r57G
-----END PGP SIGNATURE-----