From: steve <steve@mistfpga•net>
To: Matt Corallo <bitcoin-list@bluematt•me>
Cc: Bitcoin Development List
<bitcoin-development@lists•sourceforge.net>,
Bill Hees <billhees@gmail•com>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
Date: Wed, 26 Sep 2012 13:28:40 +0100 [thread overview]
Message-ID: <5062F4F8.6040504@mistfpga.net> (raw)
In-Reply-To: <1348605677.2284.2.camel@localhost.localdomain>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Matt,
Glad to have another ninja onboard :)
On 25/09/2012 21:41, Matt Corallo wrote:
> Although Jenkins may not be the best system, we already have
> jenkins and pull-tester (which is a dumb python script I wrote to
> test all incoming pull requests from github).
I have never heard of jenkins before. I need to do some more digging.
is this the right thing?
https://wiki.jenkins-ci.org/display/JENKINS/Mantis+Plugin
Mantis on the other hand, I know exceptionally well. I hate
duplication of work/data unless absolutely necessary. I will check
jenkins out (just out of interest what is it actually meant to do? the
website implies framework, but not what its for)
So, currently there are 4 potential places for bugs to be reported
1 - jenkins (and unit tests)
2 - git
3 - mailing list
4 - forum (bitcointalk...)
5? - is there still the ability to add bugs via sourceforge?
Adding to this doesnt make sense. Each one of these reporting methods
is for a different thing. I am not seeking to replace these (or even
unify them) I am looking for software that will take testcases and bug
reports against them [and allow for test campaigns]. Mantis is so
flexible and industry standard and if the jenkins plugin works... then
we can keep things as they are until they fit into better places.
The reason I am so behind mantis as the backbone is it works with more
or less anything, and can easily modded to work with whatever people
are most comfortable with - however it is exceptionally powerful and
has had a constant stream of workflow improvements over the past few
years.
>
> They both run the same set of scripts, namely those at
> https://github.com/TheBlueMatt/test-scripts (its pretty basic right
> now, but since it is on github, I was hoping someone would find
> the inspiration to add to it).
I will check it out. I wrote a very basic script that wikified the
changelog, and linked to the changes and created wiki pages for the
testcases. have you seen the stuff I put on bettermeans? bits keep
vanishing then re appearing.
This is the outline of the testing that I setup for 0.7
https://secure.bettermeans.com/projects/4256/wiki
>
> I dont really care if we keep using jenkins, but I figure we might
> as well keep all the tests in one place?
Yes, I would love to unify all build testing and testcases into one
place. I am still on the fence as to including unit tests into this.
However I do see 3 distinct type of testcases
1 - requirements based testcases (requirements based off the current
block chain rules - these are edge cases and known interoperability
issues)
2 - Acceptance based testcases - these are testcases that should be
run for every build. Check out the General Acceptance Tests in the
wiki link for examples and testcases
3 - Testcases for reference implementations of things (like multisig -
i see these working like the /test folder when you install a new perl
module)
These three things alone are a massive task. and they still wont cover
everything. I would like to get the workflow so that people can
sponsor or donate to a specific campaign (eg a new feature is
implemented, people want it tested so can donate just for that
campaign [developing testcases, structure, requirements, etc])
Once this is done, I will get to do some exciting stuff (like writing
fuzzers, automation, etc) unfortunately I do not know python, only perl.
>
> Anyway, I'm all for more testing (I'm always complaining about how
> we need more tests for stuff...).
Nice, I love testing. I think we will get on :)
And I would rather go for interoperability between testing rather than
rewriting it all.
Cheers,
steve
>
> Matt
>
> On Tue, 2012-09-25 at 19:32 +0100, steve wrote:
>> Hi All,
>>
>> After the failure to get any real testing done for the 0.7
>> release (all of which is my fault) I have decided to rejig
>> things.
>>
>> I am heavily into test driven development, and I have a strong
>> background in requirements management, and automation.
>>
>> I want to leave bettermeans behind, maybe we might be able to
>> keep the voting aspect of it, and link it into mantis.
>>
>> So, what I have been doing over the past few weeks is developing
>> a rudimentary requirements set, basic requirement tracking, tests
>> to prove/stress the requirements.
>>
>> The next most important thing is to get release/acceptance tests
>> done - these primarily focus on new stuff doesnt break old (ie
>> lose a wallet, etc) and needs no special requirements.
>>
>> To this end I have installed various opensource applications
>> (mantis, salomeTMF, bugzilla, etc) and am currently evaluating
>> the best workflow process.
>>
>> This was a much longer post, but decided against it. :)
>>
>> So, what I want to know is who wants to be a part helping me out
>> with all this? I am finalising the workflow flow diagrams and
>> they should be ready for inspection soon.
>>
>> Anyone interested in helping out/reviewing processes? even if it
>> is just some encouragement, it is all greatly appreciated.
>>
>> Drop me an email if you want access to the current setup and help
>> me review the different software for the bitcoin workflow
>> process.
>>
>> cheers,
>>
>> steve
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iQEcBAEBAgAGBQJQYvT4AAoJEFvEB9dQFvtQlgkIAJX7JYel5RGmCsbptGdQrCnT
BR42tUwTg1t/NRUJ6RA8/Ou8lzallztQquShpLn4mZdQpoalvETdtAwcPnQKnaZb
M5inZE/IEq8WJM1y4YkHt3BLou4BJbjwncCNy1/jqcm6f2Oonrg7isVbDwY/7JlP
y/epm7XELS7NU4vVubBwQCunwvtsuydXRzuI812LiLXNqpXFMHvG2m8a2RajXE0/
xW4lOMy/hUFzEgYRQWCTAru4Ts2x3Xt26NaEUh/uKvHLwBZJ4xbdu3gpupiPb4sI
bCHnVFOC7zoQKOAnfPkCMyvtyoqpzM9HW2+DWI51FoOz851Y2F36N3Fpk/2lii4=
=W5xI
-----END PGP SIGNATURE-----
next prev parent reply other threads:[~2012-09-26 12:29 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-25 18:32 steve
2012-09-25 20:41 ` Matt Corallo
2012-09-26 5:49 ` Wladimir
2012-09-26 11:41 ` Daniel F
2012-09-26 12:00 ` Luke-Jr
2012-09-26 12:28 ` steve [this message]
2012-09-26 12:49 ` Wladimir
2012-09-26 13:22 ` steve
2012-09-26 16:06 ` Mark Friedenbach
2012-09-26 17:10 ` Jeff Garzik
2012-09-26 17:44 ` steve
2012-09-26 18:09 ` Gavin Andresen
2012-09-29 18:26 ` steve
2012-10-01 13:52 ` Arklan Uth Oslin
2012-10-01 14:28 ` steve
2012-10-01 16:52 ` Peter Vessenes
2012-10-03 1:15 ` steve
2012-10-03 2:02 ` Gregory Maxwell
2012-10-03 3:00 ` steve
[not found] ` <CAMGNxUu=LTZyAxKt3pAYSVxyhHBU9pyJPCiFs-tA_weYNNXbtw@mail.gmail.com>
[not found] ` <CAMGNxUuHRBkE_MbmY=A0vQvq=gMfzCFG8Us7SdBn-14KiKMaNg@mail.gmail.com>
2012-10-03 5:04 ` Peter Vessenes
2012-10-03 16:06 ` steve
2012-10-03 16:11 ` Arklan Uth Oslin
2012-10-03 16:15 ` [Bitcoin-development] Fwd: " steve
2012-10-03 17:09 ` Peter Vessenes
2012-10-03 17:30 ` Gavin Andresen
2012-09-27 0:53 ` [Bitcoin-development] " Matt Corallo
2012-09-27 2:29 ` Gregory Maxwell
2012-09-25 20:49 ` Daniel F
2012-09-25 21:25 ` Gary Rowe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5062F4F8.6040504@mistfpga.net \
--to=steve@mistfpga$(echo .)net \
--cc=billhees@gmail$(echo .)com \
--cc=bitcoin-development@lists$(echo .)sourceforge.net \
--cc=bitcoin-list@bluematt$(echo .)me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox