Hi Steve, This looks like a good idea to me. The test suites could act similarly to the 100% Pure Java approach that successfully fended off a lot of corrupting influences to Java over the years. Maybe it's worth putting together a small starter suite of tests and showing them to the community then providing a suitable process, perhaps through BIPs, to allow tests to be created, reviewed and updated before getting incorporated into a reference. I imagine a BIP would cover an aspect of the blockchain rather than a single test or test suite since having that many BIPs would get onerous fast. Kind regards, Gary On 1 August 2012 12:38, steve wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi all, > > I know most of you have more important things to do, and not enough > time as it is, but I would really like your feedback on the testing > project. > > Bettermeans seemed ideal, it allowed for leaderless leadership with > people being able to dip in and out of what they wanted to do. > however it seems that bettermeans is in no way a finished product and > is rotting fast. > > I would like to move away from bettermeans - whilst retaining the > voting and sub workstream style of working. I would like to set > something up around MantisBT (Mantis Bug Tracker) - this is an > exceptionally versatile bit of software with plugins and interaction > with other testing products (everything from test setups and plans to > test runs and results) I would go as far as saying it is industry > standard (yeah, i know about bugzilla, etc. mantis beats them all > hands down.) obviously we would need a wiki and other software - this > is not a problem. > > I am happpy to pay for the VPS' to host this stuff and set it all up. > > I have quite a bit of experience with mantis and other opensource > testing stuff. > > I see the testing of bitcoin to be very similar to the testing process > in the games industry. for those that do not know how this works it > is like this:- > > A company comes up with a device (xbox/ps), > They then publish a set of dos and donts for this device (TRC for sony > and TCRs for microsoft. - I wrote quite a few of the MS TCR's for the > original xbox.) > They (ms/sony) then test your game against these rules and depending > on how many you pass/fail your game can be released or not. > > I see this as mapping to bitcoin very well, the device is the > blockchain, and the TRC/TCR _tests_ are published so third party > developers can see how they fair [look at gavins recent blockchain > edge cases for an example] (ms/sony do not publish their testcases > only the requirements) > > I believe that this will allow the bitcoin testing project to be able > to cope with the stable builds, bleeding edge builds and 3rd party > implementations all at the same time. > > It doesnt matter what the app is, it is its interaction with the > blockchain, the safe guarding of the blockchain and compatibility with > the previous/future versions that are tested. > > [for the bitcoin dev list] > A little about me:- > The below is more or less a cut and paste of some of the stuff I sent > gavin in my initial email about wanting to be in on the project. > > I can back all of this up with references. I can go into more detail if > needed. > > I was heavily involved with setting up the microsoft xbox european cert > department > I set up qa department for europes largest independant games developer > (although they are no longer) > > worked for microsoft secure science designing security automation tools > > setup the internal pentest for thales e-security (now TITS [Thales > Information Technology Solutions (or Trotters Independant Traders ;), > on thier datacryptors (fpga, crypto and product) - have done full test > cycles on hsm 8000, payshield 9000, dc2k and thier latest line of > military spec comms equipment. > > Setup and pentested Thales and nCiphers credit control software (to > fips level 4 standard - This shit bitcoin exchanges need!! I know bank > is a dirty word in the bitcoin world, but we should be at least as > secure as them) > > I currently find exploits in stuff like office, quicktime, ie, ff, etc > and sell them to companies like ZDI (3com) iDefense (verisgn) and some > pentest companies that require zero day exploits. > > however I would like to ditch this and get my bitcoin related stuff > off the ground - with the BTP being top of the list. > > my spelling is rubbish, and sometimes i forget to spellcheck before i > hit send. sorry about that. > > any feedback would be really appreciated. please! I feel that this has > stagnated enough and I want to get my work out there and I want it to > be useful. (I lost 60 or so testcases because Kev left himself logged > into bettermeans and I added them under his account... none appeared > on the wiki, none appeared anywhere...) > > There is already lots of stuff on the wiki that outlines how i see > things holding together - but you cant see who posted what because of > a bug that has appeared recently [meta info is just displayed as its > meta tag, {name} on {date} wrote.] > > as apposed to Mistfpga on 30/8/2012 wrote... > > so the tl;dr > 1 - I want to use something other than bettermeans > 2 - I can admin opensource software to do the same > 3 - I want to take the voting/hiearachy style from bettermeans and > apply it to mantis and test workflow. > 4 - I want to get some testing done asap. > 5 - I have a full msdn and technet licence > 6 - I have a vast array of machines [nearly 100 cores] that I can use > to automate testing and to test different setups. > 7 - been mining for 18 months or so. > > cheers, > > steve > > I do not belive this to be related, but I am not ashamed and feel no > stigma. I have posted this on a public forum. I suffer from fast > cycling (withing a day) type two bipolar disorder. This has been and > is being managed very well by my meds (300mg quetiapine fumerate and > 30mg mirtrazipine at night) I have been well for over a year now. [and > yes, even with these mental health problems I still hold SCE clearance] > > - -- > my PGP public key is at pgp.mit.edu id: 0x5016FB50 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.17 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJQGRU7AAoJEFvEB9dQFvtQY9oH/RiNBeWDs+tz0NQgkgtXmxcs > tDtlLWB91gRtmxvXv+yYHE0WXS/qQ5gwaor8Vek61Cvnhij4AQBf7JVYLj03/7fy > cMAFJ5k7ZQmH9URo8Jx8xzQ2tLgZ5BqLbP6fTzDstpExNGJlkZvsC6hTx9OFCubU > bzWuWmcIRzBiMhdz7DPoskOgie5sQRzzO0zrK7NvS1P2hyEhM3eFYFcYecpS8jq9 > SbkH77qnc5pZD2JULq38htVwtI16vxmg/s4GIs3OpEPaMk5xtVc5XHf6gwBZQtYw > fErwZpNDEAmGidRCB4LAqdSx1Xi0ix7GOyvTxBLWTnfB7wIAQNQpBHB9zN8diiA= > =am8B > -----END PGP SIGNATURE----- > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >