Testing - Pt 2- Choosing Your System
Chose a system that has a low barrier to entry, something people are keen to learn, or will already know:
- One where there's value in learning, such as a popular industry standard, and those systems will be better documented, better understood, more reliable, and your colleagues will be easier to get on board.
- Use the system "in-paradigm", by which I mean, use it as was meant to be used, not in an unusual "out-of-paradigm" way, this will make your colleagues life difficult, and prevent adoption.
Can you can test multiple configurations, where some tests are only applicable to some modules and configurations?
Is it robust?
- Will changes to test subjects lead easily to identifying the tests that need changing? A change to your underlying implementation shouldn't silently break the tests.
- Avoid completely dynamic languages, compile time checking prevents typographical errors and identifies tests that might need changing if the test subject changes.
Consider if the system is usable by both developers, and to less technical people - will you want testers or QA to be able to write tests?
Once upon a time I thought this was a no brainer: is the test system fully automated? Or, is it going to cost your company money each time you run them?