"But tools and processes on their own are stupid things." - Jonathan Kohl

For Assignment 4 we are to compare and contrast these two articles.

http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf
and
http://www.useit.com/papers/heuristic/heuristic_evaluation.html

I have a hard time comparing the two articles because the exploratory testing article is about breaking away from automated tests and allowing tests to 'go with their instincts'. The Heuristic evaluation article is about using testers to determine interface flaws according to standard heuristics.

I really enjoyed the exploratory testing article, even though I find the analogy of musical improvisation being similar to exploratory testing a bit of a stretch...I love the quote that I used for the title of this post, you can have the best automated scripts and tools and processes, but you need talented testers to be able to properly apply those tools and processes to get real results. While the standard automated tests are a good start, it does make a lot of sense to free your testers to investigate and play around with anything they can come up with, after all, the human mind is very powerful as Jonathan mentions. The test scripts will only find errors they have been programmed to find...

The Heuristic evaluation article was interesting, but not quite as much fun to read I have to say...I do like the charts showing the effectiveness of a single tester to multiple testers, it really shows how different people approach things differently. Jakob Nielsen does a great study on how many errors multiple testers find and shows how many errors that were found overlap. He recommends 3-5 testers for medium size projects to find the most errors with the greatest cost efficiency. It makes sense to me to have people go off the list of usability heuristics to analyze a program, that way everyone has the same foundation, but each person may interpret the interface differently. What is easy and seems logical to one person may not be the same to another.

Exploratory testing is very much different from Heuristic evaluation, but both testing methods will provide great results for projects. It is interesting to think of someone making a career out of software testing, but it is easy to see how valuable a talented tester would be to a project.