Here at Freeport Metrics our team of QA testers has gathered many tricks and techniques to ensure excellent software product quality. My go-to list includes a number of exploratory testing ideas for uncovering those pesky bugs as soon as possible.
What is Exploratory Testing?
Exploratory testing is a term coined by Cem Kaner back in 1984; my favorite definition is that of a well-known tester James Bach: “Exploratory testing is simultaneous learning, test design, and test execution.” Exploring is all about discovery, creativity and thinking. Rather than following pre-written scripts, testers are free to play with the software, follow various paths, and change plans as they go. As there are no expected results written in stone, the tester takes the responsibility of deciding which outcome is desired and which one is not. With the growing popularity of automation, exploratory testing is sadly often overlooked. But there is really no need to choose between the two! As we have seen time and again at Freeport Metrics, you can gain a lot of value and drastically improve quality by combining both automated and manual exploratory testing.
When to Use Exploratory Testing?
Experts in the field will point out that exploratory testing is a perfect idea for these situations:
- You don’t know much about the product
- You have a general idea of the product but want to learn it deeply
- There is no documentation
- You need to quickly find the most critical bugs
- You want to audit another tester’s work
I would like to add several more uses of exploratory testing that have served us well in many projects:
1. First Impressions
Testers can provide first “user” feedback about design, ease of use and overall experience/product satisfaction. That button which is far too small to tap on a phone? No possibility to share your results with friends? Menu too cluttered to find what you need? Automation will not tell you this, but exploration will.
2. Getting out of a Rut
When you have gone over all of your scripted tests, you have run your automated ones, and you are running out of ideas what else can be tested… you may be tempted to think there are no bugs left! That’s where exploratory tours help you with some fresh ideas and more bugs to be fixed!
3. Creating Test Scripts for an Already Existing Product
Exploring is perfect when you need to create a test plan and a set of test cases for a product already on the market. Starting such a task without thorough exploration first will often lead to incorrect functionality descriptions and further need to partially or completely rewrite your test cases.
Exploration Techniques and Ideas
There are two books I can’t recommend highly enough – “Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design” by James. A. Whittaker, and “Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing” by Elisabeth Hendrickson. While there is a huge choice of books on software testing in general, these two are written specifically about exploratory testing and provide deep insights into the wonderful world of creative and efficient manual testing.
Here are my favorite techniques from both of these books that have worked especially well in our projects.
I first heard about the idea of testing tours from James A. Whittaker’s articles, and found it brilliant in its simplicity. Instead of randomly clicking around, the tester takes on the role of a tourist, and the product to test is the city he visits. Unless you are a very rich tourist with tons of free time on your hands, you can’t really afford to see all the sights of the city, so you need to have a plan. Basically, this is what tours are about – they are exploratory testing with a certain goal/theme in mind. So you may have a guidebook tour (test the features highlighted in the user manual), a historical tour (check legacy functionality and old features) or money tour (make sure all the parts that attract customers to the software work fine). The great thing with tours is that you can always think of your own “routes” across the product that are unique to the software you’re working on. They help you focus and give you ideas of what to look for.
Nouns & Verbs
This is a great technique, especially if you’ve run out of testing ideas. You just compile two lists of words – one with verbs pertaining to what the user can do in your application (ie type, edit, delete, hover over etc) and the other one with nouns (ie button, image, option, field etc). Then you combine the words from list 1 with words from list 2, randomly (throwing a dice works well), and either test what comes up or try to invent possible ways of how the word combination could be interpreted and tested. For example, a mix of “edit” and “text” is fairly easy to do but if you get, say, “move” + “version” you would need to get creative (and that’s where interesting bugs often show up). Personas If you ever dreamt of being in a Hollywood movie, this one’s for you – you need to play the role of your typical (or not-so-typical) user. You can test your program pretending to be someone extremely intelligent – or vice versa; you can imagine what an annoyed businessman or a bored teenager would do or how your application would be handled by an inexperienced user. Behaving in a way that you do not normally behave as a tester will inevitably lead you to new ways of interacting with your application. Boundaries This is a testing classic that never grows olds. Using too much/too little/just right; too long or too short; fitting within constraints or breaking them always does wonders for any field validation but may also come in handy in other cases, such as uploading a file over the allowed size limit or tapping on a button not once but five times in a row. Follow the Data One more useful technique for working with data-hungry applications is to create a piece of data and then watch it as it goes through its lifecycle. If you have an application where users must register, you could see how Register -> Edit User Information -> View User Information -> Delete works. Is all information presented correctly? Can user A see what user B is up to? What happens if someone tries to edit something they should not be able to? There are plenty of questions to ask, and ‘Follow the Data’ scenario gives you all the answers you need.
Naturally, these are only a few of the (very) many ideas you can use while doing exploratory testing. As Cem Kaner says himself, exploratory testing is first and foremost “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester”. So let’s arm ourselves with all kinds of tools but remember to be creative and treat each case individually.