Skip to main content

Do we need step by step test cases?


Many companies have moved to agile software development but they still maintain large suites of step by step test cases for regression testing. At the company I work for, I’ve managed to get rid of these step by step test cases altogether.

What is wrong with step by step test cases anyway?


If you are in a traditional waterfall setup with high level requirements and test cases, then the test cases are actually quite valuable as they document how the system works. But if you start to move quality to the left and specify requirements in more detail upfront with user stories, then test cases make much less sense. You are essentially duplicating the requirements in test cases and end up with a whole set of assets that need to be maintained in a separate test management or ALM tool. A big problem with that is that you do not have a single source of truth on how the system behaves.

A second problem with step by step test cases is that they are not visible to the business. I can’t think of many product owners or even business analysts who want to trawl through detailed test cases to understand how a product works. So we get to a situation where the business has a high level idea of how the product works through requirements but only the testers know how the product really works.

Another big problem from a tester’s perspective is that regression testing is boring! Who wants to go through step by step test cases over and over again? Surely testers are more intelligent than that. Once a tester understands a product they do not need step by step test cases to test it effectively. I prefer the idea of an intelligent tester who has a good knowledge of the product and is finding issues, rather than blindly following a step by step test case.

To summarise, the problems with step by step test cases are:
  • They duplicate requirements which means you have two sets of assets to maintain and  there is no single source of truth about how the product works
  • Step by step test cases are not visible to the business
  • It encourages the notion that anyone can test by blindly following a step by step test case

How can you get rid of step by step test cases?


If you would like to find out how I got rid of test cases, why not come to the talk I’m giving on this subject at the CASTx18 conference in Melbourne, Australia. CASTx18 is on 1 March 2018 and you can register here:

In the talk I describe the steps I took to change processes and convince management there is a better way to do things. The approach I pioneered hinges on creating user stories to define features and, in particular, writing acceptance criteria  that effectively become the test cases. This approach means that we have one source of truth for how a feature works and what has been tested. I’ll also describe how other types of tests, such as shakedowns, were replaced with mind maps.

If you live overseas or are not able to attend the conference, I’ll try and get a video of the talk and/or write a follow up post that expands on the details of my approach.

Comments

Popular posts from this blog

How to install a test app onto your mobile device

In this post I'll describe how to actually get test versions of the app onto a device. But first let's discuss whether you should test on an actual device or on an emulator. An emulator is a desktop application that mimics the hardware and OS of a mobile device. The developers will generally do their app development on an emulator and you can use them for early stage testing but when it comes to meaningful end to end testing, a device is a must have. There is no other way to get a feel for the performance of the app and how users will use it real life.
Of course you will probably need to test on multiple devices as they vary not only by OS (iOS and Android) but also OS version, device make (e.g. Samsung or Motorola), and screen size. We also now have a new OS for iPads called iPadOS. In a future blog post I'll look at device fragmentation and how to handle it.
For now let's assume you are only testing on one iOS device and one Android device. Your developers have de…

Testing Mobile Apps versus Websites

You all no doubt own a mobile phone, most likely an iOS or Android device, so you probably think it wouldn't be too difficult to test mobile apps. While the principles of testing remain the same, there are significant differences between how you test websites and mobile apps. For example: You can't just open a browser on your laptop to test the latest version of the application. Somehow you need to get the latest version of the app onto your test device.iOS and Android have completely different design patterns.It's more difficult to get under the hood of an app to see what's going on. You can't just open Chrome DevTools.
This is the first in a series of posts where I'll give you some background on how to test mobile apps and include a number of tips and tricks that I've learnt the hard way in my 10 years of mobile testing experience. I'll focus on iOS and Android since those are the most common apps but I also test Windows apps.…

How to access files and databases on your mobile device

In this post I'll describe how to access files and databases associated with a mobile app. On the apps that I test we have log files and SQLite databases that are useful to look at when investigating potential bugs. How do you access files and databases on a mobile device? As usual that depends on whether you are testing iOS or Android. App specific files are available on both devices but they way you access them is different for each OS.
iOSFor Apple devices, you can use iTunes to access app specific files. Open iTunes on your PC or Mac and connect your device. You will see the device:



Select the device icon. This will open the apps page. Under File Sharing you can then select your app to view its associated files.


You can then add files, save files to your PC or delete files on the device.
Android On Android devices there is usually a folder called "MyFiles". App specific files are stored under that folder. To access them, simply connect your device to your PC or Mac and…