Skip to main content

How can we improve product quality?

Recently I joined a small company as their test lead and we have been looking at ways to improve the quality of our products. It got me thinking about all the factors that influence quality. This blog will be about my observations and lessons learnt as we go about creating high quality products for our customers.
Improving the quality of a product relies on so much more than just testing. It involves a diverse range of factors including the process we use to develop the product, how the software is designed and coded, and of course how it is tested. It is not possible to test quality into a product. Instead, quality must be considered right from the start.
So what are some of the factors that influence quality?
  • It all starts with requirements and how they are written. Do they make crystal  clear to both the developers and the testers what is being created? We have found it very valuable to document high level customer requirements as user stories with acceptance criteria. The acceptance criteria then make clear what is to be developed and testers test against them. There is no need for a separate set of test cases; i.e. we have only source of truth. The acceptance criteria are written in the Given, When, Then syntax:
Given provides context for the behaviour
When specifies the set of actions that triggers the behaviour, such as user or subsystem actions
Then specifies the expected result of the behaviour
  • It's also very useful to have a customer available to answer questions such as: "what exactly did you mean by that requirement?", or "is this issue we keep coming across a known issue in the third party system?". Otherwise you are often guessing which leads to miscommunications and building the wrong product.
  • The next big factor is the product development process. Adopting an agile process can make a big difference to product quality, provided it's done well. If the current process is waterfall then adopting Kanban can be an easier transition to make, rather than going directly to a full blown agile approach such as Scrum or Extreme Programming.
  • Another important factor is coding practices. Are developers aware of ways to write software that promotes quality, and are they given enough time to produce good quality software? Are code reviews part of the process?
  • Testing the product is only one part of the quality equation but it's obviously an important part. While the acceptance criteria cover expected behaviour, we still need to do exploratory testing to examine areas of high risk and see how the system operates as an integrated whole. Another important quality measure is running automated checks as part of a continuous integration pipeline. I'm in favour of developers writing unit and integration tests as part of their development. Test Driven Development (TDD) is an excellent practice that drives the design of the software by thinking about how to test it first.
As you can see, a large number of factors contribute to creating a quality product and all must be in place before you will see major improvements.  Your aim should be to build quality into the product from the start and not rely on testing at the end to find bugs. Ideally, we would get to a point where most defects have been prevented or detected by the time we reach a formal testing phase.

In my next blog posts I'll delve into these factors in more detail.

Comments

Post a Comment

Popular posts from this blog

Let’s stop writing automated end to end tests through the GUI

What’s the problem? I have not been a fan of Selenium WebDriver since I wrote a set of automated end-to-end tests for a product that had an admittedly complicated user interface. It was quite difficult to write meaningful end-to-end tests and the suite we ended up with was non-deterministic i.e. it failed randomly. Selenium Webdriver may be useful in very simple eCommerce type websites but for most real world products it’s just not up to scratch. This is because it’s prone to race conditions in which Selenium believes that the UI has updated when, in fact, it has not. If this happens, the automated check will fail randomly. While there are techniques for reducing these race conditions, in my experience it is difficult to eradicate them completely. This means that automated checks written with Selenium are inherently flaky or non-deterministic. Maintenance of these automated checks becomes a full time job as it is very time consuming to determine whether a failing check is actuall...
6 comments
Read more

How I got rid of step by step test cases

In my last blog post I told you what I think is wrong with step by step test cases. In this blog post I’ll tell you how I got rid of step by step test cases at the company I work for. When I joined Yambay about 18 months ago, the company was following a fairly traditional waterfall style development approach. They had an offshore test team who wrote step by step test cases in an ALM tool called Test Track. Over the past 18 months we have moved to an agile way of developing our products and have gradually got rid of step by step test cases. User Stories and how I use them to test Getting rid of step by step test cases didn’t happen overnight. Initially we replaced regression test cases and test cases for new features with user stories that have acceptance criteria. The key to using a user story to cover both requirements and testing is to make sure that the acceptance criteria cover all test scenarios. Often product owners and/or business analysts only cover typical scenarios. It...
3 comments
Read more

How to write valuable unit tests

In my last blog post I discussed what makes unit tests valuable and how to structure your code so that you can write valuable unit tests. But so far I haven’t got into the nitty gritty of how you should write these valuable unit tests. I’ll address that in this blog post. There are three styles of unit test: Output Verification , also known as the functional style, involves checking the output of a method for a given input. This style of unit testing does not concern itself with the internals of a method. State Verification involves checking the state of an object rather than the output of a method. Collaboration Verification is where collaboration between classes is tested, and it usually involves test doubles such as mocks. See Vladimir Khorikov’s blog post for further information and code examples: http://enterprisecraftsmanship.com/2016/06/09/styles-of-unit-testing/ So which style is best for writing valuable unit tests? Here are the four attributes of a v...
2 comments
Read more