If you are not taking advantage of test automation within a large organisation, you probably should be. It can create game-changing efficiencies and improve software quality. But when we talk to organisations who do utilise test automation, the general sentiment is that it doesn’t provide the expected benefits. Unfortunately, this is typically because it is not implemented in a way that will return the most value. The drivers for Test Automation can sometimes come from bad places – ill-considered directives from senior management, because it is a ‘hot topic’ or simply an attempt at cutting costs. When quality is not the main consideration when making these decisions, this is when it can go horribly wrong and ultimately your customers are the ones that suffer with a poor quality experience.
A mistake that a lot of organisations make, is attempting to automate everything without real thought behind it. There is often an unnecessary goal of automating a percentage of a test pack, without properly analysing the scripts and selecting the ones which would be the most beneficial to the business. This sometimes creates pointless work, just to meet an SLA. It is also common to find third party suppliers only adhering to the SLA’s set out by the customer and not capitalising on opportunities to improve things, increase speed of delivery or save money.
There is a heavy reliance on applying test automation to regression testing and this is usually because test automation tends to be an after-thought or people try to apply test automation retrospectively. Although regression testing is clearly important, and automating these tests is a good way to create some efficiencies, this is usually where the least bugs are found. In some cases this way is the only option but there are still many new projects where test automation is left until the end.
Failing automated tests can also be excused as “that one always fails”. For test automation to be effective, these failing tests should be actioned to provide the value that it should be. Either, there is a real reason why the test is failing or you need to re-write the test so that it provides true results.
For most organisations – the end-user is a human. Yet there seems to be a drive to eliminate the human element within a critical aspect of software delivery. Test Automation serves as super-fast feedback on critical aspects of the application/system/software you are testing but it doesn’t do what a human would. A human can respond to the look and feel of the application they are testing and use their intuition to explore different user journeys. Automated tests on the other hand, are rigid and should only target areas of the application that are similarly rigid.
Critically, the time saved by automating scripts should be used to take advantage of exploratory testing. There is a misconception that once a test is automated, there is 100% saving in a manual testers time. But if you strive for better quality, this should not be the case. Humans don't always do what you expect them to do or how you would like them to use your application (the happy path). Exploratory testing can be a great way to discover what happens when the 'unhappy path' is taken.
In an ideal world, we should be automating earlier in the software delivery life cycle. This can be done in parallel, with a test automation engineer working alongside a manual tester, leading to the natural formation of a concise regression pack. Or better yet, this could be done by the same person (should they possess the correct skills). A person who fully understands the business, processes and the associated risks should be able to thoroughly analyse test scripts and create a true risk based approach.
As we have already mentioned, not everything can be automated, and actually, with the size and complexities within some organisations, not everything can be tested. So a risk based approach must be taken to mitigate any issues to the business. Spending your time on the most important areas of the application will then free up time to move on to the next big feature or new application. This will enable you to move more nimbly and keep up with customer expectations.
The other human element which should not be forgotten, is within your supplier arrangements. Your suppliers should be experts in their chosen areas. You need to spend time building strong relationships with suppliers and third parties, communicating the business objectives and your expectations of them. If you have done this well, you should then be able to trust them to do the job you have brought them on board to do, and more – not just stick to SLA’s.
Your customers are humans, so don’t lose the human element from the most important part of the process – quality. Of course there are great benefits of automating repetitive, rigid tests but we should only apply it to the relevant tests. Going forwards, we need to re-think the way that we implement test automation either by having manual testers working alongside a test automation engineer, or finding and developing people to be able to do both. Finding the right balance will help to create amazing experiences for your customers.