At the most recent North West Tester Gathering held at Amaze in Manchester, Jonny Fletcher, Test Architect at ROQ delivered a stimulating presentation on how DevOps is driving the adoption of wider test automation. Here are some of the key take outs from his talk.
Across the IT industry, the demand to deliver solutions ever faster whilst satisfying the expectation of improving quality continues to gather pace, as users become ever more sophisticated and demanding. Against this background, the DevOps approach has become a hot topic and is becoming widely adopted as established companies seek not to be left behind by the so called ‘disruptors.’
With the widespread adoption of agile development methodologies and the need for more frequent deployments, the volume of testing required has increased significantly, especially around regression testing. With this increased volume of development, and the associated testing this brings, comes the need for test automation. Put simply, DevOps cannot succeed if it still requires lots of test cases to be executed manually.
When and Where to Automate
In order to be truly effective your test automation efforts need to address at least three different levels across the solution. In non-DevOps software development, many people end up inadvertently falling into the ‘ice cream cone anti-pattern’ for testing by putting more emphasis on automating at the UI level. However, a more practical approach is one that flips that ice cream cone upside down. This approach, the ‘ideal test automation pyramid’ made popular by Mike Cohn, improves the ROI of automation and guarantees that you will receive the most benefits from your testing effort.
DevOps Demands of Test Automation
Automated scripts must provide instant and accurate feedback to support decision making. In the CI/CD context we need to make decisions up-front around what to do in the event of failure, i.e., which failures should prevent deployment or halt the process. Whilst there has also been a requirement for automated scripts to be reliable, the emphasis becomes even more acute when any false negative/failed run means an unwarranted delay to delivery or release. The emphasis on portability can also become more important, as the automated build process may spin up and deploy to an entirely new environment or set of devices and invoke our scripts to run on them.
In a fast-changing world, which demands more frequent deployments to keep up with the pace of change, test automation can help drive huge efficiencies in a DevOps environment. It is imperative that all parties engage as early in the process to maximise test coverage and to provide feedback to support the decision-making process. However, test automation isn’t a silver bullet and doesn’t replace the need for great manual testers; it purely enables them to expand the scope of the overall test strategy and to concentrate on exploratory testing and the overall user experience.
Look out for Jonny’s full article which will be featured in this month’s Test Magazine.