With a plethora of test automation tools and the rapidity of change, an organisation’s first steps into automation can be quite daunting. The upfront costs typically associated with tooling and upskilling staff can make a real indent in the desired Return on Investment – especially if done incorrectly. Typically, this is the reason that automation never begins or doesn’t realise the full benefits. Here are the main initial considerations that should be made, in order to make automation a real success.
Automation can solve all of the above - with the right approach.
Identify the Automation Opportunities
Identifying the root cause of the problem will influence when and where to start. Consider your appetite for risk. For less risk, choose a smaller software component and conduct an 'Alpha' project. For those with a greater appetite, selecting a critical path will be quite challenging. Although you could make the biggest impact, if unsuccessful it could have a negative effect – far from the desired outcome.
For the biggest wins, identify scenarios for which there are lots of test cases – code once, execute many, many times versus the manual equivalent.
Also, consider testing your API’s, as well as testing at the UI. API testing is much faster and will identify the faults earlier in the SDLC, which will itself reduce costs.
Options for Automation
Tools which offer a record and playback capability lend themselves to GUI testing only, which is far slower than API and typically takes place later in the SDLC. The resulting tests are often ‘brittle’, that is to say they break as soon as the design or functionality changes slightly - the result being that they require greater maintenance. If you do opt for a record and playback tool, consider the fact you will then need a different tool and skill set for API testing.
The choice of tooling is often influenced by the skills in-house, but this could result in an inappropriate tool. A framework should be chosen because it is fit for purpose and future-proof (at least for the foreseeable), also consider supportability. ‘Automation Engineers’ and ‘Software Developer in Test’ are interchangeable job descriptions - ideal candidates will require both testing and development skills.
Coding skills will give you a wider choice of frameworks, making the use of open source tools a more viable option and will inevitably keep the costs down. Consider developing the skills in-house by pair-programming or employ consultants to implement the core framework and train your staff. Alternatively use a test partner who will also maintain the framework and provide ongoing support.
Consider the Costs
Initial test automation effort and costs will require sponsorship, so it is essential to get buy-in from key stakeholders. There are core automation components that can be re-used across projects, not to mention the skills acquired and lessons learned during the Alpha. Automation development plus execution should never be slower than manual testing, so the Return on Investment should be realised quickly. If it is not, you are doing it wrong – automation engineers without the key skills or experience could easily burn up the Alpha budget. On the surface, the cost of an experienced Test Automation Architect may not look as attractive as going it alone but their wealth of knowledge would provide an almost-instant, reliable framework and will quickly get the figures back in your favour.
As well as the cost savings achieved by automated test suites, consider the ongoing costs of automation. As the software product changes, the automation pack will also need updating. With a well-designed, maintainable framework, these costs should be kept to a minimum.
When Does Automation Fail?
A framework must have two key attributes: maintainability and robustness. Without those, test automation in your organisation will fail due to unjustifiable automation effort plus a lack of trust in the results. Considering all of the above will give automation the greatest chance of success in your organisation!
Lindsey McKechnie - Test Automation Architect