The client is an innovative market leader in the television broadcasting industry, with customers ranging from; global broadcasters, tele-production facilities, service providers and independent video professionals.
The client develops and produces technology and services for live video production, news and play out by providing glass (camera) to glass (television) solutions. The client delivers end-to-end television production and content distribution workflows that enable the efficiency and flexibility broadcasters need to stay ahead of the competition.
Amongst other innovative products, the client has been developing a cloud-based media workflow solution and, to support the development and ongoing maintenance of this product, approached ROQ to design and implement an automated test framework.
The automation framework had to fulfil two main objectives; to ensure that the existing product was not impacted by new development (Regression Testing) and that new features/enhancements meet the end user requirements, with both functional and non-functional testing taken into consideration.
ROQ has delivered an automation framework responsible for testing over 70% of the functional areas provided through the cloud-based workflow solution with tests ranging from creating brand new television channels all the way through to managing “Breaking News” updates.
There were several key challenges that needed to be overcome, including the very nature of the solution itself. As with any media management tool these can be highly dynamic with constantly changing values (queued items in a playlist, timestamps/durations etc.) and this posed a problem in how to ensure the tests executed reliably. By implementing a robust exception management policy this ensured that the tests were consistent and that the results could be relied upon to give an accurate picture of the state of an environment at any given time.
ROQ’s responsibilities ranged from selecting the core technologies that underpin the foundation of the solution to implementing an end-to-end automation framework including integrating with the clients’ existing test case management system; source control and delivering a reporting suite to provide immediate feedback on the health of the product in a continuous integration environment.
By choosing open-source tools to achieve the goal set by the client (Selenium WebDriver, NUnit amongst others), whilst also acknowledging the clients’ existing/preferred technology choices (having “in-house” C# developers/TFS source control tool), ROQ has also helped the client keep costs to a minimum and ensured that the framework delivered can be maintained going forwards.
ROQ also provided support in terms of future development of the test framework; giving advice on best practises (using the “Page Object Model” approach); how to structure tests; and passing on “lessons learnt”.
By engaging ROQ in the development of the automation framework the client immediately benefitted from a wealth of experience our automation consultants have. ROQ were able to source the correct tools for the job while taking into consideration the clients’ existing tools/skillsets and delivered an end-to-end automation framework much faster than the client would have been able to if this was done in house. All of this was done remotely from our Test Lab in Chorley reducing the costs to the client.
The automation framework was designed to complement the clients’ new Agile way of working by exposing functionality to the development team to integrate with the tools they use. This ensured that any changes were tested immediately not only to prove that the system acted as expected but to ensure that existing features continued to function normally. By catching defects early this reduced the amount of rework required and ensured that the team could concentrate on enhancing the solution by implementing new features and refining the end user experience without the worry of having to execute a whole Regression suite manually.