Performance Testing for Global Data Archiving and Email Management Project

The Client 

The Client is a global supplier of fast-moving consumer goods with instantly recognisable and widely trusted brands. They have made major acquisitions in the food industry and are adapting to consumers’ increasing desire for healthy living. They are committed to a major sustainable living programme and reducing the environmental impact of this sustained growth.

To support the sustainable living programme and reduce the environmental impact that their IT function was having, the Client decided to centralise all e-mail archiving to a UK-based data centre. This also allowed them to meet regulatory obligations and significantly reduce costs.

Integrating this archiving with a large SharePoint instance posed a performance risk as the impact was unknown. A number of third party suppliers were to be involved in the project.

Having already worked for the Client, Roq was engaged as a trusted supplier to provide expertise and capabilities regarding performance testing.

Roq’s Solution

Roq used their Performance Testing framework, which is based around the ‘four Ds’ of Define/Design/Develop/Deploy, and which is supported by the appropriate level of test governance. An initial scoping exercise provided an indication of the timescales, costs, resources, and tools required, and the suggested format of the engagement.

The team included a WAN emulation specialist and performance testers using the EggPlant performance tool. The project planning focussed on both defining a realistic Transaction Volume Model, and on the identification of the WAN characteristics appropriate for the solution deployment.

A key concern was the potential poor response times experienced by global users due to issues such as latency and bandwidth, so a WAN emulation tool (Shunra), was selected to emulate the network conditions found worldwide. This provided the focus of the testing engagement.

Before Roq developed test scripts, 17 test scenarios were identified to simulate areas such as PST ingestion, auto-archiving and journaling. Initial synchronisation scripts were generated, used first in isolation, then as part of a normal work load, and then peak work load test scenarios. Utility scripts were developed to monitor the size of mailboxes and the backlog of journalled e-mails, and these form part of the legacy for future testing.

Some of the key challenges included:

1. The mis-match of environment sizes (Exchange, SharePoint and the archiving solution) meant that Roq had to carefully select the transaction volumes generated directly. In some instances, Roq used alternative methods to generate the required load, bypassing the undersized application environment.

2. The delay in availability of any scaled test environment: In the absence of physical environments, cloud-based environments were developed in order that Roq could commence  scripting at the earliest opportunity.

3. Meeting aggressive timescales after upstream projectdelays: development and testing of load generation scripts were completed in parallel with some execution tasks. A series of normal and peak load tests was then completed with tuning and defect resolution between cycles.

4. Answers to key questions about how the solution had performed under load were disparate across many log files: an additional tool (Splunk) was employed in order to allow more detailed analysis of the files.

A key deliverable was a detailed test report covering the completed testing cycles, defects raised/resolved and details of the systems behaviour under load, the risks that had been mitigated and those risks that remained.

The Outcome  

Roq’s contribution to the project enabled the Client to fully understand the performance risks faced by the project and to successfully mitigate them within the constraints of the test environment. The remaining risks were clearly defined to facilitate informed decision making. The nature of the iterative testing enabled the client and the application team to fix defects and re-test, so that a complete picture of the performance under load was available.

Additionally, the information provided about the new application’s behaviour under load, and the impact on existing systems, will be vital to the on-going support and monitoring of the system. Effective and pro-active management will significantly reduce the Client’s costs in the long run.

Importantly, the WAN emulation tool gave an early view on expected response times for the Client’s global users with a variety of different WAN characteristics. This approach was new to the Client and now gives a performance baseline for future deployments.

Roq’s engagement also left the Client with  a set of re-usable performance test and utility/monitoring scripts which can be used for future regression testing as application or database upgrades are planned and executed.

If you’d like to find out more about how Roq can support your business, please email us at ask@roq.co.uk.