Monday, November 10, 2014

Performance testing environment

One of the very essential things for implementing performance testing for a web application is to have a suitable performance testing environment. Ideally, it should be a replica of the production environment. However, as the production environments tend to normally be quite expensive, reproducing them for a non functional testing activity such as performance testing does not make for a good business case. Hence, most small and medium scale IT businesses tend to undergo their performance testing on the same environment that they undertake their other testing activities on.

One thing to note about performance metrics for a website is that we cannot scale results. Test environments have hardware far less capable of handling as many users as what the production systems can handle, would certainly have different performance results. The results achieved for the tests on the performance environments are far difficult to scale to match the hardware capabilities of the production environment. For a test environment which is a replica of the production environment the results could be straight away used to predict the performance of release code. But as in most cases where the test environment is not the same as the production environment how do we predict the performance wise health of a release? This was a question I had put across to one of the senior managers from another branch of the project that I worked for. He first said that he was lucky to have an environment that was similar to the production environment, so he didn't have to bother about such a situation. But to answer my question, he said if he were in my situation he would try and extrapolate results by plotting a graph based on numerous tests with incremental hardware capabilities.

Eg.
For a production web server with following hardware specifications:
  • 16GB RAM
  • 8 cores
  • 160 GB Disk space
We would have to start a performance test with web server in the test environment having specifications 2 GB RAM, 1 core, 20 GB disk space (assuming the network bandwidth is similar) note down results, and then increase hardware specifications to 4GB, 2 cores and 40GB of hard drive space and note down results again. This way we can plot a graph of parameters against response times and extrapolate results to predict results in the production environment.