Cloud Computing has been there for a long time. However, in the span of last few years, it has evolved into a rapidly growing sector with an ability to handle large volumes of data in an impeccable manner, while ensuring enhanced performance. Cloud Computing market is projected to reach $411b by 2020, and Gartner predicts the worldwide public cloud services revenue to reach $260b by 2017. This reflects the increasingly mainstream status that Cloud Computing is gaining.
Cloud Performance Testing – A Prologue
Cloud performance testing entails that the application has to be tested for scalability, availability, fault tolerance, and security while being hosted in the Cloud. It has to be evaluated for system throughput, latency, the number of parallel users using the app, and the speed under different load conditions along with other performance metrics. The bugs and issues need to be detected and fixed suitably before the app goes live. Overall, cloud performance testing ensures that the app runs efficiently and remains available during critical times.
Done aptly, this considerably reduces costs and risks and requires lower maintenance. Data backup, disaster recovery, and business continuity is far simpler and less expensive with the cloud. Moreover, vast amounts of data can be provisioned in no time, further offering enhanced scalability, productivity, and performance.
Crafting a Cloud Performance Testing Strategy
Let’s try to figure out what should be the right strategy for conducting performance testing in the cloud. The project environments, business drivers, technology stack, acceptance factors, skill set and resource availability are some of the factors that should be taken into account before building up a strategy.
Also, the ease of infrastructure access, cost savings, shorter cycle times, and the cloud type must be synced well with the strategy. Besides, the tests need to be effectively customized for various applications/projects, and different implications of testing on a public cloud against a private cloud must be taken into account. It should also be considered that there are performance tests that are not completely applicable in a Cloud set-up, and they should be dealt with accordingly.
Let’s focus on typical tests applicable in the context of cloud, and the strategies that should be applied:
1. Load Testing: In case of the public cloud platform, information should be extracted from the cloud provider on the load statistics of the other customers sharing the platform to get a deep insight into the kind of response time that can be expected. It should be considered that the response time in a cloud environment may differ from the time taken in a non-cloud environment.
The tests must be run repeatedly when the load is expected to be high to get a range of response times — to establish minimum, maximum and average response time. This shall be helpful in mimicking real-time load situations and keeping a track on the app’s response time.
2. Stress Testing: Stress tests confirm the performance characteristics of the system or app under test when it is subjected to conditions past the anticipated ones during production operations. It determines the maximum load the system may handle before it breaks.
As the public cloud is shared by multiple users unlike private cloud, hence, testing in public cloud requires robust planning and execution. A test plan should be prepared with clearly defined goals, and tools must be evaluated by measuring their effectiveness in materializing the goals.
3. Scalability and Elasticity Testing: Elasticity testing ensures if the performance is in sync with the defined SLAs, and is additionally scalable up to an expected level.
It should be conducted by gradually increasing the load to cross the threshold of the current limit, to verify if the system scales up as the load increases and scales down as the load decreases. The boundary value analysis testing technique might come handy to help determine the threshold.
4. Capacity Testing: Cloud offers testers to scale from thousands to n number of concurrent users to analyze the breaking points and capacity thresholds to contest highly predictable demand levels. This offers testers with a better picture of possible runtime errors and prepares companies for peak demand times by giving them an idea of the number of users their app can handle.
Capacity testing may be used during the design phase to benchmark the number of users a system may handle, given a particular set of circumstances. Additionally, with capacity testing, it is possible to proactively increase infrastructure before any limits are reached, or UX deteriorated.
5. Latency Testing: Cloud Service latency refers to the delay between a client’s request and a cloud service provider’s response.
Measuring latency is a bit complicated in the cloud. Tools like trace route and pings work best on the ICMP packets, which are not ordinarily used. Hence, you may use hping, using which, own packet can be crafted with the protocol and flags one wants.
6. Failover Testing: Cloud infrastructure has made Disaster Recovery remarkably affordable but the Disaster Recovery plan is good only when it is tested and ensured that it operates well.
This entails directing high loads to your production servers and disabling various zones, servers, and services to test the ability of your infrastructure to withstand failure.
7. Browser Performance Testing: With the cloud, the companies can simulate the production scenarios to authenticate the scenarios in the different environments, with multiple browsers in different OSs and latest versions available in the market.
8. Endurance Testing: These tests must be run to ensure that the app is capable enough to handle the extended load without any deterioration of response time.
Memory monitoring tools can be installed in the virtual environment itself. Memory leakage in the app, response time, database connection etc. can be effectively verified through endurance testing. Perform this testing as the last stage of the performance run cycle.
The strategy includes setting up of the test environment, creating a test plan, scenarios, offering test estimation, risk analysis, and action, and setting up the test schedule. Finally, understand that this is a long process and will take time.
After the strategy is finalized and the setup is done, execution of tests is the next step. It’s then important to regularly monitor and analyze test results in real-time, offering the requisite scope to improve the app performance over a period of time. Finally, all policies must be discussed and clarified beforehand with the cloud services provider. The service usage is closely associated with payments – organizations need to pay as per usage and as they scale.
Now we are well aware of the strategies for an effective cloud performance testing. So, let’s get started!!