We practice a flexible approach to system testing on all development stages in order to reduce eventual future optimization costs. Not only different testing types are utilized but also different approaches such as Concept proof, Regular Performance Drop Analysis, and Capacity Planning are implemented for the purpose of performance analysis.
Performance Analysis Procedure
Each performance analysis stage may consist of several iterations and includes:
- A detailed analysis of actual or expected system load characteristics (during the System initial development stage);
- Precise test planning and appropriate platform preparations;
- Test execution;
- Problems identification and recommendations for their solutions;
- Additional test executions when necessary;
- Composing a summary report for the customer.
Load testing is conducted to assure the system meets the performance requirements. For this purpose, a load equal to the expected working load is applied to the system.
Understanding the actual load upon the System cannot be linear, we have developed a special system capable of reproducing any relevant load by means of copying a real or an assumed model in all the details and modifying it using multiplication factors or altering its profile.
The purpose of stress testing is to analyze durability of your system. Here, the load applied to the system either significantly exceeds the actual load in the operating environment, or is disproportional
Stress testing can reveal problems related to component failure under high load, thus allowing to prepare your system for drastic data flow increase.
During a stability test, a real load is being applied to the testing platform for a long time. Taken in combination with our approach to load testing, a stability test allows to precisely simulate the behavior of the system within a working week.
Performance analysis made in the stage of architectural solution development is especially effective for putting system into production in the future. This approach is called Concept proof testing and is usually implemented at early development stages in order to find out whether the design created is viable or not and to predict its final compliance with performance requirements. Load Testing and Stress-testing are generally used for the purpose of such analysis.
Regular Analysis of Performance Deviation
Regular performance analysis made for evaluative System allows detecting performance deviations in time and making relevant business decisions.
Our automated system for monitoring of key performance parameters provides instant access to the history of changes for the whole period of regular analysis, while graphical presentation of this information makes it easy to understand not only for internal but also for external customers requesting a test.
Being a long-term approach, this method has other benefits as well: for instance, it does not require expensive hardware resources accurately reproducing the live platform configuration.
Being a metric which allows addressing strategic business questions regarding long-term System development, Maximum System Capacity depends on multiple factors such as end-user behaviour, system balance, hardware performance, operating system, and environment configuration.
Capacity analysis of a platform in production is one of the most challenging tasks in the system performance area. This complex research includes not only determination of the maximum system capacity and detection of system bottlenecks which prevent the increase of this level, but also various system durability and balance tests and other processes pointed to find out its optimal configuration.
The criteria which determine maximum system capacity are not limited to component failure only, but also include a number of measures imposed by business requirements to the system, e.g. order processing speed or delay in quote distribution by the system.
Regular system capacity review allows precise planning of further system development in scope of relevant business needs.
We do realize that sometimes our standard well-functioning performance analysis procedures are not enough to answer all the arising business questions:
- Which hardware should be installed to make sure that the system handles specified load?
- How long will it take the system to fully restore functionality for all online users after failure and restart?
- How will restart of particular components affect platform availability to online users?
- How will the platform behave in case of clustering of its components?
These are only a few examples of business questions which clients need an answer to, in order to make the right strategic decisions. Each test designed to answer this kind of questions is developed individually, with due regard of the system, the requirements, and peculiarities of the necessary result.