Availability
The amount of time an application is available to the end user.
Throughput
The rate at which application-oriented events occur.
Utilization
The percentage of the theoretical capacity of a resource that is being used. Examples
include how much network bandwidth is being consumed by application traffic and the
amount of memory used on a server when a thousand visitors are active.
(from: The Art of Application Performance Testing)
Donnerstag, 10. Februar 2011
Montag, 24. Januar 2011
Important things before you start performance testing
Questions to ask
Performance specifications should ask the following questions, at a minimum:- In detail, what is the performance test scope? What subsystems, interfaces, components, etc. are in and out of scope for this test?
- For the user interfaces (UI's) involved, how many concurrent users are expected for each (specify peak vs. nominal)?
- What does the target system (hardware) look like (specify all server and network appliance configurations)?
- What is the Application Workload Mix of each application component? (for example: 20% login, 40% search, 30% item select, 10% checkout).
- What is the System Workload Mix? [Multiple workloads may be simulated in a single performance test] (for example: 30% Workload A, 20% Workload B, 50% Workload C)
- What are the time requirements for any/all backend batch processes (specify peak vs. nominal)?
- Make sure, your functional tests have gone through at least once. It is way too expensive to find a dead lock or a priority inversion bug in performance testing.
- The system you do performance testing should be as close as possible to production.
- Setup a realistic environment. Especially databases!
- see http://video.google.com/videoplay?docid=-6891978643577501895# at 15:34
Links & Books regarding Stress testing
Tools for Performance Testing
Commercial
- IBM Rational Performance tester
- A broad List of performance testing tools
- loadUI
- HP LoadRunner
- AppPerfect
- Silkperformer (Borland)
- WebLoad
- For API: Grinder
- For anything GUI: JMeter
- http://www.opensta.org/ (Windows only)
Basics of Performance Testing
Different types of performance testing:
Load Testing
A load test is usually conducted to understand the behavior of the application under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.
Stress Testing
Stress testing is normally used to understand the upper limits of capacity within the application landscape. This kind of test is done to determine the application's robustness in terms of extreme load and helps application administrators to determine if the application will perform sufficiently if the current load goes well above the expected maximum.
Endurance Testing (Soak Testing)
Endurance testing is usually done to determine if the application can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good or better than at the beginning of the test.
Spike Testing
Spike testing, as the name suggests is done by spiking the number of users and understanding the behavior of the application; whether performance will suffer, the application will fail, or it will be able to handle dramatic changes in load.
Configuration Testing
Configuration testing is another variation on traditional performance testing. Rather than testing for performance from the perspective of load you are testing the effects of configuration changes in the application landscape on application performance and behaviour. A common example would be experimenting with different methods of load-balancing.
Isolation Testing
Isolation testing is unique to performance testing but a term used to describe repeating a test execution that resulted in an application problem. Often used to isolate and confirm the fault domain.
Excerpts from http://en.wikipedia.org/wiki/Software_performance_testing.
Availability Testing
How long does it take for the system to recover after it has failed (bcp-testing)
How long does it take for the system to recover after it has failed (bcp-testing)
Excerpts from http://en.wikipedia.org/wiki/Software_performance_testing.
Abonnieren
Posts (Atom)