Shunra's Network Catcher: http://www.shunra.com/shunrablog/index.php/2010/10/21/getting-accurate-results-from-scalability-testing/
None of us want our sites to crash like the Chase site outage in mid September. I’m not privy to the details of that site crash, but I can tell you that many sites degrade under peak user traffic unexpectedly and eventually crash. Why? It’s often incorrectly performed scalability tests. Over the years, I’ve met many load testing clients that were interested in “seeing the load level where the site crashes.” While this is a very interesting idea, it really shouldn’t have a place in today’s web site and Web Application load testing arena. Here is the real question that you should be asking yourself, “Will end users perceive good performance at peak traffic levels?” Let’s face reality, it does not matter if your site technically did not “crash” while supporting up to a million user visits per hour if your site was a total turn-off for all of those visitors due to poor site performance. It’s simple, validate that your site is fast under peak artificial traffic, and it will stand up under your peak end-user traffic, right? Only if your artificial traffic behaves the same as production traffic!
Good performance at peak loads should be something that we think about from design and development all the way through to the production environment. It starts with understanding what users are going to do on the site, producing test scenarios that match that behavior, and applying those scenarios to your functionality and load testing efforts in the preproduction environment. Hopefully you are nodding your head as you just read my last sentence, but here is something that you may not have considered. Have you measured the Latency, Packet Loss, and bandwidth for your typical user population and recreated that in your preproduction environment? “Why does it matter?”, you ask. It can dramatically affect system performance and stability and always has an impact on end user performance.
Think about it this way, if you are load testing and functionality testing in the preproduction environment with lightning fast latency, near infinite bandwidth, and zero packet loss then you are gaming the system. In the lab, your packet latency is less than 1 millisecond, but your users are going to be experiencing latencies of 30, 50, or even 100 milliseconds latency, which translates into keeping network resources tied up for much longer timeframes. That leads to longer Web Session with more simultaneous HTTP connections and longer Application sessions, which can even affect the database. These are all infrastructure resources that are utilized differently using identical load scenarios and load levels, but have a dramatic impact on end user performance and reduce the scalability of the system.
Applying end user latency, loss, and bandwidth is easy with tools like Shunra’s VE Desktop, which can be used in functional testing to accurately apply the correct network conditions to boost accuracy with the latency that end users would experience in the real world. Shunra can go a step further and help you to determine the latency, loss, and bandwidth that you can plug right into the environment during testing using the Network Catcher tool.
Shunra also has a variety of options for emulating network conditions during load testing ranging from appliances Like the STA to software. The appliance acts like a router on steroids applying network impairments dynamically to the network traffic during the test treating traffic from different URLs and IPs like they are in different cities.
If you are using HP’s LoadRunner load testing tool, Shunra has an elegant solution called Shunra for HP Software, which works seamlessly with the LoadRunner during the load tests to apply the latency, packet loss, and bandwidth during these critical scalability tests.
Regardless of your testing methods, be sure to include identical network conditions into your testing for the most accurate results.
Good performance at peak loads should be something that we think about from design and development all the way through to the production environment. It starts with understanding what users are going to do on the site, producing test scenarios that match that behavior, and applying those scenarios to your functionality and load testing efforts in the preproduction environment. Hopefully you are nodding your head as you just read my last sentence, but here is something that you may not have considered. Have you measured the Latency, Packet Loss, and bandwidth for your typical user population and recreated that in your preproduction environment? “Why does it matter?”, you ask. It can dramatically affect system performance and stability and always has an impact on end user performance.
Think about it this way, if you are load testing and functionality testing in the preproduction environment with lightning fast latency, near infinite bandwidth, and zero packet loss then you are gaming the system. In the lab, your packet latency is less than 1 millisecond, but your users are going to be experiencing latencies of 30, 50, or even 100 milliseconds latency, which translates into keeping network resources tied up for much longer timeframes. That leads to longer Web Session with more simultaneous HTTP connections and longer Application sessions, which can even affect the database. These are all infrastructure resources that are utilized differently using identical load scenarios and load levels, but have a dramatic impact on end user performance and reduce the scalability of the system.
Real Network Conditions Play a Big Role in System Scalability
Shunra also has a variety of options for emulating network conditions during load testing ranging from appliances Like the STA to software. The appliance acts like a router on steroids applying network impairments dynamically to the network traffic during the test treating traffic from different URLs and IPs like they are in different cities.
If you are using HP’s LoadRunner load testing tool, Shunra has an elegant solution called Shunra for HP Software, which works seamlessly with the LoadRunner during the load tests to apply the latency, packet loss, and bandwidth during these critical scalability tests.
Regardless of your testing methods, be sure to include identical network conditions into your testing for the most accurate results.
No comments:
Post a Comment