Products

Problems
we solve

We can help your business

Request a Free Demo / trial

Insights

Insights | Performance Testing
22 July, 2022

Performance Testing: How to Choose Performance Tools

How to Choose the Perfect Performance Test Tool

I’ve been involved in performance testing for 3 decades. During that time, it has evolved massively and continues to change. If you’re a newbie, performance testing can be a tough subject to get your head around.

Increasingly, I talk to people at companies who’ve never done proper performance testing, but whose solutions are starting to come under additional scrutiny.

Often the people I talk to, and their companies, need to start performance testing but don’t know where to begin.

Choosing the right performance testing tools can be mind-boggling.

There are several considerations to bear in mind. For example:

  1. Do you want to test earlier or later?
  2. Are you going to generate load from the cloud, on-premises, or both?
  3. Is open-source the right option?

In this insight, I’ll arm you with the basics so you can make an informed decision when choosing performance test tools.

Performance Testing Terminology: Simplified

Before you start looking for tools, you need to get your head around the terminology.

One of the most off-putting things about performance testing are the myriad terms people use to describe the activity. You’ll hear stress testing, load testing, volume testing, and performance engineering, to name but a few.

I have heard of up to 26 different types of performance testing. Most people agree that there are more than 5. Personally, I would argue that there are at least 8 main variations.

Technically there are subtle difference between these terms, but most people use them interchangeably. At this stage, you really don’t need to know the differences.

Performance testing is a convenient umbrella term which covers things perfectly adequately.

Performance Testing is Not a Dark Art

Performance testing has a reputation for being complex, especially if you’re new to the field. In fact, I’ve heard performance testing described as a ‘dark art’, although is mostly by consultancies or contractors looking to increase their day rate.

The performance testing crowd have a vested interest in conjuring this imagery, they want to make performance testing seem as mysterious as possible.

However, things have changed, and now modern performance testing tools do much more of the heavy lifting, making it easier for your to test you website and applications.

In reality, performance testing is 0% arcane mysticism and 100% maths and engineering.

With the right test tool, you can remove the veil of mystery surrounding performance testing.

Things To Consider When Selecting Performance Test Tools

The real fun starts once you’ve got past the nomenclature. Choosing an appropriate approach and selecting the right tools. This can be a bit of a minefield.

It’s worth considering a few questions before you start looking at specific tools:

  1. Are you going to test web, ERP, mobile apps, or something else?
  2. Do you want your developers to run performance unit tests?
  3. Do you want to test after regression testing and before UAT?
  4. Do your tools need to simulate mobile/LAN/WAN network conditions?
  5. Do your tools need to simulate a global user portfolio?
  6. How quickly do you need to start performance testing?

You might not know the answers to these questions, especially if you’re only testing the water.

If you do know the answers, great! If not, don’t worry – There’s a perfect solution 

How to Choose Performance Test Tools

If you’re just starting out on your performance testing journey, you want to put your best foot forward. You need a solution that will allow best practices, without breaking the bank.

The perfect performance test tools must:

  1. Allow your developers to get started quickly and easily with IDE-driven testing.
  2. Allow detailed, comprehensive end-to-end testing of your final solution.
  3. Have cloud-based infrastructure – this allows you to test easily and quickly without having to provision any servers for load injection.
  4. Be easy to configure and use
  5. Provide easy scripting options for beginners
  6. Provide advanced functionality for experienced users
  7. Make setting up and running a test fast and easy
  8. Provide clear and understandable results.

Unfortunately, not all test tools can do all of this. Most are let down by poor analysis and reporting, among other things. However, there is a perfect way to satisfy all the above criteria.

Based on years of experience in the performance testing field, my recommendation is that you choose not one, but two performance test tools:

  1. A free, lightweight solution that your developers can use
  2. A comprehensive, cloud-based solution for your final end-to-end scenarios

Of course, these two systems must be compatible with each other.

Why You Should Avoid Open Source Tools

A word of warning; when it comes to performance test tools, you’re going to run into very vocal groups who will try to sway your opinion towards imperfect solutions.

The most vocal of these groups is the Open-Source community. If you haven’t met one of these yet, then you can spot them because they will always tell you that open-source is the answer, even before you have explained the question. After all, they are free, right?

These are the same people we mentioned earlier, the consultancies and contractors, who like to maintain a shroud of mystery around everything they do.

They’ll push you onto open-source tools, so they can sell you more days.

The problem is, whatever you save on the tool, you will likely spend more resources. Even if you’re using your own people, open source tools take considerably more effort.

Open-source tools are more complex to use and require significantly more time to achieve the same results as tools like LoadRunner.

Plus, once you’ve started down the open-source path, you can end up locked into it. These consultancies or contractors will likely have developed a unique solution for you. One that you need them to maintain.

If you’re looking for fast and efficient performance testing, Open-source tools are simply not the right option for you.

Open-source tools require far more effort in both creation and maintenance, plus you need to spin up your own performance testing environment. While the free aspect might sound great initially, the total cost of ownership can be quite scary.

Whilst initially it might appear free to use, once you add on the hidden costs (such as hosting and effort), it will quickly overtake a paid tool.

These Are The Perfect Performance Test Tools

If you’re looking for the perfect test toolset, I strongly recommend the Micro Focus LoadRunner toolset. It meets all criteria defined above and so much more:

  1. LoadRunner Developer is a free IDE-based solution that your developers can use
  2. LoadRunner Cloud is the most powerful cloud-based performance testing tool available. It is simple, smart and scalable. Plus, it is very competitively priced.

These two tools are also entirely compatible with one another.

The LoadRunner tools are the most comprehensive performance tools on the market. They come out on top in head-to-heads with other tools.

The LoadRunner suite of tools covers more protocols than any other solution, allowing you to test everything from web to enterprise software, to App testing, and pretty much anything else you can think of.

If your performance testing partners are not suggesting them, maybe you should ask them why and what the difference in consultancy costs would be if they used LoadRunner rather than an open source solution.

Plus, with Micro Focus, you know what you’re getting. The tools are incredibly quick to deploy, resilient and come with professional-level support.

Arrange a demo of the Perfect Performance Test Tools

Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

22nd July 2022
cut testing costs

Cut Software Testing Costs With These Essential Tactics

Balancing rigorous testing with cost-effectiveness is a challenge. You can test too much, take too much time, and spend too much money on testing. Luckily, there are proven ways to refine your testing practices, increase efficiency, and cut your software testing costs.

performance testing is like herding cats

Herding Cats: Performance Testing Strategies for 2024

Performance testing is more challenging than ever. Technology is more complex, customer expectations are higher, and an increasing number of people and departments need to be involved to make performance testing effective. This article shines a light on strategies to help you execute successful performance test cycles in 2024.

skimping on performance test tools

Why Skimping on Performance Tools Is a False Economy

Performance tool cost is likely to be less than 10% of the performance testing budget, yet many still think that slashing this figure saves money. Common sense tells us that if you do not spend on quality tools, you will spend more on people. In this insight, I look at why investment in professional performance test tools is so important.

The Human Cost of Software Failure

Horizon: The Human Cost of Software  Failure

The Post Office Horizon scandal destroyed lives and livelihoods and is rooted in defective software. This insight looks at the scandal from a testing perspective and serves as a cautionary tale for all involved in deploying large-scale IT systems.

LoadRunner Cloud Migration

Remote Testing with LoadRunner Might Be Breaching Your Contract

Did you know that most LoadRunner Professional licenses are only valid for a specific location? This means that (unless you’ve bought the country or worldwide usage add-on) working from home, other offices or geographic locations is prohibited and doing so could incur financial penalties.

Software Testing Steps For an Easier Life

Simplify Software Testing With These 3 Easy Steps

Software test professionals like us are under constant pressure to get more done in less time. These three software testing steps will make your life easier, help you achieve your goals, and allow you to focus on your own progression.

2023 top testing articles

The Most Popular Testing Articles of 2023

in 2023 we produced 50+ testing insights to help software professionals test better than ever before and consider testing from a different perspective – today we look back at the 3 most popular of these insights.

rudolph the red nosed tester

Rudolph the Red-Nosed Tester

It was December 23rd at the North Pole, and Santa’s software team were enjoying their Christmas go-live party. They’d pushed the new CRM (Christmas Resource Management) system over the line just in time, and they were very pleased with themselves about it. Everyone apart from Rudolph, the tester.

Insights

Search

Related Articles

InsightsTrending

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.