Products

Problems
we solve

We can help your business

Request a Free Demo / trial

Insights

Insights | Performance Testing
22 July, 2022

Performance Testing: How to Choose Performance Tools

How to Choose the Perfect Performance Test Tool

I’ve been involved in performance testing for 3 decades. During that time, it has evolved massively and continues to change. If you’re a newbie, performance testing can be a tough subject to get your head around.

Increasingly, I talk to people at companies who’ve never done proper performance testing, but whose solutions are starting to come under additional scrutiny.

Often the people I talk to, and their companies, need to start performance testing but don’t know where to begin.

Choosing the right performance testing tools can be mind-boggling.

There are several considerations to bear in mind. For example:

  1. Do you want to test earlier or later?
  2. Are you going to generate load from the cloud, on-premises, or both?
  3. Is open-source the right option?

In this insight, I’ll arm you with the basics so you can make an informed decision when choosing performance test tools.

Performance Testing Terminology: Simplified

Before you start looking for tools, you need to get your head around the terminology.

One of the most off-putting things about performance testing are the myriad terms people use to describe the activity. You’ll hear stress testing, load testing, volume testing, and performance engineering, to name but a few.

I have heard of up to 26 different types of performance testing. Most people agree that there are more than 5. Personally, I would argue that there are at least 8 main variations.

Technically there are subtle difference between these terms, but most people use them interchangeably. At this stage, you really don’t need to know the differences.

Performance testing is a convenient umbrella term which covers things perfectly adequately.

Performance Testing is Not a Dark Art

Performance testing has a reputation for being complex, especially if you’re new to the field. In fact, I’ve heard performance testing described as a ‘dark art’, although is mostly by consultancies or contractors looking to increase their day rate.

The performance testing crowd have a vested interest in conjuring this imagery, they want to make performance testing seem as mysterious as possible.

However, things have changed, and now modern performance testing tools do much more of the heavy lifting, making it easier for your to test you website and applications.

In reality, performance testing is 0% arcane mysticism and 100% maths and engineering.

With the right test tool, you can remove the veil of mystery surrounding performance testing.

Things To Consider When Selecting Performance Test Tools

The real fun starts once you’ve got past the nomenclature. Choosing an appropriate approach and selecting the right tools. This can be a bit of a minefield.

It’s worth considering a few questions before you start looking at specific tools:

  1. Are you going to test web, ERP, mobile apps, or something else?
  2. Do you want your developers to run performance unit tests?
  3. Do you want to test after regression testing and before UAT?
  4. Do your tools need to simulate mobile/LAN/WAN network conditions?
  5. Do your tools need to simulate a global user portfolio?
  6. How quickly do you need to start performance testing?

You might not know the answers to these questions, especially if you’re only testing the water.

If you do know the answers, great! If not, don’t worry – There’s a perfect solution 

How to Choose Performance Test Tools

If you’re just starting out on your performance testing journey, you want to put your best foot forward. You need a solution that will allow best practices, without breaking the bank.

The perfect performance test tools must:

  1. Allow your developers to get started quickly and easily with IDE-driven testing.
  2. Allow detailed, comprehensive end-to-end testing of your final solution.
  3. Have cloud-based infrastructure – this allows you to test easily and quickly without having to provision any servers for load injection.
  4. Be easy to configure and use
  5. Provide easy scripting options for beginners
  6. Provide advanced functionality for experienced users
  7. Make setting up and running a test fast and easy
  8. Provide clear and understandable results.

Unfortunately, not all test tools can do all of this. Most are let down by poor analysis and reporting, among other things. However, there is a perfect way to satisfy all the above criteria.

Based on years of experience in the performance testing field, my recommendation is that you choose not one, but two performance test tools:

  1. A free, lightweight solution that your developers can use
  2. A comprehensive, cloud-based solution for your final end-to-end scenarios

Of course, these two systems must be compatible with each other.

Why You Should Avoid Open Source Tools

A word of warning; when it comes to performance test tools, you’re going to run into very vocal groups who will try to sway your opinion towards imperfect solutions.

The most vocal of these groups is the Open-Source community. If you haven’t met one of these yet, then you can spot them because they will always tell you that open-source is the answer, even before you have explained the question. After all, they are free, right?

These are the same people we mentioned earlier, the consultancies and contractors, who like to maintain a shroud of mystery around everything they do.

They’ll push you onto open-source tools, so they can sell you more days.

The problem is, whatever you save on the tool, you will likely spend more resources. Even if you’re using your own people, open source tools take considerably more effort.

Open-source tools are more complex to use and require significantly more time to achieve the same results as tools like LoadRunner.

Plus, once you’ve started down the open-source path, you can end up locked into it. These consultancies or contractors will likely have developed a unique solution for you. One that you need them to maintain.

If you’re looking for fast and efficient performance testing, Open-source tools are simply not the right option for you.

Open-source tools require far more effort in both creation and maintenance, plus you need to spin up your own performance testing environment. While the free aspect might sound great initially, the total cost of ownership can be quite scary.

Whilst initially it might appear free to use, once you add on the hidden costs (such as hosting and effort), it will quickly overtake a paid tool.

These Are The Perfect Performance Test Tools

If you’re looking for the perfect test toolset, I strongly recommend the Micro Focus LoadRunner toolset. It meets all criteria defined above and so much more:

  1. LoadRunner Developer is a free IDE-based solution that your developers can use
  2. LoadRunner Cloud is the most powerful cloud-based performance testing tool available. It is simple, smart and scalable. Plus, it is very competitively priced.

These two tools are also entirely compatible with one another.

The LoadRunner tools are the most comprehensive performance tools on the market. They come out on top in head-to-heads with other tools.

The LoadRunner suite of tools covers more protocols than any other solution, allowing you to test everything from web to enterprise software, to App testing, and pretty much anything else you can think of.

If your performance testing partners are not suggesting them, maybe you should ask them why and what the difference in consultancy costs would be if they used LoadRunner rather than an open source solution.

Plus, with Micro Focus, you know what you’re getting. The tools are incredibly quick to deploy, resilient and come with professional-level support.

Arrange a demo of the Perfect Performance Test Tools

Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

22nd July 2022
How to Choose A Test Management Tool

How to Choose The Right Test Management Tool

Test management tools ensure efficient, effective, and auditable testing processes. When choosing an enterprise-level test management solution, it’s essential to use a proven and trusted solution.

why choose loadrunner

Performance Testing: 6 Reasons Companies Choose LoadRunner

Despite what some might think, LoadRunner is not a single performance testing tool but a family of world-class load performance testing solutions. The family has recently undergone a name change that you may not be familiar with. Today, I will explain why LoadRunner family products are relied on by the largest companies.

System 38

Carry On Testing: A Couple of Funny QA Mishaps

Ah, software testing—where meticulous planning meets unexpected chaos and where even the most rigorous quality assurance can sometimes go awkwardly awry. As we bid farewell to 2024, it’s time to take a step back and indulge in some light-hearted reflection on the quirky side of our profession.

Tomorrow’s World v. Today’s Reality: My WQR Thoughts

The World Quality Report (WQR) 2024-25 looks at the current state and future trends of Quality Engineering—albeit from a particular perspective. It is based on a survey of 1,775 executives across multiple sectors and regions. While it is a potentially interesting source of information, I’m always left asking the question: Is it useful?

LoadRunner Enterprise 243

LoadRunner Enterprise 24.3 – Release Highlights

Today we look at OpenText LoadRunner Enterprise 24.3, which introduced several enhancements to their enterprise-grade performance testing solution. We’ve been through the release notes so you don’t have to.

OpenText Customer? Read This to Avoid Overpaying for Support Costs

Following its acquisition of Micro Focus, OpenText has implemented a more stringent policy regarding support renewals and term/subscription renewals. This policy could increase your support costs. Today I’ll explain how to minimise costs when renewing your OpenText software support or subscriptions.

AI caution

WQR: This 1 AI Recommendation Could Derail Your QA Strategy

The World Quality Report 2024-25 (WQR) provides a few interesting insights into adopting AI in software testing. And, while many their recommendations are sound for most companies, one specific recommendation regarding AI implementation is a terrible idea for most businesses.

Insights

Search

Related Articles

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.