I’ve been involved in performance testing for 3 decades. During that time, it has evolved massively and continues to change. If you’re a newbie, performance testing can be a tough subject to get your head around.
Increasingly, I talk to people at companies who’ve never done proper performance testing, but whose solutions are starting to come under additional scrutiny.
Often the people I talk to, and their companies, need to start performance testing but don’t know where to begin.
Choosing the right performance testing tools can be mind-boggling.
There are several considerations to bear in mind. For example:
- Do you want to test earlier or later?
- Are you going to generate load from the cloud, on-premises, or both?
- Is open-source the right option?
In this insight, I’ll arm you with the basics so you can make an informed decision when choosing performance test tools.
Performance Testing Terminology: Simplified
Before you start looking for tools, you need to get your head around the terminology.
One of the most off-putting things about performance testing are the myriad terms people use to describe the activity. You’ll hear stress testing, load testing, volume testing, and performance engineering, to name but a few.
I have heard of up to 26 different types of performance testing. Most people agree that there are more than 5. Personally, I would argue that there are at least 8 main variations.
Technically there are subtle difference between these terms, but most people use them interchangeably. At this stage, you really don’t need to know the differences.
Performance testing is a convenient umbrella term which covers things perfectly adequately.
Performance Testing is Not a Dark Art
Performance testing has a reputation for being complex, especially if you’re new to the field. In fact, I’ve heard performance testing described as a ‘dark art’, although is mostly by consultancies or contractors looking to increase their day rate.
The performance testing crowd have a vested interest in conjuring this imagery, they want to make performance testing seem as mysterious as possible.
However, things have changed, and now modern performance testing tools do much more of the heavy lifting, making it easier for your to test you website and applications.
In reality, performance testing is 0% arcane mysticism and 100% maths and engineering.
With the right test tool, you can remove the veil of mystery surrounding performance testing.
Things To Consider When Selecting Performance Test Tools
The real fun starts once you’ve got past the nomenclature. Choosing an appropriate approach and selecting the right tools. This can be a bit of a minefield.
It’s worth considering a few questions before you start looking at specific tools:
- Are you going to test web, ERP, mobile apps, or something else?
- Do you want your developers to run performance unit tests?
- Do you want to test after regression testing and before UAT?
- Do your tools need to simulate mobile/LAN/WAN network conditions?
- Do your tools need to simulate a global user portfolio?
- How quickly do you need to start performance testing?
You might not know the answers to these questions, especially if you’re only testing the water.
If you do know the answers, great! If not, don’t worry – There’s a perfect solution
How to Choose Performance Test Tools
If you’re just starting out on your performance testing journey, you want to put your best foot forward. You need a solution that will allow best practices, without breaking the bank.
The perfect performance test tools must:
- Allow your developers to get started quickly and easily with IDE-driven testing.
- Allow detailed, comprehensive end-to-end testing of your final solution.
- Have cloud-based infrastructure – this allows you to test easily and quickly without having to provision any servers for load injection.
- Be easy to configure and use
- Provide easy scripting options for beginners
- Provide advanced functionality for experienced users
- Make setting up and running a test fast and easy
- Provide clear and understandable results.
Unfortunately, not all test tools can do all of this. Most are let down by poor analysis and reporting, among other things. However, there is a perfect way to satisfy all the above criteria.
Based on years of experience in the performance testing field, my recommendation is that you choose not one, but two performance test tools:
- A free, lightweight solution that your developers can use
- A comprehensive, cloud-based solution for your final end-to-end scenarios
Of course, these two systems must be compatible with each other.
Why You Should Avoid Open Source Tools
A word of warning; when it comes to performance test tools, you’re going to run into very vocal groups who will try to sway your opinion towards imperfect solutions.
The most vocal of these groups is the Open-Source community. If you haven’t met one of these yet, then you can spot them because they will always tell you that open-source is the answer, even before you have explained the question. After all, they are free, right?
These are the same people we mentioned earlier, the consultancies and contractors, who like to maintain a shroud of mystery around everything they do.
They’ll push you onto open-source tools, so they can sell you more days.
The problem is, whatever you save on the tool, you will likely spend more resources. Even if you’re using your own people, open source tools take considerably more effort.
Open-source tools are more complex to use and require significantly more time to achieve the same results as tools like LoadRunner.
Plus, once you’ve started down the open-source path, you can end up locked into it. These consultancies or contractors will likely have developed a unique solution for you. One that you need them to maintain.
If you’re looking for fast and efficient performance testing, Open-source tools are simply not the right option for you.
Open-source tools require far more effort in both creation and maintenance, plus you need to spin up your own performance testing environment. While the free aspect might sound great initially, the total cost of ownership can be quite scary.
Whilst initially it might appear free to use, once you add on the hidden costs (such as hosting and effort), it will quickly overtake a paid tool.
These Are The Perfect Performance Test Tools
If you’re looking for the perfect test toolset, I strongly recommend the Micro Focus LoadRunner toolset. It meets all criteria defined above and so much more:
- LoadRunner Developer is a free IDE-based solution that your developers can use
- LoadRunner Cloud is the most powerful cloud-based performance testing tool available. It is simple, smart and scalable. Plus, it is very competitively priced.
These two tools are also entirely compatible with one another.
The LoadRunner tools are the most comprehensive performance tools on the market. They come out on top in head-to-heads with other tools.
The LoadRunner suite of tools covers more protocols than any other solution, allowing you to test everything from web to enterprise software, to App testing, and pretty much anything else you can think of.
If your performance testing partners are not suggesting them, maybe you should ask them why and what the difference in consultancy costs would be if they used LoadRunner rather than an open source solution.
Plus, with Micro Focus, you know what you’re getting. The tools are incredibly quick to deploy, resilient and come with professional-level support.
