If you’re looking for a new performance tool or new to performance testing, it can be a tough subject to get your head around. I’ve been involved in the industry for 3 decades, and during that time, it has evolved massively.
Increasingly, I talk to people at companies who’ve never done proper performance testing, but whose solutions are coming under additional scrutiny. They know they need to start performance testing, but don’t know where to begin.
Choosing the right performance testing tools can be mind-boggling.
There are several considerations to bear in mind. For example:
- Do you want to test earlier or later?
- Are you going to generate load from the cloud, on-premises, or both?
- Is a professional or an open-source option right for you?
In this insight, I’ll arm you with the basics to help you make an informed decision when choosing performance testing tools.
Performance Testing Terminology: Simplified
Before you start looking for tools, you need to get your head around the terminology.
One of the most off-putting aspects of performance testing is the myriad terms people use to describe the activity. You’ll hear stress testing, load testing, volume testing, and performance engineering, to name but a few.
I have heard of up to 26 different types of performance testing. Most people agree that there are more than 5. Personally, I would argue that there are at least 8 main variations.
Technically, there are subtle differences between these terms, but most people use them interchangeably. At this stage, though, you really don’t need to know the differences.
Performance testing is a convenient umbrella term which covers things perfectly adequately.
Performance Testing is Not a Dark Art
Performance testing has a reputation for being complex, especially if you’re new to the field. In fact, I’ve heard performance testing described as a ‘dark art’, though mostly by consultancies or contractors looking to increase their day rate.
The performance testing crowd have a vested interest in conjuring this imagery; they want to make performance testing seem as mysterious as possible.
However, things have changed: modern performance testing tools do much of the heavy lifting, making it easier for you to test your website and applications.
In reality, performance testing is 0% arcane mysticism and 100% maths and engineering. With the right test tool, you can remove the veil of mystery surrounding performance testing.
Things To Consider When Selecting Performance Test Tools
The real fun starts once you’ve got past the jargon. Choosing an appropriate approach and selecting the right tools.
This can be a bit of a minefield, so it’s worth considering a few questions before you start looking at specific tools:
- Are you going to test web, ERP, mobile apps, or something else?
- Do you want your developers to run performance unit tests?
- Do you want to test after regression testing and before UAT?
- Do your tools need to simulate mobile/LAN/WAN network conditions?
- Do your tools need to simulate a global user portfolio?
- How quickly do you need to start performance testing?
You might not know the answers to these questions, especially if you’re only testing the water. If you do know the answers, great! If not, don’t worry, there’s a perfect solution
How to Choose Performance Test Tools
If you’re just starting out on your performance testing journey, you want to put your best foot forward. You need a solution that will allow best practices, without breaking the bank.
The perfect performance test tools must:
- Allow your developers to get started quickly and easily with IDE-driven testing.
- Allow detailed, comprehensive end-to-end testing of your final solution.
- Have cloud-based infrastructure – this allows you to test easily and quickly without having to provision any servers to generate load.
- Be easy to configure and use
- Provide easy scripting options for beginners
- Provide advanced functionality for experienced users
- Make setting up and running a test fast and easy
- Provide clear and understandable results.
Unfortunately, not all test tools can do all of this. Most are let down by poor analysis and reporting, among other things. However, there is a perfect way to satisfy all the above criteria.
My High-Level Recommendation
Based on years of experience in the performance testing field, my recommendation is that you choose two performance test tools to work together:
- A free, lightweight solution that your developers can use
- A comprehensive, cloud-based solution for your final end-to-end scenarios
But which tools should you choose, and which should you avoid?
Beware of People Pitching Open-Source Tools
A word of warning: when it comes to performance test tools, you’re going to run into very vocal groups who will try to sway your opinion towards imperfect solutions, because they’re free.
These are the same people we mentioned earlier, the consultancies and contractors, who like to maintain a shroud of mystery around everything they do. They’ll often push you onto open-source tools, so they can sell you more days, develop a bespoke solution, and keep you locked in.
If you’re looking for speed and efficiency, open-source tools are not the right option.
Open-source tools require far more effort for both creation and maintenance, and you also need to spin up your own performance testing environment. While the free aspect might sound great initially, the total cost of ownership can be quite scary.
While it might initially appear free to use, once you factor in the hidden costs (such as hosting and effort), it will quickly become more expensive than a paid tool.
Professional Tools Make More Sense
If you’re looking for a great test toolset, the LoadRunner suite of tools covers more protocols than any other solution. You can test everything from web to enterprise software, to app testing, and pretty much anything else you can think of.
- LoadRunner Developer is a free IDE-based solution that your developers can use
- LoadRunner Cloud is the class-leading cloud-based performance tool.
The LoadRunner tools are the most comprehensive performance tools on the market. They come out on top in head-to-heads with other tools.
If your performance testing partners are not suggesting them, maybe you should ask them why and what the difference in consultancy costs would be if they used LoadRunner rather than an open source solution.
