Accurate tool comparisons are crucial for organisations to make rapid, informed decisions. However, inaccurate tool comparisons are misleading, threaten to undermine the whole process, and may lead businesses to make expensive mistakes.
So naturally, I was disappointed to see the grossly misleading LoadRunner vs. BlazeMeter page on the BlazeMeter website—a somewhat shoddy attempt at misdirection and false equivalency.
Note that the page refers to LoadRunner as a specific product—which it isn’t. LoadRunner is a family of four products that do different but overlapping things.
While BlazeMeter doesn’t explicitly name LoadRunner Professional, the implications are clear. They are deliberately comparing their solution against the wrong version of LoadRunner.
By pitting its cloud-based solution against OpenText’s on-premises offering, BlazeMeter deliberately obscures its actual competition: LoadRunner Cloud.
It’s obvious to see why…
BlazeMeter and The Art of Misdirection
BlazeMeter knows very well that there are multiple versions of LoadRunner, each with unique features.
They also understand that people looking for performance testing might not know this and are trying to prey on their ignorance.
Omitting the version name is pure sleight of hand—BlazeMeter is dodging the proper comparison with the relevant OpenText tool, LoadRunner Cloud, which (as we’ll see later) yields very different results.
BlazeMeter is Comparing Apples to Oranges
By comparing their cloud-based solution to LoadRunner Professional, BlazeMeter is creating a false equivalency.
LoadRunner Professional is a class-leading on-premises solution and the appropriate tool for thousands of globally leading institutions. It is much better than BlazeMeter in terms of protocol support, complex testing scenarios, reporting capabilities, scripting flexibility, and reliability… but it isn’t a cloud-based performance tool.
I repeat: the appropriate comparison for BlazeMeter must be with LoadRunner Cloud, which offers:
- SaaS availability for immediate testing
- Cloud-based load generators
- Scalability to over 5 million virtual users
- 42+ geo-locations for testing
These features align closely with BlazeMeter’s offerings, allowing a valid comparison.
Fact-Checking BlazeMeter’s Traffic Lights
BlazeMeter’s comparison page employs this deceptive traffic light slider system:
This isn’t just visual trickery; it’s factually inaccurate, and these claims are easy to rebut:
- LoadRunner Cloud can be fully integrated into CI/CD Pipelines.
- You can start testing with LoadRunner Cloud in minutes.
- All LoadRunner products provide native support for open-source scripts (JMeter, Gatling, Selenium, etc).
- LoadRunner Cloud can be driven from a UI or an IDE (using the free LoadRunner Developer tool)
- LoadRunner Cloud scales to 5 million concurrent virtual users
- LoadRunner Cloud includes free IP whitelisting to allow testing behind a firewall
I can back up all of these bullets with product documentation and my personal experience and deep understanding of the LoadRunner Family of products.
It’s Easy to See Why BlazeMeter Were Hiding The Truth
When we look at more balanced comparisons, LoadRunner Cloud consistently outperforms BlazeMeter in all key areas:
- Analysis and Reporting: LoadRunner Cloud provides sophisticated and comprehensive reports, while BlazeMeter’s offerings are basic and require additional effort.
- Performance Testing Capability: LoadRunner Cloud demonstrates superior scalability and can run more extender tests in parallel.
- Additional Features: LoadRunner Cloud offers more robust service virtualisation and integrations.
But you know what, I’m not angry at BlazeMeter’s lies or their poorly executed legerdemain; I’m just disappointed.
The Importance of Integrity in Comparisons
BlazeMeter’s approach does a disservice to the performance-testing community.
By deliberately comparing their product to a less equivalent alternative, they’re misleading potential customers and undermining their integrity.
Organisations deserve transparent, apples-to-apples comparisons, allowing them to make informed decisions based on their needs.
While BlazeMeter’s sales tactics may be effective for their short-term revenue, they ultimately erode trust in their organisation.
