Products

Problems
we solve

We can help your business

Request a Free Demo / trial

Insights

Insights | From a different perspective | Insights Featured
10 October, 2024

Should We Expect Developers to Test?

should developers test

In my latest Testing Times newsletter, I discussed Microsoft and their poor software quality. This led to a couple of conversations on LinkedIn comments concerning Microsoft’s approach to testing… and whether developers should ‘test their own homework’ as the saying goes. Click the newsletter link to read the comments in full.

These conversations prompted me to examine Microsoft’s approach to quality more closely. Among the many sources I read, Gergely Orosz’s article How Microsoft does Quality Assurance (QA) stood out.

My Personal Experience

In all candour, I find it hard to be entirely neutral on this topic. I have been involved in projects where developers have been asked to test, and this firsthand experience has strongly influenced my views.

On those projects, when developers tested, they almost exclusively focused on what the solution should do. They wanted to validate that a system conformed to the spec or user story.

The dedicated testers, on the other hand, in addition to the happy path also looked for edge cases and weird behaviour. They sought to understand and imagine all the weird and wonderful ways a user might interact with the solution and find areas where the system didn’t work as expected.

But that was just my experience…

So What Does Happen At Microsoft?

In his article, Gergely explains how Microsoft drove efficiencies by switching from separate Software Development Engineer (SDE) and Software Development Engineer in Test (SDET) roles to a combined Software Engineer (SE) role.

However, there was an interesting quote from Brian Harry, Technical Fellow at Microsoft: “We combined the dev and test orgs into a consolidated ‘engineering’ org. For the most part, we eliminated the distinction between people who code and people who test. That’s not to say every person does an identical amount of each, but every person does some of everything and is accountable for the quality of what they produce.”

Now, when something intrigues me, I like to investigate in detail … and when I apply this approach to Brian’s statement, it suggests that there is still a bit of an SDE/SDET split.

With that in mind, I was still left asking: Is it really possible to completely merge the roles of developers and testers? Just because they have the same name doesn’t mean they’re all doing the same thing.

Also, in the article, Gergely mentions “We became a lot more productive by removing the SDET role from our team! SDETs still focused mainly on testing-related work, but also picked up development tasks.”

So, did they remove the SDET role or just the label? Of course, I don’t know; I wasn’t there, but it doesn’t sound like they completely removed the separation. And then what happened in the medium to long term? Did they recruit or train specific testing skills fo pick up the SDET-type tasks?

There’s A Clear Conflict of Interest

Look, I get that this practice is seemingly efficient and sounds impressive, but I still have concerns about objectivity and thoroughness in the quality process.

I can’t ignore the fact that asking development teams to test presents a fundamental conflict of interest, even when they work in pairs. After all, mutual back-scratching is the basis of many working relationships.

I’m not saying that developers will consciously pass defective software; instead, human nature will play its natural part, and our evolutionary history is difficult to counter.

Plus, developers and testers necessarily have different mindsets: creative versus thorough.

In a recent article, I discussed how software testers’ brains play tricks on them, and this is even more likely when developers are asked to fulfil that role.

Devs are more likely to unconsciously overlook errors or omissions in their own work, having confidence in their coding ability. This confidence can lead to confirmation bias, where they seek confirmation of their assumptions and potentially miss critical flaws.

Moreover, developer testing lacks the fresh perspective that independent testers can provide, which is crucial for identifying issues that someone too close to the project might miss.

The Case for Independent Testing

I have over three decades of experience in software development and QA, and I firmly believe that truly independent testing, carried out by trained, process-driven, pedantic test professionals, is critical to high-quality software. Attention to detail and a hunger for destruction are also very beneficial.

I’m joking, of course, well, half joking. But sometimes, having niggly testers who really get into a system is key to mitigating business risks and producing high-quality software.

First and foremost, independent testers approach the software without preconceptions or emotional investment, allowing for a more objective assessment.

They also bring diverse perspectives and methodologies, increasing the likelihood of uncovering a more comprehensive range of issues.

As I said in the previous Testing Times, I have to deal with many bugs in Microsoft products. Almost universally, these are bugs that would not appear in the happy path.

Sometimes, You Need to Find a Middle Ground

I appreciate that complete separation of development and testing isn’t necessarily desirable, as they need a close working relationship built on trust and mutual respect.

As we’ve seen with Microsoft, there can be tempting business reasons to integrate the roles, such as:

  • Shifting to Agile or DevOps
  • Aiming to increase velocity with earlier testing
  • Cutting costs by reducing headcount and/or overheads

If, for whatever reasons developers are going to test, you can actively lessen the risks associated with self-testing by adopting the following:

  • Implementing a peer review system where testers evaluate each other’s work can help introduce an additional layer of scrutiny.
  • Regularly rotating testing responsibilities among team members can also provide fresh insights into the software.
  • Involving end-users or stakeholders in user acceptance testing can provide invaluable feedback from those who will ultimately use the product—although be prepared—they might not be happy with the additional effort and responsibility.
  • Last but not least… use automated testing!

Reading between the lines of Gergely’s blog, this seems to be Microsoft’s approach, but again, I wasn’t there so I can’t say for sure.

Test Automation Is Priceless If You Don’t Have Dedicated Testers

As mentioned above, test automation is crucial if you expect developers to perform your testing. It helps mitigate risks and introduces efficiencies and time-savings into the process. Of course, automation should be used even when you have dedicated testers; the benefits are too substantial to ignore.

While automated tests do not completely eliminate bias, they objectively measure whether the code meets specified requirements, counteracting some subjectivity inherent in self-testing.

Automated tests ensure consistent execution, reducing human error and oversight. They also reduce the burden on developers, allowing them to focus on what they are good at.

Obviously, test automation has intrinsic benefits, such as rapid execution and feedback. This immediate insight enables them to catch issues early in the development process, reducing the cost of fixing bugs later.

Where they are used, developers can integrate automated tests into their continuous integration and delivery (CI/CD) pipelines, ensuring that code changes are verified before deployment and maintaining high quality in production environments.

Once established, automated tests require minimal effort to run repeatedly, allowing developers to focus on writing new features. As software complexity grows, manual testing becomes less feasible; test automation scales effectively, maintaining thorough testing practices as the codebase evolves.

While not a complete substitute for dedicated testers, strong test automation practices can help development teams uphold software quality when independent testing isn’t an option.

So, Should We Expect Developers to Test?

Clearly Microsoft are no mugs, and software development practices continuously evolve, but I am just not convinced that merging test and dev roles is practical or even realistic.

The simple fact is that some people will naturally gravitate to one role more than the other; they both require fundamentally different mindsets and perspectives.

Both Gergely Orosz and Brian Harry admitted, or at least strongly hinted at a separation between the day-to-day activities, even after combining the SDE and SDET roles.

I believe it’s better to have clarity and a clear distinction and that organisations should use independent testing where possible.

Train your testers to be exceptional testers, train your devs to be great at that, and train them all how to best work together to drive efficiencies rather than merging them all and risking losing that gorgeously inquisitive tester mindset.

Where this is not possible, they must implement best practices to ensure thorough and unbiased quality assurance, including test automation with tools like UFT One—did you know it covers more applications than any other automation tool?

Drop me a note if you want a UFT One demo or a quote.

Ultimately, a rigorous quality assurance process is essential for delivering high-quality software that meets user needs and expectations. Achieving this requires skills, mindset, perspective, and independence that I believe only dedicated testers can provide.

Do you agree, or am I completely wrong on this? I’d love to hear your thoughts and continue the conversation.

Related Products

UFT One
Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

10th October 2024
LoadRunner Enterprise 243

LoadRunner Enterprise 24.3 – Release Highlights

Today we look at OpenText LoadRunner Enterprise 24.3, which introduced several enhancements to their enterprise-grade performance testing solution. We’ve been through the release notes so you don’t have to.

OpenText Customer? Read This to Avoid Overpaying for Support Costs

Following its acquisition of Micro Focus, OpenText has implemented a more stringent policy regarding support renewals and term/subscription renewals. This policy could increase your support costs. Today I’ll explain how to minimise costs when renewing your OpenText software support or subscriptions.

AI caution

WQR: This 1 AI Recommendation Could Derail Your QA Strategy

The World Quality Report 2024-25 (WQR) provides a few interesting insights into adopting AI in software testing. And, while many their recommendations are sound for most companies, one specific recommendation regarding AI implementation is a terrible idea for most businesses.

LoadRunner Cloud 244

LoadRunner Cloud 24.4 – 3 Release Highlights

OpenText has released LoadRunner Cloud 24.4, The latest version of their class-leading SaaS performance test tool. We’ve been through the release notes and selected 3 things you need to know about this and other recent LoadRunner releases.

BlazeMeter Are Lying About LoadRunner

BlazeMeter is Lying About LoadRunner

The people at BlazeMeter are comparing their solution against the wrong version of LoadRunner. This is a deliberate attempt to obscure their actual competition: LoadRunner Cloud.

Microsoft issues

When Will Microsoft Step Up? A Customer’s Plea for Improvement

Microsoft (MS) has dominated the software landscape for decades, boasting vast resources and some of the brightest engineering minds. Yet, despite this wealth of talent, many users struggle with persistent issues that raise questions about the company’s commitment to quality and customer satisfaction.

Insights

Search

Related Products

UFT One

Related Articles

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.