Products

Problems
we solve

We can help your business

Request a Free Demo / trial

Insights

Insights | From a different perspective | Insights Featured
12 February, 2024

Horizon: The Human Cost of Software  Failure

The Human Cost of Software Failure

The Horizon scandal at the UK Post Office is a sobering reminder of the potentially devastating human cost when technology fails and those in charge fail to act responsibly.

  • 3,500 sub-postmasters blamed for financial loss
  • 700 found guilty of crimes (93 convictions have been overturned so far)
  • 236 sent to prison

To summarise the scandal, the Post Office wrongly prosecuted people for stealing. This was due to a number of issues in a software solution called Horizon, which Fujitsu developed. Prosecutions in which Horizon evidence is thought to feature started in 1999 (the year the system went live) and continued until 2015. Many innocent people were sent to jail, and many more had their lives destroyed.

This insight looks at this dire affair from a testing perspective and serves as a cautionary tale for all involved in deploying large-scale IT systems.

What Happened to Their Software QA Processes?

At the heart of the scandal, there must have been some sort of QA failure during the development or commissioning of the Horizon system; it is abundantly clear that the system went live with critical defects.

Software QA processes are about more than just testing; they rely on stakeholders taking test results seriously and mitigating risks appropriately.

This begs the question: was there inadequate testing, or did Fujitsu and/or the Post Office ignore the results?

Sadly, neither would surprise me. We’ve all seen these scenarios play out in more projects than we can remember.

Please note: I do not mean inadequate testers; I mean that testers are often not allowed to perform adequate testing, with under-resourced teams being asked to do too much in too little time.

Were Test Results Ignored?

It’s uncomfortably common for project stakeholders to ride roughshod over the testing team when it comes to phase exit meetings. How often have you seen the leadership team go live with known critical defects despite the test team’s protestations?

Paul Patterson, Fujitsu’s European boss, knew of at least 29 bugs in the system over a 20-year years. Mr Patterson is on record that the Post Office knew about these issues and concealed the evidence in court cases against the sub-postmasters.

So, let’s unpack what he said:

  • 29 bugs? According to this article from Computer Weekly, a senior software developer who worked at Fujitsu alleged that the system should never have been launched and bosses at Fujitsu knew this. He alleged that everybody knew it had problems, stating, “It had gone through the test labs God knows how many times, and testers were raising bugs by the thousand.”
  • 20 years – we have all seen systems go live with bugs, but you find and fix them. 20 years though?! Then, you use the unreliable system to provide evidence to convict people. Why are we not hearing about people from Fujitsu and the Post Office facing criminal charges?

Did They Do Enough Testing?

It’s often standard practice for testing windows, and therefore coverage, to be reduced when things get tight – after all, the project must hit those arbitrary deadlines, even if the software is delivered late to test.

It goes without saying that the less testing you do, the higher the possibility of defects leaking through to live solutions. …but again, for those bugs to remain for over 20 years is outrageous.

The ITV drama Mr Bates vs The Post Office, broadcast in January 2024, gave one of the reasons why the problems occurred: data loss between the Post Office computer and the central systems.  Was this tested adequately? If so, did they simulate packet loss and unstable communications? I doubt we will ever know.

Was There Any User Acceptance Testing?

User Acceptance Testing (UAT) is a critical phase where potential end-users of the system test the software to ensure it meets their needs.

I have always believed that UAT is more about acceptance than testing, allowing users to walk through their processes and sign off on the solution. The real testing should have already been done.

The scandal has highlighted the possibility that the Post Office may have neglected this crucial step. If they did not involve real-world users, the Post Office ignored the standard protocols of software implementation and the voices of those who would be most affected by the system’s failures.

Catastrophic Communication Breakdown or Conspiracy?

What staggers me and many others about the whole debacle is the failure of the Post Office and Fujitsu to listen to warnings from users, insiders and the investigators they paid to look into this.

What could be more important than triaging and fixing the critical defects being repeatedly reported? Surely, there should be mechanisms to safeguard against such failures.

Whether this resulted from a culture of dismissing feedback or a more profound systemic issue remains a topic of debate.

What The Hell Was The Support Desk Doing?

It has been said that the support desk repeatedly told each sub-postmaster that they were the only ones experiencing these issues.

This clearly was not true, as the Post Office blamed 3,500 sub-postmasters for financial losses caused by Horizon.

With this many issues, it should have been easy to pinpoint the errors; the information should have been acted on, with the defects fixed and any testing coverage gaps plugged, with new tests added to the regression pack.

The Human Cost of Systemic Failure

Pinpointing the ultimate responsibility for this tragedy is beyond my remit, but there is no doubt that there was an inadequate process. This system was not ready for deployment.

However, for years, The Post Office senior management said there was not a problem, and Fujitsu remained quiet until called to account by a government Select Committee. I am sure you can draw your own conclusions.

You don’t need me to tell you that the consequences have been dire; lives and livelihoods have been destroyed.

  • 700 sub-postmasters were found guilty of crimes
  • 236 sub-postmasters were sent to prison
  • 4 sub-postmasters took their own life
  • 18 have died without seeing justice or full compensation

All because of a low-quality computer system. Just let that sink in for a minute.

Yes, the way the software issues were handled was atrocious, but I firmly believe that a rigorous QA process could and should have prevented this from happening in the first place and prevented the immense human cost.

Plus, let’s not forget that this continued for 20 years!

What Lessons Can We Learn?

The Criminal Cases Review Commission (CCRC ) said the Horizon scandal was “…the most widespread miscarriage of justice the CCRC has ever seen and represents the biggest single series of wrongful convictions in British legal history.”

The Horizon scandal underscores the importance of rigorous testing, effective communication, and genuine collaboration.

It is a stark reminder that accountability and transparency are not just corporate buzzwords but essential to public trust.

As our industry moves forward, we must continue to push for a culture that prioritises end-users and acknowledges the real-world implications of software quality and the critical importance of testing. I do think that, in general, we are heading in the right direction on this, but this just highlights that we are not there yet and have some way to go.

It’s not rocket science:

  • Give testing enough time
  • Test against real-world conditions
  • Take test results seriously
  • Do not go live with low-quality solutions
  • Communicate system issues with the end users
  • Do not cover up and do not lie.

Incidentally, if you haven’t already, I recommend watching the ITV drama ‘Mr Bates vs The Post Office’ and the associated documentary. They’re both available on ITVX in the UK. Be warned, the behaviour of The Post Office and Fujitsu and their treatment of the sub-postmasters is likely to make you very angry.

Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

12th February 2024
the next generation of test tools

The Evolution of Test Tools: Passing the Baton

Test tools have long been essential to software quality and testing efficiency, but over the years, they have transformed significantly, becoming more powerful, cost-effective, and reliable than ever before. One test tool suite has been at the forefront of this evolution, tracing its lineage back to the 1990s.

Service Virtualization

Seriously, Why Aren’t You Using Service Virtualization?

Service Virtualization (SV) remains one of the most criminally underused tools in software testing. It’s astonishing to me just how many organisations still ignore its benefits – test earlier and save money. If you’re not using SV, you are missing out—plain and simple.

UFT Digital Lab

UFT Digital Lab 24.2: What’s New?

If you need to test multiple devices, OS versions, or browser types, you need UFT Digital Lab from OpenText (previously known as Mobile Center and UFT Mobile). It has a comprehensive suite of features specifically designed for mobile, tablet and web testing across myriad platforms.

TruClient performance testing

TruClient: Quick, Easy & Powerful Performance Testing

TruClient significantly speeds up the script creation process and allows you to capture true end-to-end application performance, including protocol level and front-end performance, e.g. how long screens or pages load.

ALM QC 241

What’s New in ALM/QC 24.1: Release Highlights

If you’re involved with waterfall projects, you may well be familiar with the test management tool ALM/Quality Center (ALM/QC), but you might be surprised at its capabilities. The latest release, Version 24.1, is a modern test management powerhouse.

VSM Building Blocks

VSM is Easy with ValueEdge from OpenText

Value Stream Management (VSM) can revolutionise your business, driving customer value and removing unnecessary costs and investments. However, it does require a rethink of internal processes and culture, which can be a daunting prospect. Fortunately, ValueEdge from OpenText is an end-to-end suite of powerful, user-friendly, intuitive tools that make VSM easy!

Unlock Value with Software Testing

Unlock Value with VSM: The Crucial Role of Software Testing

Delivering high-quality solutions is an essential part of Value Stream Management (VSM). After all, substandard solutions don’t deliver value to customers or shareholders. That’s why software testing is a central component in VSM, and it’s now easier than ever with the powerful and user-friendly OpenText ValueEdge testing tools.

How to implement VSM

How to Implement Agile & Test Management Within VSM

ValueEdge Agile and ValueEdge Quality work together to enable smooth software development within VSM landscapes. By integrating the two tools, you increase the efficiency and effectiveness of Agile and DevOps testing processes, driving quality and value.

VSM

VSM: Reduce Project Costs The Right Way

It’s hard for businesses to understand where their investments are going and whether they will translate to business and customer value, especially when it comes to software development…. This is where VSM comes in.

AI the unseen software development partner

AI: The Unseen Partner in Modern Software Engineering

Within software projects, AI (and ML) is increasingly used in areas including code generation, test automation, and project management, although many of these uses are often not in an officially sanctioned capacity. Where ChatGPT is a bit of a Jack-of-all-trades, enterprise AI solutions are far more targeted, and far more robust.

Insights

Search

Related Articles

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.