The Horizon scandal at the UK Post Office is a sobering reminder of the potentially devastating human cost when technology fails and those in charge fail to act responsibly.
- 3,500 sub-postmasters blamed for financial loss
- 700 found guilty of crimes (93 convictions have been overturned so far)
- 236 sent to prison
To summarise the scandal, the Post Office wrongly prosecuted people for stealing. This was due to a number of issues in a software solution called Horizon, which Fujitsu developed. Prosecutions in which Horizon evidence is thought to feature started in 1999 (the year the system went live) and continued until 2015. Many innocent people were sent to jail, and many more had their lives destroyed.
This insight looks at this dire affair from a testing perspective and serves as a cautionary tale for all involved in deploying large-scale IT systems.
What Happened to Their Software QA Processes?
At the heart of the scandal, there must have been some sort of QA failure during the development or commissioning of the Horizon system; it is abundantly clear that the system went live with critical defects.
Software QA processes are about more than just testing; they rely on stakeholders taking test results seriously and mitigating risks appropriately.
This begs the question: was there inadequate testing, or did Fujitsu and/or the Post Office ignore the results?
Sadly, neither would surprise me. We’ve all seen these scenarios play out in more projects than we can remember.
Please note: I do not mean inadequate testers; I mean that testers are often not allowed to perform adequate testing, with under-resourced teams being asked to do too much in too little time.
Were Test Results Ignored?
It’s uncomfortably common for project stakeholders to ride roughshod over the testing team when it comes to phase exit meetings. How often have you seen the leadership team go live with known critical defects despite the test team’s protestations?
Paul Patterson, Fujitsu’s European boss, knew of at least 29 bugs in the system over a 20-year years. Mr Patterson is on record that the Post Office knew about these issues and concealed the evidence in court cases against the sub-postmasters.
So, let’s unpack what he said:
- 29 bugs? According to this article from Computer Weekly, a senior software developer who worked at Fujitsu alleged that the system should never have been launched and bosses at Fujitsu knew this. He alleged that everybody knew it had problems, stating, “It had gone through the test labs God knows how many times, and testers were raising bugs by the thousand.”
- 20 years – we have all seen systems go live with bugs, but you find and fix them. 20 years though?! Then, you use the unreliable system to provide evidence to convict people. Why are we not hearing about people from Fujitsu and the Post Office facing criminal charges?
Did They Do Enough Testing?
It’s often standard practice for testing windows, and therefore coverage, to be reduced when things get tight – after all, the project must hit those arbitrary deadlines, even if the software is delivered late to test.
It goes without saying that the less testing you do, the higher the possibility of defects leaking through to live solutions. …but again, for those bugs to remain for over 20 years is outrageous.
The ITV drama Mr Bates vs The Post Office, broadcast in January 2024, gave one of the reasons why the problems occurred: data loss between the Post Office computer and the central systems. Was this tested adequately? If so, did they simulate packet loss and unstable communications? I doubt we will ever know.
Was There Any User Acceptance Testing?
User Acceptance Testing (UAT) is a critical phase where potential end-users of the system test the software to ensure it meets their needs.
I have always believed that UAT is more about acceptance than testing, allowing users to walk through their processes and sign off on the solution. The real testing should have already been done.
The scandal has highlighted the possibility that the Post Office may have neglected this crucial step. If they did not involve real-world users, the Post Office ignored the standard protocols of software implementation and the voices of those who would be most affected by the system’s failures.
Catastrophic Communication Breakdown or Conspiracy?
What staggers me and many others about the whole debacle is the failure of the Post Office and Fujitsu to listen to warnings from users, insiders and the investigators they paid to look into this.
What could be more important than triaging and fixing the critical defects being repeatedly reported? Surely, there should be mechanisms to safeguard against such failures.
Whether this resulted from a culture of dismissing feedback or a more profound systemic issue remains a topic of debate.
What The Hell Was The Support Desk Doing?
It has been said that the support desk repeatedly told each sub-postmaster that they were the only ones experiencing these issues.
This clearly was not true, as the Post Office blamed 3,500 sub-postmasters for financial losses caused by Horizon.
With this many issues, it should have been easy to pinpoint the errors; the information should have been acted on, with the defects fixed and any testing coverage gaps plugged, with new tests added to the regression pack.
The Human Cost of Systemic Failure
Pinpointing the ultimate responsibility for this tragedy is beyond my remit, but there is no doubt that there was an inadequate process. This system was not ready for deployment.
However, for years, The Post Office senior management said there was not a problem, and Fujitsu remained quiet until called to account by a government Select Committee. I am sure you can draw your own conclusions.
You don’t need me to tell you that the consequences have been dire; lives and livelihoods have been destroyed.
- 700 sub-postmasters were found guilty of crimes
- 236 sub-postmasters were sent to prison
- 4 sub-postmasters took their own life
- 18 have died without seeing justice or full compensation
All because of a low-quality computer system. Just let that sink in for a minute.
Yes, the way the software issues were handled was atrocious, but I firmly believe that a rigorous QA process could and should have prevented this from happening in the first place and prevented the immense human cost.
Plus, let’s not forget that this continued for 20 years!
What Lessons Can We Learn?
The Criminal Cases Review Commission (CCRC ) said the Horizon scandal was “…the most widespread miscarriage of justice the CCRC has ever seen and represents the biggest single series of wrongful convictions in British legal history.”
The Horizon scandal underscores the importance of rigorous testing, effective communication, and genuine collaboration.
It is a stark reminder that accountability and transparency are not just corporate buzzwords but essential to public trust.
As our industry moves forward, we must continue to push for a culture that prioritises end-users and acknowledges the real-world implications of software quality and the critical importance of testing. I do think that, in general, we are heading in the right direction on this, but this just highlights that we are not there yet and have some way to go.
It’s not rocket science:
- Give testing enough time
- Test against real-world conditions
- Take test results seriously
- Do not go live with low-quality solutions
- Communicate system issues with the end users
- Do not cover up and do not lie.
Incidentally, if you haven’t already, I recommend watching the ITV drama ‘Mr Bates vs The Post Office’ and the associated documentary. They’re both available on ITVX in the UK. Be warned, the behaviour of The Post Office and Fujitsu and their treatment of the sub-postmasters is likely to make you very angry.
