“15 different types of spaghetti”

The most recent failure of UK bank RBS, and its group of companies, to process 600,000 payments, illustrates yet again the major problems underlying the IT systems of not just the banking sector, but any industry where there have been multiple, often rapid mergers and acquisitions.

One RBS executive reportedly said to the UK’s BBC that the bank’s systems we like “15 types of different spaghetti”. To my knowledge, and from my experience of the industry, that is a significant underestimate of the complexity.

RBS, before the financial crash of 2007/8, when it had to be bailed-out by the UK government to the tune of £46 billion, was for a short time the largest bank in the world. Its size came from a rapid period of acquisitions, most notably under the direction of CEO Fred Godwin, who became known as ‘Fred the shred’, reflecting the bank’s penchant for divesting itself of personnel in the newly acquired subsidiaries in order to boost profits, and thus the share price.

Sadly the rapidity of mergers and the loss of people who understood the IT systems of the acquired companies did not bode well for RBS trying to operate and report as a group.

In 2012, for several weeks, 6.5 million customers of RBS and its Nat West and Ulster Bank subsidiaries  could not use their online banking facilities, some could not make mortgage payments, and others outside the UK could not withdraw cash. This not only resulted in a loss of face and confidence, but a £56 million fine from the UK regulator.

The problem was diagnosed as software incompatibility which became apparent following a software upgrade, amplified by the fact that the bank didn’t have contingency processes in place to mitigate the problems.In December 2013 more than 1 million RBS, Nat West, and Ulster Bank customers found they could access their accounts after what appeared to be a similar failure, but with a different system.

Finally to June 2015, after the bank had reportedly spent the originally planned £2 billion on upgrading its systems, plus another £750 million, 6% of the transactions to be processed overnight failed because ‘the file could not be read properly’.

Having spent the majority of the last three years working around FINTECH, principally in the UK banking sector, I can only say I am not surprised.

Most of the major banking core back-end systems were written several decades ago, based around batch processing, are robust, well documented and supported. However, the ‘front-ends’ to those systems for on-line access, such as via the internet and latterly mobile devices, have not been built as well, and their desire for near real-time processing is fundamentally at odds with the back-end systems.Add to that RBS, like the majority of banks following mergers and acquisitions, is running multiple back and front-end systems, and has had to write ever more complex code to integrate those systems together, the numbers of different spaghetti types could be many multiples of 15.Sadly for us as bank customers RBS' recent predicament is just an example of what can (and will) go wrong.

My thoughts about RBS’ exposure to such problems are that during ‘rationalisation’ following its acquisitions, it lost much of the knowledge regarding the integration of systems within the IT systems of its subsidiaries. Furthermore integration between the systems of the business units was done tactically, and not as part of a planned strategic investment. Thus, when upgrading central and cross-business systems, not all can be tested properly before the upgrades are applied. This was compounded by a lack of mitigation plans, which could have led to a faster resolution of the resulting problems.

To be fair to RBS, it has put a ‘shed-load’ of somewhat belated investment into getting its systems together, and it put it's 'hands up' very quickly to the failure.  Another bank I worked with had a server it didn’t dare switch off, because at the time it did not know which other systems across the bank and its subsidiaries that the server’s information supported. Having looked at them all in a lot of detail, there are only two large banks I would feel confident with to handle my data, without a major mishap such as at RBS, and even those have some problems with their manual processes.

So what is the answer? Well thankfully it is already happening in the all banks reviewing/renewing their systems, and doing it in a planned and measured way. Not ‘on-the-fly’ to meet the next financial reporting date. Those in industries outside banking should not be complacent, the problems experienced by RBS are not sector specific. Need I remind my colleagues stuck in airport lounges of upgrade problems on the UK's air traffic control system, run by NATS?

Will we see a repeat of the repeat of RBS’ problems at it, or other banks, or in other industries? Yes, there are at least 350 different types of pasta in the world.