Visa hires external party to conduct review of latest system failure, replies to UK Treasury Committee

Maria Nikolova

EY will conduct a review of the root-cause analysis, major factors of the incident and the effectiveness of Visa’s disaster recovery.

Charlotte Hogg, Chief Executive Office (Europe) of Visa, has replied to questions posed by Rt Hon. Nicky Morgan MP, Chair of the Treasury Committee, regarding the system failure from June 1, 2018, which saw 5.2 million transactions fail to process correctly. Let’s recall that Nicky Morgan has asked Ms Hogg to provide an answer to 10 questions about the incident.

The answers, made public earlier today, provide some assessment of the disruption scale. Overall, for cards issued both in the UK and elsewhere, during the period of the service disruption, 51.2 million Visa transactions were initiated and sent to Visa’s European systems for processing. Of these, 5.2 million (10%) failed to process correctly. In the UK, during the period of disruption (about 10 hours), a total of 16 million cards were used and around 1.7 million were impacted (10.4%). During the incident window, there were 27.6 million Visa transactions initiated on UK-issued cards and sent to Visa for processing. Of these, 25.2 million transactions were processed properly (91 %) and 2.4 million (9%) failed to process.

Visa also elaborated on the “hardware failure” that allegedly cause the problems.

The incident was caused by the failure of a switch in one of Visa’s data centres, says Ms Hogg.

“We do not yet understand precisely why the switch failed at the time it did, but we are working with the manufacturer to conduct a forensic analysis of the switch”, Charlotte Hogg, Chief Executive Office (Europe) of Visa.

In particular, Visa processes payments in Europe in two data centres in the UK. The system works as an “active-active” service so both sites are continually processing transactions. The two sites are designed to continuously synchronise with one another, so that if one site encounters issues, transactions are routed to the other for processing.

In this instance, the switch being used in our primary data centre experienced a very rare, partial failure, which impacted the secondary site and prevented it from automatically processing all transactions, as it should have. As a result, it took far longer than expected to isolate the primary site; in the interim, the malfunctioning system at the primary data centre continued to try to synchronise messages with the secondary site. This created a backlog of messages at the secondary data centre, which, in turn, slowed down that site’s ability to process incoming transactions.

It took until approximately 19:10 to fully deactivate the system which was causing the transaction failures at the primary data centre. By that time, the secondary site began processing almost all transactions normally. The impact was largely resolved by 20:15, and we were processing at normal service levels in both data centres by Saturday morning at 00:45, and have been since that time.

Among the measures that Visa aims to implement after the incident is hiring an expert to conduct an external review. Visa has engaged an external party, EY, to conduct an independent review of agreed aspects of the incident. The Terms of Reference have been reviewed by our UK regulators. This will include: o Review of the root-cause analysis; o Evaluation of the leading factors in the incident; o Review of the effectiveness of Visa Europe’s disaster recovery, its response to the incident and its communications with stakeholders.

The Treasury Committee said it was satisfied with Visa’s answers regarding its system failure and it expects to see the findings of the independent review, which will examine the lessons to be learned from the incident, in full.

Read this next

Retail FX

Italian watchdog red flags Olympus Brokers, UnicoFX and Allfina Group

Italy’s Commissione Nazionale per le Società e la Borsa (CONSOB) has shut down new websites in an ongoing clampdown against firms it accuses of illegally promoting investment products in the country.

Retail FX

XTB revenues hits zł1.45 billion in 2022, Q4 earnings disappoint

Poland-based Forex and CFDs broker, XTB has reported its final results for Q4 of 2022 and the full fiscal year ending on December 31, 2022, showing one of its most successful corporate years.

Executive Moves

Lirunex Limited recruits Waleed Salah as head of MENA sales

Maldives-based brokerage firm Lirunex Limited has secured the services of Waleed Salah, who joined the company in the role of its head of sales for the MENA region.

Executive Moves

Trading 212 parts ways with co-founder Borislav Nedialkov

Trading 212 has a void to fill at its FCA-regulated business in London, following the departure of two key players, Raj Somal and Borislav Nedialkov.

Digital Assets

Binance acquires troubled crypto exchange GOPAX

Binance, the world’s largest digital asset trading platform, has reportedly acquired a majority stake in the troubled South Korea-based cryptocurrency exchange GOPAX.

Digital Assets

Kraken exits Middle East, closes UAE office

Digital currency exchange Kraken will close down its operations in Abu Dhabi, UAE and lay off the majority of its team focused on the Middle East and North Africa.

Industry News

CFTC comments on ION Cleared Derivatives issues after Russian-linked hack

“The ongoing issue is impacting some clearing members’ ability to provide the CFTC with timely and accurate data. As this incident unfolded, it became clear that the submission of data that is required by registrants will be delayed until the trading issues are resolved.”

Industry News

FCA took down 14 times more misleading ads in 2022 thanks to technology

The FCA has made significant improvements to the digital tools it uses to find problem firms and misleading adverts. These improvements have enabled it to work through a much larger number of cases compared with 2021.

Executive Moves

HKEX appoints ex-Goldman Sachs Matthew Cheong to lead platform’s focus on derivatives

“He has worked for a number of the world’s leading investment banks and his experience will be invaluable to HKEX as we continue to enhance our derivatives product offerings and build on our innovative and robust platform business, connecting capital with opportunities.”