(Bloomberg) -- Derivatives shops, used to clearing hundreds of billions of dollars in trades every day, found themselves in a dramatically different era this week: the old days of manually processing deals.
Most Read from Bloomberg
Merck Covid Drug Linked to New Virus Mutations, Study Says
Adani Crisis Deepens as Stock Rout Hits $108 Billion
Porsche Blunder Puts $148,000 Sportscar on Sale for Just $18,000
How Extreme Bets Fueled an $11.4 Billion Fortune
Hong Kong to Give Away 500,000 Air Tickets to Revive Tourism
Early Tuesday morning in Europe, a little known but critically important software company that underpins the smooth functioning of stock, bond and commodities markets started to seize up. London-based ION Trading UK had succumbed to a cyberattack.
Suddenly, in offices across the globe, traders and brokers turned to spreadsheets to keep track of their deals, firms resorted to inputting individual trades on websites provided by exchanges, and employees explained to their families why they were going into work at night, according to people with a view of the scene.
It was like being back in the 1980s, before electronic trading took off, or in the 1990s, when the web was just starting to change the world. But there was a key difference - the banks and brokers handling client trades on bourses including Intercontinental Exchange Inc., CME Group Inc. and Cboe Global Markets no longer have hordes of employees ensuring deals are confirmed, processed and settled.
"The cyberattack on ION reminds us all that despite best efforts by any organization to protect itself, these issues will occur, and market participants need to be continuously vigilant and prepared for such instances," said Joseph Schifano, head of regulatory affairs at Eventus, a trade surveillance software firm.
For the derivatives market, it was a slap in the face. Not only did companies lack adequate staff to meet the crisis, but many of the workers were too young to know how to keep operations afloat. It was also the second time in just one week that a major market had been humbled. A human error at the New York Stock Exchange set off violent price swings at the start of trading on Jan. 24.
Banks and other financial firms frequently label cyber risk as among those they fear most - as the interconnectedness of the financial system has the potential to amplify the ramifications from any attack. Both incidents also underscored how vital the plumbing underpinning trading processes can be, and that however sophisticated they may be, vulnerabilities lurk.
ION first noticed an issue was preventing access to some of its systems at 2:30 a.m. London time. It took the Dublin-based firm - founded by Italian tycoon Andrea Pignataro - more than five hours to confirm the attack by Russian ransomware gang LockBit, according to correspondence from ION seen by Bloomberg.
It wasn't long before the 42 ION clients affected started reporting difficulties. The US clearing arm of Dutch lender ABN Amro Bank NV sent out a note to clients saying the attack would delay overnight processing, and that it was being forced to deal with transactions manually. StoneX Financial said it was taking "alternative measures" to clear trades and prioritizing expiring contracts. Marex Group resorted to providing clients "indicative" values of transactions in their accounts.
On the London Metal Exchange - one of the last venues in the world where trading still takes place face to face - the return to manual processing was familiar for many veteran brokers, but it also provided an opportunity for younger staff to prove their technological prowess.
When ION's systems went down, a team of coders at one London brokerage scrambled to build their own ad-hoc system to match off clients' trades, and they had it up and running within hours, according to one person familiar with the matter.
But while those types of creative efforts have helped to mitigate the fallout so far, the challenges are growing as the crisis rolls on. Informally, the London brokerage has warned the LME that it expects dealers to reduce activity because of friction in processing trades, reducing liquidity, the person said.
Fear of contagion prompted the Futures Industry Association to hold over half a dozen calls over multiple days to give members a chance to talk through the situation and share relevant information. More than 600 people dialed in to one of these calls. Some were clients of ION, directly impacted by the attack. Others discussed potential ripple effects.
A spokesman for ION declined to comment on whether it had taken part in the FIA calls.
By the end of the day on Tuesday, neither the FIA nor the Commodity Futures Trading Commission - the top US derivatives regulator - disclosed or could confirm how many firms had been affected and how much money was locked up in trades handled by ION, said people who took part in the calls and asked not to be identified, citing confidentiality.
The software company never joined the discussion, the people said.
The outage, which is still ongoing, affected vital processes including the matching of trades, the calculation of margin calls and regulatory reporting on large market positions. That left many clients in the dark about whether they were making or losing money, and prompted calls for more collateral, the people said.
It was only then that customers found out there was a problem, with many more only discovering it when Bloomberg News reported the event on Wednesday morning, one of the people said.
On Wednesday, CME, Intercontinental Exchange and Cboe said that their members had experienced issues with a third-party software vendor. Those issues could affect the timing of publishing exchange reports by the end of the day, the firms said. The London Metal Exchange and Euronext also acknowledged that some of its clients had been affected.
"The LME has been closely monitoring liquidity across all venues since the incident occurred, and has not yet seen any evidence of liquidity being affected," the exchange said in an emailed statement. "We continue to work closely with affected members to help them continue their business as normally as possible, and reduce any wider impact."
The issue is "currently isolated to a small number of smaller and midsize firms, and does not pose a systemic risk to the financial sector," according to a statement from Todd Conklin, deputy assistant secretary of the US Treasury's Office of Cybersecurity and Critical Infrastructure Protection.
Regulators in the UK, including the Financial Conduct Authority, started an investigation into the incident, according to people familiar with the matter who asked not to be identified because the matter is private.
The Federal Bureau of Investigation is also seeking information on the cyberattack and reached out to ION executives, people familiar with the matter said. The agency is aware of the situation, it said in a statement.
On Thursday evening, the CFTC said the incident was impacting the ability of some clearing members to provide it with accurate data and that it would delay its weekly trading report for the derivatives market until all trades can be reported.
ION told clients on Thursday that its systems won't be fully operational until Feb. 5, and the firm still hasn't been able to start several crucial recovery steps, according to email correspondence seen by Bloomberg. The firm also told broker StoneX that it has brought in "multiple industry leading security firms to assist in their investigations and remediation plans," according to a copy of the memo sent to clients.
It's unclear if ION paid or plans to pay the ransom, and the industry is still just getting to grips with the ripple effects the incident may have. Beyond clients who are directly affected, banks and brokers that are trading with them aren't able to match off trades.
The result for now is that derivative shops are turning the clock back by years in an impromptu test of their middle and back offices.
--With assistance from Natasha Doff, Lydia Beyoud, Jenny Surane, Yvonne Yue Li, Marvin G. Perez and Jack Farchy.
(Updates with statement from CFTC in 24th paragraph.)
Most Read from Bloomberg Businessweek
A Billionaire's Son Battles a Turbulent WWE Over the Future of Pro Wrestling
A 3D Printer Isn't Cool. You Know What's Cool? A 3D-Printing Factory
Wall Street: We Want Lower Rates. The Fed: Not So Fast
Sean Penn's Disaster-Relief Charity Ended Up a Money Mess
How Stockpickers Finally Beat the Index Funds
©2023 Bloomberg L.P.