Sophisticated hackers may exploit flaws in AI systems or use “adversarial” inputs to dupe them. For example, machine learning models can be tricked by manipulated data – an attacker might subtly alter transaction information to fool a bank’s fraud detection AI into approving fraudulent transfers ​(Zenus). There is also the risk of “data poisoning,” where criminals compromise the training data used by an asset management firm’s trading algorithm or a bank’s credit model, skewing its decisions. 

Another growing concern is the use of model inversion and membership inference attacks, where adversaries can extract sensitive data from trained AI models. In the financial sector, this could mean that attackers who gain access to a model, such as a robo-advisor trained on high-net-worth client portfolios, could reverse-engineer personal financial data or investment strategies. A 2017 study published in the Proceedings of the IEEE Symposium on Security and Privacy demonstrated how attackers could infer whether specific individuals were part of a model’s training set, raising serious privacy concerns for banks and asset managers using AI for personalized services. In an era of increasingly stringent data protection regulations, such vulnerabilities not only pose technical risks but also legal and reputational ones (NIST).

The SolarWinds breach in 2020 underscored how AI itself can become collateral damage in broader supply chain attacks. In that case, attackers infiltrated major IT and cloud infrastructure providers, many of which supported AI-powered analytics and decision-making systems used by financial institutions. If AI models depend on cloud services compromised by attackers, their integrity—and the decisions they inform—can no longer be trusted. Similarly, researchers at HiddenLayer revealed that AI models stored in cloud environments could be tampered with directly, enabling attackers to implant subtle changes in weights or logic without altering the underlying application code. For financial firms increasingly deploying AI via APIs or third-party platforms, this opens up a new cybersecurity frontier where the model itself is the attack vector (HiddenLayer).

Generative AI tools might be used for large-scale market manipulation or to provoke algorithmic trading cascades. In a forward-looking scenario, autonomous AI agents could conceivably coordinate trades or rumors that lead to flash crashes or bank runs via herd-like behavior ​(Roosevelt Institute). Financial regulators are acutely aware of these dangers. The White House’s 2023 executive order on AI specifically called out financial-sector AI cybersecurity, directing the U.S. Treasury to issue best practices for managing AI-related cyber risks ​(Skadden)

In the EU, the new Digital Operational Resilience Act (DORA) will require banks and firms to report serious ICT (information and communications technology) incidents – which would include AI system hacks or failures – to regulators ​(Skadden). Industry-wide, there is a growing emphasis on “red-teaming” AI models (stress-testing them for vulnerabilities) and on sharing threat intelligence about AI-driven attacks.

Imagine a future where loan approvals, stock trades, insurance claims, and payment flows are all decided by interlinked algorithms: a glitch or bias in one AI system could cascade through the financial network.