In a regulated industry like finance, plausible-sounding but incorrect outputs — known as hallucinations — pose significant dangers. Unlike a simple lack of response, these errors can spread quickly, be trusted more readily, and leave weaker audit trails. Hallucinations are not just technical quirks; they represent a fundamental control problem. They raise critical questions about the evidence systems are allowed to use, how they are constrained, how they are monitored, and who is accountable when they fail.

The Risks of Ungoverned GenAI

Deploying GenAI without strong controls can lead to serious business liabilities, including:

• Fabricated data: Incorrect financial figures, market commentary, or pricing could enter client communications or internal decision-making.
• Biased guidance: Unsuitable investment or product recommendations could trigger conduct risks and complaints.
• Regulatory breaches: Unapproved forward-looking statements or incorrect interpretations of regulations could lead to compliance failures.
• Audit failures: Missing provenance for outputs (e.g., “What sources were used?” or “Who approved this?”) could undermine audits.
• Operational errors: Misrouted complaints, inaccurate customer letters, or erroneous Suspicious Activity Report (SAR) drafting could disrupt workflows.
• Reputational damage: Clients identifying nonsensical AI-generated outputs could erode trust.
• Legal exposure: Treating GenAI outputs as advice, assurance, or commitments could lead to legal challenges.

Why Hallucinations Are a Bigger Problem in Finance

The financial services sector has a uniquely high-risk profile when it comes to GenAI:

• Materiality: A single incorrect figure or assumption can significantly alter decisions.
• Propagation: GenAI outputs can easily be copied into critical documents like board packs, client communications, or regulatory submissions.
• Asymmetry of harm: While the upside of a slightly better summary is modest, the downside of a confident error can be catastrophic.
• Explainability burden: In finance, “The model said so” is not an acceptable justification for decisions affecting investments, credit, compliance, or customer outcomes.

GenAI’s design compounds these risks. It is built to be helpful and fluent, often filling gaps with outputs that resemble expertise, even when it lacks the necessary knowledge.

Mitigating GenAI Risks

Leading financial institutions are taking proactive steps to reduce the risks associated with GenAI hallucinations and ensure compliance:

• Grounding outputs in trusted data: Using domain-tuned models and retrieval-augmented generation (RAG) techniques that cite approved, up-to-date sources.
• Human review for high-risk use cases: Ensuring that outputs related to advice, regulatory language, or sensitive decisions are carefully reviewed by experts.
• Guardrails and safe failure design: Implementing systems that de-escalate rather than guess when uncertain.
• Continuous monitoring: Running programmes that include logging, red-teaming, drift detection, and incident playbooks to manage model behaviour over time.

In practice, GenAI is safest when used for evidence-based tasks like summarisation and drafting. Use cases involving advice or regulatory language require much stricter controls or should be avoided altogether.

The Shift in Operating Models

To use GenAI effectively, financial institutions must treat these systems as dynamic collaborators rather than deterministic engines. This requires embedding control frameworks into the design and operation of GenAI systems from the outset.

The firms that succeed will not necessarily be those with the flashiest chatbots. Instead, they will be the ones that can demonstrate to regulators, auditors, and customers that their GenAI outputs are grounded, traceable, and governed, with clear human ownership at every critical point.

2026: A Turning Point for AI in Finance

As GenAI tools become widely available and foundational capabilities are established, the competitive edge in financial services will shift. Success will no longer depend on who has the most AI but on who uses it best. The next 12–24 months will require sharper focus and strategic discipline. Institutions must move beyond experimentation and prioritise use cases that deliver measurable business value, operational resilience, or compliance improvements.

Where to Double Down

• AI in controls and compliance: Automating documentation, stress-testing, and reporting functions to reduce costs and regulatory risks.
• Human-centric AI adoption: Investing in design, training, and explainability to drive cultural change and responsible use.
• Value-led prioritisation: Using cross-functional scoring frameworks to identify high-impact, scalable use cases.
• AI observability and monitoring tools: Ensuring traceability, performance tracking, and real-time governance of AI models.

Where to Pause or Proceed Cautiously

• Uncontrolled GenAI in client-facing roles: Without grounding mechanisms and human oversight, the reputational risks outweigh the benefits of automation.
• Siloed, small-scale pilots: Proofs-of-concept that don’t align with broader business processes are unlikely to scale or deliver ROI.
• Vendor sprawl: Managing too many AI tools without integration can increase technical debt and obscure governance visibility.

Conclusion

Generative AI has immense potential to transform financial services, but it also introduces significant risks. To harness its benefits while mitigating its hazards, financial institutions must adopt a disciplined, strategic approach. This means grounding outputs in trusted data, embedding robust controls, and treating GenAI as a dynamic collaborator with clear human oversight.

The firms that succeed will be those that move beyond experimentation, focus on high-value use cases, and build systems that are not only powerful but also trustworthy, explainable, and compliant.

To explore these strategies in greater depth and learn how to navigate the challenges of GenAI in finance, read the full report.