The problem with using AI to review financial promotions

Alex is a senior marketing manager at a Building Society. Before sending a retail promotion to compliance for sign-off, they try something that makes absolute sense in principle: they download the FCA Handbook modules, upload them into Microsoft Copilot along with the assets to review and ask the machine to check the promotion against the relevant requirements.

The output looks credible. It references specific rules, flags a couple of issues. Alex makes the changes and sends the draft across to compliance for final sign-off, feeling confident everything has been caught. 

From the outside, this sounds like a reasonable pre-check that should save everyone time, but how can you be sure it holds up under scrutiny?

The first challenge is consistency

Ask any standard off-the-shelf generative AI model to assess the same financial promotion twice and you are likely to get get two different answers.

This is because large language models are probabilistic by design. They produce statistically likely outputs against each prompt, which means outputs that vary each time you prompt them. 

Run the same check across the same document on ten separate occasions and you will get ten slightly different assessments, different flags, different severity, different wording around the issues. 

In a regulated context, that creates an immediate problem. 

A rigorous compliance check needs to produce the same result each time, against a fixed set of rules. If the output changes depending on statistical variables, when you run it or how the session is set up, you can not be confident in the results. 

To bring this to life in context, COBS 4.5 requires risk warnings in retail investment promotions to be prominent. Ask a generative AI whether a risk warning is prominent enough and the answer will depend on how the question is phrased, what else is in the session and the model’s statistical weighting at that moment. 

AI can have memory issues 

When Alex uploads the handbook, it goes into the model’s context window (how much the machine can remember) for that session. The context window is temporary. It does not carry forward between sessions. The next time Alex runs a check, the model has no memory of the previous assessment, no awareness of what was flagged before and no accumulated understanding of how Alex’s firm applies those rules. 

Within a single session, the model’s ability to apply a lengthy regulatory document consistently also degrades as the conversation grows. The FCA handbook is certainly not a short document and there is no guarantee the model has processed all of it correctly. 

Uploading a document gives the model something to refer to, but doesn’t mean it will enforce it the right way. When the model reads the handbook and assesses a promotion, it is making interpretive judgements – i.e. about which sections apply, how heavily to weight them and what they mean for this specific piece of content. Those judgments will not be the same twice. A compliance check cannot function on that basis.

The hallucination problem

We have all been given an incorrect answer by a very overconfident AI chatbot at this point.

In most contexts, a hallucination is an inconvenience, a fact that needs to be checked. In a regulated compliance workflow, hallucination is a huge liability and risk. A model that misattributes a rule, invents a requirement that does not exist or fails to flag something it should have caught is producing a false sense of security.

No AI system eliminates hallucination entirely. What matters is whether the tool has been purposefully designed to minimise it, is tested against known outputs and keeps a human in the loop for final sign-off decisions. A general-purpose AI model has none of those built in.

FCA rules around financial promotions have not changed. What has changed is how firms are expected to demonstrate compliance. Under SM&CR, responsibility sits with named individuals. Under Consumer Duty, firms must show they have taken appropriate action to achieve fair outcomes for retail customers.

“It was AI-generated” is not a sufficient defence. 

That is the principle the FCA applies when firms attempt to share accountability with a tool. 

The regulator expects firms to know what checks were applied, who made the sign-off decision and what the evidence looks like. It is hard to evidence that if you are using a general AI model on a session by session basis. 

For a full account of what the FCA expects from firms using AI in financial promotions workflows, and a practical governance framework you can apply now, read our guide: Governing AI in Financial Promotions.

Where is the audit trail?

When an auditor reviews a financial promotion, they need the full approval history: what was assessed, when, what was found and who made the decision to proceed. A general AI output copied into an email or pasted into a shared document gives them a text file. No structure, no chain of custody, no connection to the asset it was meant to assess.

The check and the asset need to be linked. Without that link, the check does not form part of the compliance record. From a regulatory perspective, a check you cannot retrieve and demonstrate may as well not have happened.

How we embedded AI in SeeDynamic

SeeDynamic was built from the ground up for FCA-regulated marketing and compliance teams, helping automate compliance checks, reduce manual work and improve right first time benchmarks. 

The foundation is rules-based. SeeDynamic applies predefined, configurable checks against regulatory and internal standards: barred words, readability thresholds, required risk warnings, missing disclosures. 

AI is layered on top of this to extend what basic rules can do. It helps flag inconsistencies, ambiguous language, undefined terminology and issues that simple keyword matching would miss. It can also apply the right checks automatically based on the context of the asset, the product or service being promoted.

For example, where a rules-based check flags a missing risk warning, the AI layer can assess whether similar language elsewhere in the document creates ambiguity about whether the disclosure requirement has actually been met. That is a more nuanced finding than a keyword match.

Teams can also write new rules in plain English, which means you can configure and update the system to check different things without needing technical support each time requirements change.

We spent several months working out how to embed AI in a way that holds up under scrutiny. SeeDynamic builds on top of existing AI models, but our approach differs from off-the-shelf tools: we select the best models for each task, control the parameters and test every new model version before it goes into production. To address the consistency problem directly, SeeDynamic’s AI operates at temperature zero, reducing the probabilistic variation that makes general-purpose models unreliable for compliance use. 

Hallucinations cannot be eliminated entirely from any AI system, but our rigorous testing process, whereby AI enabled rules are repeatedly tested for consistency, reduces the risk before AI-enabled rules are introduced to the platform. Our rules-based foundation also means the AI operates within defined parameters rather than interpreting requirements from scratch.

The structured compliance report SeeDynamic produces is attached to the asset throughout the approval workflow, forming part of the permanent record from the first pre-check through to final sign-off. When an auditor asks, the team can show everything, including the AI’s decisions, in sequence, with no gaps.

Speed matters. Getting to sign-off faster matters. But a faster AI-enabled check that cannot be evidenced or defended does not solve the problem.

The right tool for the right part of the workflow

Generative AI is changing how most marketing teams work. In research, briefing and drafting, it adds real value. 

Teams should not be afraid to experiment with it.

But experimentation works when you are clear about what each tool can and cannot do. For compliance checking in a regulated environment, that means using a system that stands up to scrutiny.

A general-purpose model given a regulatory handbook for the duration of a session does not meet that bar. SeeDynamic does. 

Sign up for a free trial today and test it yourself.