AI in Financial Services: Turning FCA Good Practice into Working Controls

This week at Sedona, we are reviewing the FCA’s publication from 19 October 2026, AI in financial services: good practice and areas for improvement. In this article, we look at some of the key information, what it means to our customers, what they are doing and how we are helping them follow FCA advice.

The publication does not introduce a new AI rulebook. Instead, it uses supervisory observations, industry engagement and work from the FCA AI Lab, AI Live Testing and the Supercharged Sandbox to show how existing expectations apply to machine learning, generative AI and agentic AI. The examples are illustrative, but they give insurers and other financial services firms a much clearer view of what the FCA considers effective in practice.

Let us first consider three clear takeaways from the publication, to set the scene.

Three takeaways from the FCA publication

• The guidance marks an interesting shift in the AI governance conversation. The leadership question is increasingly less “Do we have an AI policy?” and more “Can we demonstrate that our controls actually work?” Through some really tangible examples, the FCA has put significant emphasis on real-world consumer outcomes, clear accountability, meaningful human oversight, continuous monitoring, audit trails and operational resilience. Importantly, that extends beyond traditional models into GenAI, third-party models and increasingly agentic AI. For insurers, this starts to turn AI governance from a predominantly policy question into an engineering and assurance challenge.
  
  

• AI governance is moving out of PowerPoint and into production. The good practice message on governance is clear: firms need more than policies and approval gates. They need to understand how AI behaves once deployed, monitor outcomes, maintain audit trails, establish meaningful human oversight and know when to intervene. For insurers adopting GenAI and agentic AI, that is an important shift. Governance increasingly needs to be engineered into the solution itself.
  
  

• Prove your AI governance actually works. Beyond policies and principles, firms need continuous monitoring, defined thresholds, AI interaction logs, audit trails, structured evaluations, human overrides and escalation, model and provider dependency management, and evidence of senior-management oversight. AI governance is something firms need to test, observe and evidence.

For some months now, new customers have been asking us to help lead, design and implement their people and process governance frameworks before starting AI engineering work. But often they have not engaged enough leadership to collaborate on and approve the approach, so things can stall. Operating outside interactions with Sedona, their own risk functions, plus the likes of Deloitte, PwC, EY and KPMG, are happily living in this operational governance space, albeit sometimes disconnected from the practical, engineering-led concepts above.

To make faster progress, financial-services organisations may need to focus on the question: “How can we quickly operationalise our AI governance controls and produce the evidence that shows they are working?” Delivering high-value work in this area has been a large part of our core business for years. Let us break it down and consider five practical approaches that make a large contribution to following the FCA’s advice.

1. AI Governance Control Implementation

Risk and compliance teams may already have defined a policy. Alternatively, a policy can be assembled from the FCA publication combined with vendor experience. Either way, it then needs turning into working controls, and Sedona can help here.

For example, regulators and insurers may agree that AI-supported claims decisions must be subject to human review where confidence is below an agreed threshold. Sedona helps implement the control as a working workflow: AI recommendation, confidence or quality signal, threshold, escalation, human review, decision and audit event. The FCA explicitly says human oversight should be designed around the particular use case, autonomy and potential impact, with clear thresholds for intervention, escalation and override. That's software and engineering, not policy consulting.

2. AI governance Observability

Ensuring holistic, correlated observability may mean building a common telemetry data layer, connecting AI activity with human decisions and business outcomes. This can capture:

• Request: the AI use case, application, user and model version, with an interaction identifier to trace activity across systems.

• Context: approved data sources, retrieved material and relevant rules or controls, showing what information supported the response.

• LLM actions: outputs, evaluation results and safety or quality signals, including whether checks passed or triggered intervention.

• Human-in-the-loop activity: acceptance, edits, rejections, overrides and escalations, recording how reviewers challenged or changed the AI’s recommendation.

• Business workflow: the final decision, customer outcome and compliance exceptions, linking model behaviour to what happened in practice.

The FCA identifies logging AI interactions and maintaining audit trails as good practice, supporting monitoring, complaints handling, explainability and investigation. At Sedona, we help build or augment systems that connect these records, giving firms the evidence to investigate issues and demonstrate how their controls operate.

3. AI QA and Evaluation

The FCA’s AI claims example assessed outputs for completeness, faithfulness to source, conciseness, adequacy of reasoning and safety. It combined human
assessment with automated evaluation, including “LLM-as-a-judge”.fca.org

This reflects the AI QA services we already deliver to multiple insurers. We help teams build reusable evaluation harnesses that test outputs against agreed criteria and examples reviewed by subject-matter experts. These cover:

• Underwriting: checking extraction accuracy, risk summaries, evidence grounding and recommendation consistency. Have important facts been captured? Does the evidence support the recommendation?

• Claims: assessing document extraction, coverage reasoning, claim summaries, correspondence and settlement recommendations. Does the output reflect the policy wording? Is uncertainty flagged for human review?

• Customer service: testing accuracy, vulnerability handling, hallucination, clarity and escalation. Is the response correct and understandable? Does it recognise when the customer needs additional support? 

These tests can be repeated when a model, prompt or data source changes, helping teams spot where performance has improved or deteriorated. Automated evaluation makes that process more efficient, while human reviewers assess the judgements that need their expertise.

Insurers should not have to invent an evaluation framework for every new use case. Reusing proven QA patterns gives teams a practical starting point, with tests adapted to the risks of each application. Smaller firms can begin with a focused set of representative scenarios and build from there.

4. Consumer-outcome Testing

The FCA says firms should test AI outcomes across consumer characteristics and scenarios, manage bias and accuracy, and monitor live outcomes. It also notes that smaller firms, including some of our Friendly Society and Mutual customers, have successfully used sample outputs, vulnerability markers and scenario comparison instead of running enormous test programmes. We help them build manageable test packs. A claims AI scenario pack, for example, might contain around 1,000 representative or synthetic scenarios varying claim complexity, vulnerable-customer indicators, financial distress, accessibility needs, incomplete information, ambiguous policy wording and unusual circumstances. This allows us to test model A against model B and against a human baseline, then compare outputs to flag material or gold-data-set variation. It is an extremely tangible governance engineering deliverable.

5. AI operational Resilience Testing

AI resilience is about keeping the insurance process operating safely when technology fails, changes or produces unreliable results. The FCA highlights practices including stress testing, red teaming, dependency mapping and tested contingency arrangements.fca.org

At Sedona, we work alongside software delivery and testing teams to build and automate scenarios such as:

• LLM unavailable: checking whether work can move to manual handling or an approved alternative without cases being lost.

• Model changed: rerunning evaluations to identify changes in accuracy, reasoning and recommendations.

• RAG source unavailable: testing whether the system flags missing supporting information rather than generating an unsupported answer.

• Severe latency: checking timeouts, queues and escalation routes so delayed cases have a clear owner.

• Poor-quality responses: testing whether unsuitable outputs are detected and referred for review.

• Provider switches model version: checking whether an update affects established controls or customer outcomes.

• Guardrail failure: testing whether unsafe activity can be detected, stopped and escalated.

The important questions go beyond whether a system comes back online. Can staff continue safely? Who activates the fallback? How are interrupted cases recovered? What evidence shows that the response worked?

Third-party contingency planning also needs practical testing. An alternative provider is only useful if the workflow, data access and controls work with it. Our approach is to test those arrangements before they are needed, including how a service could transition away from a provider.

For smaller firms, a manageable starting point is one important AI-supported process, its most consequential dependencies and a focused fallback exercise. The aim is evidence that the business can respond safely, not a large testing programme for its own sake.

What This Means for Insurers in 2027?

For insurers planning their next phase of AI adoption, our view is that governance, QA and resilience need to develop alongside delivery. Controls are harder to apply consistently when they are added after the workflow has already been built.

Our services and accelerators roadmap reflects that. For example, we are currently developing an AI FinOps and total cost of ownership platform accelerator, while assessing where further capabilities could offer customers the most value.

One area is Evidence-as-a-Service, bringing together design decisions, test results, runtime records, customer outcomes and governance approvals into evidence packs for each use case. The aim is to make it easier for teams to demonstrate how their controls operate without having to assemble records from disconnected systems.

We are also exploring AI risk threshold software rules. These would provide red, amber and green performance indicators linked to agreed responses, with alerts and escalation built into dashboards or business workflows.

For agentic AI, we are assessing workflow orchestration governance that makes an agent’s actions, tool use, data access and intermediate decisions visible. This would help teams understand where human approval is required and how the final outcome was reached.

These are areas of investment and exploration, not a claim that one platform can solve AI governance. We want customers’ priorities for 2027 to help shape where we focus, particularly where teams need support turning agreed policies into working controls.

There is no silver bullet for following the FCA’s advice. Sedona’s role is practical: working alongside insurers’ teams on implementation, testing and monitoring, so they can demonstrate how AI governance works in the services their customers use.