Imperial College - Follow On - Regulatory View On The Bot

As part of the Imperial AI for Business programme, I built a RAG system & agent that drafts responses to FCA complaints, grounded in a ficticious firm's own policy and procedure documents. I wrote up how it works, and where it falls short, in the last post. The programme's framing of ethics as architecture stayed with me; this post is where I tested whether that idea holds when you trace a regulated system end to end.

Nearly every regulatory question that came up could be addressed with architecture rather than anything model-specific: where the data goes, who makes the decision, and what you can prove after the fact. The regulations are essentially asking data-flow and access-control questions, not machine-learning ones.

This is that walk-through on the bot that I built rather than purely a hypothetical paper exercise. I'm not offering a certified approach; it's one system, one corpus, one set of regulators. However the design should largely generalise and that is the point.

The system in one diagram

Reference data flow for a regulated LLM deployment.

Read it left to right. A user submits a query; it passes through redaction before anything else touches it; retrieval queries a corpus of regulated documents and hands the model a prompt made of the question plus the chunks it found; the model generates a draft; an output filter checks it; a person reads the result and decides what to do with it. Underneath, every stage writes to an append-only audit log. The server-side components sit inside an EU region boundary, and the person making the decision sits outside the model's data plane. That last detail looks cosmetic. It isn't, as the section on risk tiers will show.

Everything below is a walk along this picture.

Walking the flow

Each hop is three things at once: a place something can go wrong, a control that addresses it, and a regulator who cares whether you got it right. Rather than separate those into three sections, I'll take all three at each stop because they are rarely separable in practice; the control that closes the risk is usually the same design decision that satisfies the rule, and the compliance falls out of building the thing properly.

The input boundary. Two things that you don't want reaching the model can arrive in a user's message.

The first is an instruction pretending to be a question, commonly called prompt injection. The control for this is relatively simple: treat the incoming text as data, not as instructions, and don't let it reach a position where it can redefine what the system is for.

The second is personal data. People put all manner of PII into complaints (account numbers, addresses) because that is what a complaint is made up of. So I decided to redact at the input layer, but session-scoped and reversible. Reversible matters because later on the handler still has to be able to read the real complaint so it can't be removed entirely. But I'm keeping raw personal data out of every downstream component that has no need to see it. The audit entries are HMAC-chained, which lets me prove what happened to a record without writing the personal data into the log to do so. None of that was a compliance task; it was a decision about where redaction sits and what gets logged. It discharges GDPR data minimisation and the Article 28 processor duties as a side effect.

Retrieval and the corpus. This is where prompt injection gets interesting. The model is about to be handed text pulled from documents, and if an attacker can get instructions into those documents, the injection arrives wearing the corpus's credibility rather than the user's. The defences are architectural: scope retrieval to a vetted corpus, track provenance so you know where a chunk came from, and keep retrieved text in the same untrusted-data bucket as the user's input. The corpus itself is regulated content, so its handling inherits the same residency and access rules as everything else inside the boundary.

The model endpoint. Here the question is purely infrastructural: where does the prompt physically go, and what happens to it when it gets there? For a regulated workload you want the endpoint in-region (e.g. EU), no route to the public internet, and no retention of prompts or outputs by the provider (ZDR). Those three requirements cover most of what GDPR asks about international transfers and what DORA asks about third-party dependencies. The model's cleverness is not the subject of this hop; the network path is.

The output filter. The model has produced a draft. Before a human sees it, two things are worth checking: that the answer is grounded in the retrieved context rather than invented, and that it carries its citations so the person can verify it. In a regulated setting a confident fabrication is the expensive failure, so grounding and citation are not nice-to-haves; they are the actual control. This is also where the EU AI Act's transparency expectation lands: the output is clearly AI-generated and presented as a draft for a person to check rather than a decision.

The audit log. The recurring regulatory question is not "is the model good" but "can you show what happened". So the audit surface is a first-class component, not an afterthought bolted on at the end. Every stage writes to it; it is append-only and HMAC-chained so entries can't be altered; and it deliberately does not contain raw personal data. That single design choice answers DORA's incident-reporting expectations and the FCA's record-keeping ones at the same time.

The human decision. The flow ends with a person because the human has the real responsibility and authority: they read the draft and they decide, which means the system does not act on its own. This is the most consequential design decision in the diagram, and it is the subject of the next section.

Is this high-risk?

The EU AI Act sorts systems into risk tiers, and a financial-services use case sounds like it should land in the high-risk one by default. My take is that doesn't have to, and the reason is the shape of the diagram rather than anything I added later.

The complaints bot is assistive. It drafts; a qualified person reviews every output and makes the decision; the system neither profiles individuals nor determines the outcome. That puts it on the Article 6(3) route, where a system that would otherwise sit in a high-risk area is not classified as high-risk because it doesn't materially influence the decision. The obligations that remain are transparency under Article 50 (telling people they are dealing with AI-assisted output) and maintaining genuine human oversight. Not the full high-risk regime.

The thing worth noticing is when that outcome was decided. It was decided the moment the human went at the end of the flow with real authority, not when a control was added. Put the model in the decision seat and the same system lands in a different, far heavier tier. "EU AI Act readiness", for a lot of systems, means right-sizing the obligation rather than assuming the worst case and gold-plating against it.

One caveat I'd make in any real engagement: classification is fact-specific, and the Article 6(3) route has to be assessed and documented, not asserted. The discipline is to write the reasoning down, keep it with the system, and revisit it when the system changes. The conclusion is cheap and the documented assessment is the deliverable.

Data Flow vs Frameworks

The four frameworks that apply here are not four separate projects. They are four vocabularies asking the same handful of questions of the diagram above: where does the data go, who can touch it, can you prove what happened, and who decided.

Where to run it?

The requirements at the model endpoint are provider-agnostic because whatever you run on, you want the same five things: the endpoint in-region, no public-internet egress, no provider retention of prompts and outputs, a complete audit record, and somewhere to put a human in the loop.

At the time of writing Amazon Bedrock, and Google Vertex AI can each meet that set due to regional endpoints. The selection isn't really about the provider or which frontier model; it's about where the rest of your data, your identity controls, and your existing audit tooling already live, because the cheapest secure architecture is usually the one that reuses controls you already operate.

I'm keeping this short on purpose, because the specifics change faster than a blog post can keep up with. The durable part is the requirement list; the per-provider mechanics are something to confirm against current documentation when you build, not to memorise from a post like this one.

When can it act?

Everything above describes a system that answers and then stops, leaving a person to act. The moment you let the system act for itself, through tool use or an agent calling out over something like MCP, the same data-flow discipline still holds, but two hops carry more weight. The credentials each tool runs with have to be least-privilege, scoped to exactly what that action needs; and the audit surface has to capture autonomous actions, not just generations. The human-decision hop would become a human-approval hop for anything that matters.

I haven't run this for a real regulated workload so I'm simply highlighting the direction the principles point towards. That said, the approach shouldn't change: it is the same diagram, with the boundary drawn more carefully around what the system is allowed to do on its own.

The point

The regulatory work turned out to be data-flow work and human-oversight work. The controls that carried weight were the unglamorous architectural ones: where redaction sits, where the endpoint runs, where the audit log lives, and where the human sits relative to the decision. The model-level guardrails matter, but they are not what an auditor would ask about first.

It's one system, and I'd resist drawing any real outcomes from it without talking to GRC. But the questions generalise, and so does the answer: get the architecture right, and a good deal of the compliance is something you've already built.