Conversational banking, as built.
Two assistants on one platform: one for the bank's employees, one for its customers. This is what exists, how it is secured, what it got wrong on the way, and what now stops each of those things happening again.
- Prepared for
- The bank's AI advisor
- Prepared by
- UTEEK Digital Agency
- Status
- M1 delivered · M2 in production pilot · M3 specified
- Date
- 20 November 2026
Arrow keys or scroll to move. Every panel on this deck can be clicked.
The engagement
Three milestones, one platform underneath.
The contractual baseline is the Cahier des Charges v1.0. It describes an internal assistant first, a customer assistant second, and transactions third — in that order, because each one is the previous one plus a thing that can go wrong.
| Milestone | What it is | State |
|---|---|---|
| M1 — Agents Assistant | Read-only assistant for bank employees over the internal knowledge base, with citations, guardrails and a full audit trail. Back-office for the corpus, conversations and analytics. | delivered |
| M2 — Customer Chatbot | The assistant the mobile app talks to, plus the dashboard where the bank configures everything it says — without a deployment. | in pilot |
| M3 — Transactions | Transfers and other state-changing operations: the assistant proposes, the bank's own backend confirms, runs the OTP and executes. | specified, contract frozen |
M3's interface already exists and is already honoured by M2 — the assistant emits a
trigger today and the bank's backend decides what to do with it. That is why
the transactional milestone is an integration rather than a rebuild.
As of this morning
What is running, counted rather than claimed.
The platform is two repositories that do not depend on each other: either can be released without the other, and the shared code is vendored with a checksum per file so that drift fails a build instead of a conversation.
The shape of it
One request in, one response out — and the model at the edge of it.
Click any box.
The assistant sits between the bank's backend and three things it is allowed to read. It has no path to money, and no path to the employees' corpus.
- Mobile app sends one field: the message.
- NAB Backend adds the session and history, and fills every placeholder.
- AI Backend routes, validates, audits, answers — once.
- Model, knowledge, database are the only three things it reads.
- Core banking is unreachable from it, enforced at build time.
Non-negotiables
Eight rules the code is held to, not eight intentions.
- The model never computes authorisation. It proposes an intent and some values. Code decides whether that is a thing this customer's app may be asked to do, and the bank's backend decides whether it happens.
- The model never sees customer data. It gets the typed message with personal data masked, the trimmed history as templates, the active tool list and knowledge passages. Not a balance, not a name, not an account.
- One request, one response. No loop-back, no second call, no narration of a result the assistant never saw.
- Everything a customer reads comes from the configuration — except a grounded knowledge answer and the opening line. Both of those are bounded, and both are shown to the bank before a customer sees them.
- No invented procedures. With nothing relevant retrieved, the assistant says so and points at the department that owns the answer. It never improvises a fee, a rule or a process.
- Everything is audited before it is answered. If the audit write fails, the request fails. A response nobody can account for is worse than an error.
- One-time codes never enter the assistant. A message that looks like one is dropped unlogged and answered with a fixed phrase.
- Corpus isolation. The customer channel cannot express a request for the employees' corpus — the channel is the URL, and the credential grants one route.
What a customer can do
Three layers, and the assistant behaves differently in each.
The answer is written by the model from retrieved passages only, and it carries the document, the section and the effective date. Retrieve nothing relevant and there is no answer to write: the assistant says the knowledge base does not hold it and offers the department that does.
"nextStep": "reply",
"message": "…",
"citations": [{ "title": "Fee schedule",
"section": "Internal transfers",
"version": "2026-03" }],
"configVersion": "cfg_8",
"auditId": "01M7RM…"
The assistant does not fetch the balance and never holds one. It returns a trigger: the action, the validated parameters, and one message per outcome with placeholders left unfilled. The bank's backend reads the account, substitutes, and shows the result.
"nextStep": "trigger",
"trigger": { "type": "getBalance", "tier": "read",
"parameters": { "account": "primary" } },
"messages": { "success": "رصيدك الحالي {{balance}} دينار.",
"error": "…" },
"placeholders": ["balance"]
The assistant extracts the amount and the name, normalises the money to three decimals, and checks both against the rules the bank wrote. It does not resolve the beneficiary, does not confirm, does not see the one-time code and does not execute. Every outcome — success, cancelled, wrong code, ambiguous name — already has its sentence in the response.
"trigger": { "type": "triggerTransaction",
"tier": "transactional",
"requiresOtp": true,
"parameters": { "amount": "500.000",
"currency": "LYD",
"beneficiaryName": "محمد" } }
The same engine produces all three. What differs is the tier the bank gave the tool, and the tier is configuration — not a branch in the code.
The claim, and the receipt
"The model never sees customer data" — here is the request.
The assertion is cheap; the artefact is not. Every prompt is assembled in one place, a test spies on the client and fails the build if a value that did not come from the masked message reaches it, and the dashboard shows an administrator the exact payload beside the answer it produced.
- Masked first. Card-like and account-like numbers, phones, emails and codes are replaced before anything is sent — and before anything is logged.
- History is data, never instruction. At most the configured number of turns, masked the same way, and never followed.
- The tool list is names and types. No values, no customer, no balances.
// exactly what the router is given
{
"message": "حول 500 لمحمد من حسابي ****1234"
→ masked → "حول 500 لمحمد من حسابي [ACCOUNT]",
"history": ["رصيدك الحالي {{balance}} دينار."],
"today": "2026-11-20",
"tools": [
{ "action": "triggerTransaction",
"fields": ["amount:money", "beneficiaryName:string"] },
{ "action": "getBalance", "fields": ["account:enum"] }
]
}
Note the history line: the template, with the placeholder still in it. The bank's backend filled that bubble for the customer — the assistant never received the number, so it cannot leak it back into a prompt.
Who owns the words
Everything the bot says is the bank's, and changing it is not a deployment.
Tools
Each action with its tier, its fields and their validation, the few-shot examples the router learns from, and one message per outcome per language. Thirty-one of them are live.
Phrases & placeholders
Every fixed sentence — unclear, cancelled, each refusal reason, the opening — and the declared placeholders a template is allowed to use.
Knowledge base
Upload, re-index, withdraw, and test a question against the corpus to see the passages that would ground the answer.
Playground
The real engine against an unpublished draft: the phone on the left, the exact response the bank's backend would receive on the right, and the prompt that went to the model underneath it.
Draft → publish → roll back
Edits land on a draft. Publishing validates the whole configuration and refuses on an undeclared placeholder, a missing outcome or a transactional tool without its confirmation path. Rollback republishes an earlier version.
Roles and audit
Viewer, editor, approver, admin. A change to a transactional tool needs a second approver, and every edit and publish is in the change log with before and after.
Shipped this week
The bot speaks first — and that one sentence is written by the model.
Customers were opening the chat, typing "مرحبا", waiting, and then asking something the bot could not help with. So it opens the conversation instead: a greeting and one question — do you want to do something in the app, or ask about something? — which is the distinction the whole product is built on.
- Generated per open, so it is not the same sentence a thousand times. The bank reads it in the dashboard before a customer does.
- Bounded. The model is told no products, no fees and no tools — it cannot offer what it does not know exists.
- Four checks, then the bank's own wording. It must arrive, be short, be in Derja, and actually ask something. Anything else falls back, and the response says so.
- It is not a turn. No session, no history. Ignore it and the first real message behaves exactly as it always did.
This is the one sentence in the product the bank cannot review in advance, and that trade-off is written down as a decision record rather than discovered in a meeting.
The model
DeepSeek R1 Distill 32B, in the bank's own cloud, behind one abstraction.
- Model
- DeepSeek R1 Distill 32B, quantised
- Serving
- vLLM, OpenAI-compatible endpoint
- Where
- A single GPU instance in the bank's own cloud account
- Shared by
- Both products — employees and customers
- Leaves the account
- Nothing. No third-party inference API.
Every call goes through one provider interface. Moving to a managed endpoint, to a second self-hosted one, or to a model the bank trains itself is a configuration change and a credential — not a code change. That was a requirement from day one and it is enforced by a build-time rule: no vendor SDK is importable anywhere else.
Two decoding settings, on purpose
The chat path is greedy — temperature zero. Not "low": zero. An answer is assembled from retrieved passages and must carry its citation markers, and we measured the alternative — same question, same sources, eight runs: two cited, six did not. Sampling was deciding whether the guardrail withheld the answer, which taught employees that a refusal is something you get past by asking again.
The opening line is the only sampled call, because its whole purpose is that the wording varies. It carries its own seed per request so that the platform's reproducibility — which everything else depends on — is not weakened to buy it.
Retrieval
Two corpora that cannot meet.
Ingestion is structure-aware: a document keeps its headings, so a chunk knows which section it came from and an answer can cite it. One cross-lingual embedding model covers Arabic and English — never two monolingual ones, or a question in Derja would never find an English procedure.
- Hybrid retrieval. Semantic and keyword together, because a fee table is found by its number and a policy by its meaning.
- Citations are structural. Title, section, version, effective date — a fee answer that cannot name its schedule is not shown.
- Validity windows. A superseded document stops answering on its end date rather than when someone remembers.
How isolation is actually enforced
Not by a prompt, and not by a flag the caller passes. The customer service has no retrieval code of its own: it calls one route whose URL is the channel, with a credential scoped to that route and nothing else. It then checks the collections that came back against its own declaration of what the customer channel may read, and fails the request if the two disagree.
One declaration can be wrong in silence. Two that disagree is an error — which is the whole design: the failure of corpus isolation is the only one that would be silent, so it is the one place we pay for a second opinion.
Infrastructure
Two estates: ours to break, the bank's to trust.
The vendor estate
Our own server, our own domain. Every change lands here first: the two applications, the API, the customer bot, the dashboard, the build log and this deck. It is where a migration is run for the first time, and where a mistake costs an afternoon.
The bank's estate
A cloud account the bank owns, inside the bank's own network boundary, reachable only over a private tunnel. Deployed from a generated release branch — not from the branch we develop on — so what runs there is a reviewed artefact rather than whatever was last pushed.
- Containers, one compose file. API, both frontends, the bot, the dashboard, PostgreSQL, the vector store — each with its own resource and log limits.
- Apache terminates TLS with certificates renewed automatically, and routes by hostname to a loopback port. No container is exposed to a network interface directly.
- Secrets live in one root-owned environment file on the host. Nothing is in the repository, and the deployment script refuses to run with a placeholder left in it.
- Releases are reproducible. Images are tagged with the date, so a rollback is one command and not a rebuild of yesterday's code.
The network
The assistant is not on the internet.
Four controls, in series.
The route exists only inside the tunnel; the security group admits only its ranges; Apache admits only named addresses; and the application still demands a credential. Click a region of the diagram.
- The bank's network resolves the assistant to a private address.
- A site-to-site IPsec tunnel with static routing carries the traffic.
- The cloud network admits the tunnel's ranges and nothing else.
- Apache allows named addresses only, then the application authenticates.
What UTEEK did here: specified the ranges and the routes, wrote the allow-list into the deployment script so it cannot be deployed without one, and worked the fault through with the bank's network team when the path did not come up — see the defects slide.
Defence in depth
Nine layers, and no single one is the control.
Network — the assistant is not reachable from the internet
Private tunnel only, static routes on both sides, a security group that admits the tunnel's ranges and a route table that sends them back. A scan of the public internet finds nothing to talk to. This is a written decision, not an operational habit.
Edge — an address allow-list in front of every route
Apache terminates TLS and admits only the addresses the bank named. The deployment script validates the shape of every entry and refuses to deploy the customer bot without one — the failure mode of a forgotten allow-list is a bot open to whoever found the name, so it is not possible to forget.
Identity — password plus a code to the address on the account
Dashboard sign-in is two-factor by email. A correct password buys a challenge, not a session. Access tokens are short-lived and refresh tokens are rows in the database, so a suspension or a role change bites at the next refresh rather than whenever a token happened to expire. Failed attempts lock the account.
Roles — four, enforced on the server
Viewer, editor, approver, admin. The navigation hides what a role cannot use; the server is what refuses it, and a role matrix test covers every route. A change to a transactional tool or an offer needs a second approver, and nobody can approve their own draft.
Payload — unknown fields are rejected, not ignored
Every request model forbids extras. A backend that starts sending an account number, a customer id or a location gets a 422 rather than a successful call whose extra data we quietly hold. The contract is enforced in both directions, and the examples in the contract document are executed as tests.
Prompt — injection is detected, and history is never obeyed
The router returns a suspicion flag and a flagged turn is refused with the bank's own wording. Conversation history is treated as untrusted data: masked, trimmed, and never followed as instruction. A red-team set runs in the pipeline, and the contractual bar is zero successful injections from it.
Data — masked before use, and one-time codes never enter
Account-like and card-like numbers, phones and emails are masked before the model, before the logs and before the audit store. A message that is just 4–8 digits is treated as a one-time code: dropped, never logged raw, answered with a fixed phrase telling the customer not to share it.
Audit — append-only at the database, not in the code
The audit role holds INSERT and nothing else: no UPDATE, no DELETE. Every turn is written before its response is returned, and a failed write fails the request. The change log records every dashboard edit and publish with before and after, and who did it.
Build — the boundaries are compiled, not remembered
Three import rules run in the pipeline: the AI Backend may not import anything that can execute a transaction, it may not import the dashboard, and the vendored platform may not import this repository. A developer who crosses one of those lines fails the build with the file name, in seconds, instead of passing review.
The honest slide
Nine defects, what each one actually was, and what now stops it.
Every one of these was found by us or by the bank during the pilot, and every one is fixed and deployed. Open any of them.
network The tunnel was up, and the traffic never used it.
What it looked likeThe dashboard opened from inside the bank. The API, on the same server, timed out. No request ever arrived — the access log was empty, which is a different thing from a refusal.
What it wasThe bank's internal DNS answered the dashboard's name with the private address and the API's name with the public one. Requests for the API left the bank's network looking for an address nothing was allowed to reach, and died in the dark.
The fixThe internal record was pointed at the private address. One line, after two days of elimination: service healthy locally, both hosts present, certificates valid, allow-list correct, zero traffic in the log.
What stops it coming backThe diagnosis order is written into the runbook — start at the log, because an empty log means the packets are not arriving and nothing about the application can explain that.
edge The allow-list applied to one port and not the other.
What it looked likeThe allow-list was edited, the deployment re-run, and the live rule did not change.
What it wasThe certificate tool writes its own copy of each site's configuration — the one that actually serves HTTPS — and our script could only update the plain-HTTP original. The two drifted, and the version being enforced was the older one.
The fixBoth were brought into line on the host, and the deployment runbook now ends with a command that prints the rule from every file at once.
What stops it coming backA verification step that is part of the deploy, not a thing to remember: four lines printed, and they must read identically.
deployment Ports were resolved at run time and never written down.
What it looked likeContainers started by hand failed to bind, on a host where the same containers started by the script worked.
What it wasThe script walked upward from a base port until it found a free one and exported the result for its own use. The numbers lived for the length of one run. Anything else — a developer, a second repository — fell back to defaults that another service already held.
The fixThe ports are pinned in the environment file and defined in exactly one place, and the script reads them rather than inventing them.
What stops it coming backA test that fails if two services can be handed the same port.
dashboard Signed out mid-click, every fifteen minutes.
What it looked likeAn editor working continuously was returned to the sign-in dialog. The idle timeout was thirty minutes and the activity tracking worked perfectly; neither had anything to do with it.
What it wasA browser decides whether to send a cookie by comparing its scope to the URL it is asking for. The session-renewal cookie was scoped to the path the server knows, and the browser asks for the path the proxy publishes. The two differ by a prefix, so the credential was never sent, every renewal failed, and the session ended one token later. Both halves were correct in isolation, which is why review did not catch it.
The fixThe scope is now written in the browser's terms, and is configurable for a deployment that publishes the API differently.
What stops it coming backA test that asserts the cookie's scope against the literal URL the frontend asks for — it fails on the old value.
model A sampled greeting that was the same sentence every time.
What it looked likeThe opening line is generated so it varies. It was set to a high temperature. It produced one sentence, every open, while truthfully reporting that the model had written it.
What it wasThe platform pins a random seed so that graded answers are reproducible — which is right for every other call. A fixed seed with an unchanged prompt returns the same sampled text at any temperature. The variety knob was connected to nothing.
The fixThat one call carries its own seed per request, and the instruction itself rotates through eight different angles — so the sampling moves, and it moves somewhere worth going.
What stops it coming backTwo tests: one asserts the seed differs between calls, the other that forty prompts are not all the same prompt.
model The model greeted customers with its own instructions.
What it looked likeAn opening line that began with a mangled religious invocation, and read as written by a stranger rather than by a bank.
What it wasThe instruction was written in Arabic — "greet the customer" — and the model handed the verb straight back, conjugated at the customer. Instruction-following in a dialect that is barely written down is far weaker than in English.
The fixThe instruction is in English and asks for Derja output. The register is now specified too: no Modern Standard Arabic, no religious formulae. The exact phrases the bank sent back are checked for, and a generation containing one is replaced by the bank's own wording.
What stops it coming backEach rejected sentence is a test case. They pass every structural check — short, Arabic, one question — and are still refused, which is the point.
model The opening line spoke English to Arabic-speaking customers.
What it looked likeThe app opened in English on a chat whose every other message was in Derja.
What it wasThe opening is chosen before anyone has typed, so there is no message to detect a language from. It fell back to the configuration's first enabled language, which is English.
The fixThe opening is always Libyan Derja in Arabic script. The conversation is unaffected — the assistant still detects what the customer actually typed and answers in it, English included.
What stops it coming backThe decision is a constant with a test around it, not a default that can drift when a language is added.
platform The vector store ran out of file descriptors.
What it looked likeIndexing failures under load that did not reproduce locally.
What it wasThe container inherited the host's default descriptor limit, which is well below what a segment-based vector store needs when collections grow.
The fixAn explicit limit in the compose file, so the store gets what it needs wherever it is deployed.
What stops it coming backIt is declared in the file that defines the service, next to its memory and log limits, rather than configured on one host by one person.
api A validation mistake answered with a 500.
What it looked likeA malformed request to a dashboard endpoint returned "internal server error" instead of telling the caller which field was wrong.
What it wasA validation error raised outside the handler's own try block escaped as an unhandled exception.
The fixIt is caught and returned as a 422 with the field list — the same shape every other endpoint uses.
What stops it coming backA test that sends a field the endpoint does not know and asserts a 422.
Why the list above is short
Every fix leaves a gate behind it.
3,785 tests
Unit, integration and acceptance, in Arabic and English. The acceptance tests are the contractual criteria, written as the feature is built rather than after it.
Three import contracts
The assistant cannot reach anything that executes, cannot import the dashboard, and the vendored platform cannot import the product. Checked in the pipeline.
152-utterance evaluation
English, Derja in Arabic script, Derja in Latin. The gate covers intent accuracy, slot exact-match, zero invalid triggers, and zero customer data reaching the model.
Publish validation
A configuration cannot be published with an undeclared placeholder, a missing outcome message, or a transactional tool without its confirmation and failure paths.
Interface checks
Every example in the integration contract is executed as a test. A document that drifts from the code fails the build — so the thing the bank's team integrates against is true.
Decision records
Thirty-two of them. Every deviation from the specification is written down with its cost before it is built, which is what makes a review like today's possible at all.
What we hold ourselves to
The budgets are in the contract, and in the tests.
One call per turn is an architectural choice before it is a cost one. A design that calls the model to route, again to narrate a result and again to decide a follow-up has three chances to say something nobody approved — and three times the latency to explain.
Open, in the open
What is not done, said plainly.
| Item | Where it stands |
|---|---|
| The transactional path, end to end | The assistant's half is built and under test. It needs the bank's backend to resolve beneficiaries, run the confirmation and the one-time code, and execute. |
| Reaching the rest of the bank's backend estate | One more internal range has to be carried by the tunnel. The bank's network team has the specification; our side is a route, a security-group entry and an allow-list line. |
| Knowledge coverage | The pipeline is built and the corpus is partial. Retrieval quality is measured against a validation set; the gap today is documents, not mechanism. |
| Four decision records awaiting the bank | Including the one that says an unreviewed sentence may open every conversation. They are proposals until the bank confirms them. |
| Arabic answer quality | Measured, not assumed: the evaluation set carries Derja in both scripts and the failures it finds are worked rather than explained away. |
For today
Three things worth a second opinion.
The one unreviewed sentence
The opening line is written by the model, bounded but not approved in advance. We recommended a list the bank writes and picks from; the bank chose generation for the variety. Both are defensible. Is the bound we put around it the right one?
Where the boundary sits
We hold no customer data, resolve no beneficiary and execute nothing — every placeholder is filled by the bank. That makes the assistant simpler to assure and pushes work onto the bank's backend. Is that the trade the bank wants to keep at M3?
Model and hosting
One quantised open-weights model, self-hosted in the bank's own account, serving both products, behind a swappable interface. The alternatives are a managed API — faster to scale, data leaves the account — or a larger self-hosted model at more cost.
Not slides
Everything on this deck can be opened now.
- The dashboard. Sign in, edit a sentence on the draft, open the Playground, watch the engine answer with it — then publish and watch the live bot change.
- The Playground. The phone on one side, the response the bank's backend receives on the other, and the exact prompt that went to the model below. Including the opening line: press reset twice and the wording differs.
- The Agents Assistant. Ask it an internal question and read the citation it answers with — then ask it something not in the corpus and watch it decline rather than improvise.
- The change log. Every edit made during this meeting, with who made it and what it replaced.
The demonstration estate is open; the bank's own deployment is reachable only from inside the bank's network, which is the point of the network slide.
Built to be checked.
Two assistants, one platform, one model endpoint the bank controls. The rules are in the code, the deviations are in writing, and the defects are on a slide — because a system a bank is going to trust has to be one somebody outside it can audit.
- Vendor
- UTEEK Digital Agency
- Documents
- CDC v1.0 · integration contract v0.3 · 32 decision records
- This deck
- demo-final.nab.uteek.net