End-to-end solution · Bank agent platform

Assembling frontier technology into an auditable bank agent platform

Whether agents can go live in a bank depends on the platform, not on any single application. Our researchers assemble the Frontier TOP20 technologies and the TOP20 collaboration mechanisms into an eight-layer component architecture: runtime, context, memory, knowledge, tool gateway, identity and approval, security, observability and evaluation are provided once by the platform, with governance built in rather than bolted on. The fifteen component-level blueprints below are our reference architecture and methodology for building a bank agent platform.

15 component-level blueprints8-hour session limit5-level action approval4-level end-to-end tracing
A long, symmetrical data-center aisle lined with rows of server racks under bright ceiling lights, in teal tones
Overall design

Fifteen component-level blueprints, two architecture principles

The detailed design of the bank agent platform comes in four groups: overall design and runtime (A1–A3), context and knowledge (A4–A6), control and measurement (A7–A11), and deployment and traces (A12–A15). Each one is a component-level blueprint you can build from, not an illustration.

Principle one: a platform, not per-application builds

Context assembly, memory, the tool gateway, identity, approval, observability and evaluation must be provided once by the platform. If every application builds its own, compliance checks cannot see through them and the cost is paid again and again.

Principle two: governance built in, not bolted on

Approval levels live in tool metadata and are enforced by the gateway; red-line constraints live in a fixed context zone that is never compressed; audit fields are produced automatically with every call. Governance that relies on after-the-fact checks is always late and always incomplete.

How to read the blueprints: in A1 the left column is the capability stage (what), the middle column the functional requirements (how), and the right column the platform module (who). A2–A15 then expand each module in the right column.

From Capability Stages to Platform Components

A1 Overall architecture · How the seven capability stages map to platform components

The seven-stage capability architecture is where platform design starts: for each stage we first fix what it must do, break that into functional requirements, and land them on modules the platform provides once for everyone.

Capability stage → functional requirements → platform module
Capability stageFunctional requirementsPlatform module
① Completion awarenessTask intake gateway · intent and boundary parsing · acceptance-criteria registrationSession gateway / task ledger
② Cognitive planningPlanning engine · milestone generation · human plan-confirmation pointOrchestration engine (C11)
③ Resource useContext assembly · memory retrieval · knowledge retrieval · tool selectionContext / memory / knowledge services · tool gateway
④ Controlled executionSandboxed execution · permission checks · tiered approval · single convergent writerRuntime / identity service / approval center
⑤ Result verificationExternal checks · judge scoring · cross-validation · citation trace-backQuality service (C14/C18)
⑥ Risk governanceGuardrails · injection defense · end-to-end audit trail · content labeling · circuit breakingSecurity service / observability platform
⑦ Continuous improvementExperience write-back · evaluation regression · staged rollout · drift detectionEvaluation platform / experience library

Layered Component Architecture

A2 Layered logical architecture · Eight component layers and their interface boundaries

The platform has eight layers from top to bottom, with seven cross-cutting concerns running through all of them; four interface boundaries constrain how the layers call one another.

Eight component layers and cross-cutting concerns
  1. 1Channel accessMobile banking · online banking · branch terminals · remote-banking agent desk · corporate online banking · open APIs · internal office clients
  2. 2Agent applicationsCustomer-service assistant · relationship-manager assistant · credit-review agent · AML screening agent · trade-document review agent · operations digital workers · developer assistant
  3. 3Orchestration & executionPlanning engine (C11) · orchestration pattern library (C9) · single writer (C10) · durable execution and checkpoints (C12) · approval center (C15)
  4. 4Context & memoryContext-assembly service (C8) · short-term session memory · long-term memory (extract–consolidate–reflect) · experience case library (T4) · namespace isolation
  5. 5Knowledge & dataPolicy-clause vector store · customer and related-party graph · customer 360 view · business receipts and validation results · data classification and grading tags
  6. 6Models & inferencePrivate large-model cluster · small-model matrix (T15) · routing and cascades (C3) · inference acceleration and caching (C4) · fine-tuning and distillation pipeline (C17)
  7. 7Tools & integrationMCP tool gateway (C2) · adapters for core, credit, payments and other systems · external data interfaces · code and browser sandboxes
  8. 8Governance & operationsAgent identity directory (T19) · security guardrails and red teaming (T20/C16) · observability and audit (C20) · evaluation platform (C19) · model risk management
Boundary one: applications never call models or tools directly

Every model call is routed through the inference layer and every tool call goes through the tool gateway — otherwise cost, permissions and the audit trail all slip out of control.

Boundary two: writes only through the orchestration layer’s single writer

This is how read–write asymmetry is built into the architecture (C10), and it gives the audit trail a single anchor.

Boundary three: knowledge-layer permission filtering happens at retrieval

Candidates are filtered by the relationship manager’s permissions and data classification tags, not masked after generation (see C5).

Boundary four: the governance layer can veto any call

Security guardrails and the approval center take priority over business logic, and every interception is itself recorded.

Runtime: Isolation, Lifetime & Quotas

A3 Runtime design · Session isolation, lifetime and quota constraints

Three hard runtime constraints — isolation granularity, session lifetime and idle timeout — decide how business processes must be split. The baseline below follows mainstream enterprise agent-runtime specifications.

Session lifecycle state machine
  1. CreateRouted by session ID to a dedicated microVM
  2. ActiveServes requests, keeps files and memory
  3. IdleEntered after 15 minutes without requests (default)
  4. Terminate8-hour limit, explicit stop or failed health check
  5. DestroyMemory wiped, file system cleared
Runtime constraints and what they mean for architecture
ConstraintTypical specArchitectural implication
Isolation granularityA dedicated microVM per session, with its own compute, memory and file systemThe physical floor for multi-tenant and customer-data isolation
Maximum session lifetime8 hours (configurable); up to 14 days for persistent-compute typesLonger work must be designed to run in resumable segments (C12)
Idle timeout15 minutes by defaultWaiting for approval must suspend, not spin and hold resources
Session affinityRouted by session ID; without one, every call creates a new session with a cold startChannels must generate stable session IDs and pass them end to end
Per-session resourcesAbout 2 vCPU / 8 GB / 1 GB session storageLarge files belong in external object storage, not in-session scratch space
Concurrency and rate limitsThousands of active sessions per account; new-session rate is cappedPeaks need queuing and degradation strategies — infinite scale-out cannot be assumed
Session ID

Generated centrally by the channel (a sufficiently long UUID is recommended), bound to customer, channel and employee numbers and passed end to end as the primary key for observability and audit.

Sandbox boundary

Code-execution and browser sandboxes must have outbound access disabled or go through an allow-listed proxy; shell, file-system, browser and email tools must never share one unconstrained flow (C16).

Reworking long processes

Three kinds of break point — waiting for approval, external-interface timeouts and crossing a business day — must be checkpointed; on resume, check whether someone else has changed the business state.

Capacity and degradation

Plan capacity from peak traffic and concurrency limits; when the ceiling is hit, protect customer-facing channels first and queue internal batch work automatically.

The bottom line

Isolation granularity, session lifetime and idle timeout decide how a process is split — design the business steps around these three from the start instead of bending them at implementation time.

Selection questionnaire (ready for RFPs)

Is there a dedicated microVM per session? Are memory and files wiped when a session ends? Are maximum lifetime and idle timeout configurable? How is session affinity achieved? What are the concurrency and new-session rate limits?

Sourcespublic documentation of enterprise agent runtimes (session lifecycle, quota and rate-limit specifications); exact values depend on the platform version chosen — use the same yardsticks when building in-house.

Context Assembly Pipeline

A4 Context-assembly pipeline · Zones, budgets and caching strategy

Context is not string concatenation but an assembly pipeline with zones, budgets and cache semantics: zones run top to bottom from prefix to suffix, and the nearer the top, the more stable and cacheable they are.

Assembly zones and cache behavior (top to bottom = prefix to suffix)
  1. ① System instructionsRole · boundaries · output spec · tool-use guidanceStatic · long-lived cache hits
  2. ② Compliance red linesPermission boundaries · prohibited actions · suitability and script red linesFixed · never compressed
  3. ③ Tool definitionsMinimal scenario-specific tool set (mounted by business pack)Semi-static · invalidated only by version changes
  4. ④ Long-term memoryCustomer profile · past conclusions · experience cases (injected on demand)Dynamic · no cache hits
  5. ⑤ Retrieved knowledgePolicy clauses · graph results · external material (with sources)Dynamic · no cache hits
  6. ⑥ Session historyThis interaction’s history (compressed past the threshold; red lines excluded)Dynamic · no cache hits
Reference budget split (% of the context window)
Trigger threshold

Compress once context utilization reaches about 70%, rather than being truncated passively near the limit.

Compression ratios

Session history 3:1–5:1; tool output 10:1–20:1; clear already-consumed tool results first (the safest lightweight form).

Protected list

The compliance red-line zone, the current task goal, the open-issues list and the granted authorization scope are never compressed.

Logging compression

Every compression records an event marker, token counts before and after, and a summary of what was dropped, feeding observability and audit (see hard judgment six: compression erases safety constraints).

Why the platform must own it

If each application assembles its own context, red lines get left out, cache prefixes diverge and compression cannot be audited — any one of these is a production-incident-level failure.

Memory Service Design

A5 Memory service design · Namespaces, the write pipeline and compliant deletion

Memory is a standalone, permission-bound platform component: a namespace is both the isolation boundary and the permission boundary, and four compliance questions — can it be queried, deleted, audited, protected from poisoning — must be answered at design time.

Namespace design (isolation boundary = permission boundary)
Namespace templateMemory typeLifetimeAccess control
/customer/{customerId}/profile/Semantic memoryLong-term, across sessionsRelationship managers, by coverage relationship
/customer/{customerId}/preferences/Preference memoryLong-term, across sessionsAs above, plus customer self-service viewing and deletion
/customer/{customerId}/session/{sessionId}/summary/Summary memorySession-level, expires with TTLVisible only to staff serving this session
/business/{businessType}/experience/Experience case libraryLong-term, reused bank-wideBy business line, de-identified before storage
/employee/{employeeId}/workspace/Working contextShort-term, cleared at end of dayVisible only to the employee and their manager
Engineering facts of the write pipeline
Asynchronous and lagging

Extraction, consolidation and reflection all run asynchronously in the background, so long-term memory is eventually consistent; application logic must not rely on long-term memory written in the same turn.

Writes are rate-gated

Event writes for one subject’s session are usually limited to single-digit writes per second, which naturally serializes writes — consistent with read–write asymmetry.

Extraction has a compute budget

Long-term memory extraction is bounded by a token budget (on the order of 100,000 per minute); at peak times extraction queues, so backlog depth must be monitored.

Conflicts need a policy

When different sessions update the same fact, use “new value overrides, history is kept”, and require human confirmation before key fields such as risk grade change.

Queryable

Export every memory entry for a customer number with its source session, creation time and method; provide a customer-readable view.

Deletable

A deletion request cascades from original records → derived summaries → vector index → graph nodes → caches; where the vector index cannot prove deletion, fall back on “logical invalidation + retrieval masking + scheduled rebuilds” and issue a deletion certificate.

Auditable

Every memory carries a provenance chain (source session, tool, original record ID, generating model version); memories without provenance may not enter the decision path.

Poison-resistant

Check source credibility on every write; mark external content as untrusted so it cannot drive decisions directly; run regular poisoning checks (a handful of malicious documents can noticeably contaminate retrieval).

Key design point

Use the namespace directly as an authorization condition, so the identity system — not application code — decides who may read or write which memories. This is what lets multi-tenant isolation pass an audit.

Knowledge & Retrieval Layer

A6 Knowledge & retrieval layer · Three kinds of store, with permissions applied first

Vector stores, knowledge graphs and relational / key-value stores each have their strengths and work side by side; the single most important design rule in this layer is that permission filtering happens at retrieval time.

Division of labor among the three stores
DimensionVector storeKnowledge graphRelational / key-value
What it holdsPolicy clauses, product descriptions, case texts, meeting minutesCustomer relationships, equity and guarantee chains, beneficial owners, org structuresCustomer 360 structured fields, limit status, risk grades
Questions it answers best“Which rule applies here?” (single-hop facts)“Who is this company connected to?” (multi-hop relationships)“What is the current state?” (exact values)
Update semanticsAsynchronous re-embedding, with a risk of silent stalenessExtraction-based updates; mark facts invalid rather than deleting themTransactional updates, strongly consistent
Typical bank usesPolicy Q&A, script generation, case referenceRelated-party identification, guarantee-circle checks, group-customer look-throughReal-time decision data, limit and threshold checks
Main risksIncomplete recall or recall of superseded clausesGraph build quality and refresh lagConsistency with core systems, concurrent reads and writes
Permissions applied at retrieval (the most important design rule here)
  1. ① Parse the requestIdentify the customer number and business scenario
  2. ② Fetch user permissionsCoverage relationship and role authorization
  3. ③ Build the filterData classification + permission tags
  4. ④ Filtered retrievalCandidates are pruned during retrieval
  5. ⑤ Attach sourcesEvery conclusion links to its original
  6. ⑥ Generate and logRecord what was retrieved and what was used
Why permissions must come before retrieval

If you retrieve everything and mask at generation time, the model has already “seen” content it should not, and its reasoning and wording may leak it — after-the-fact masking is false security.

Why every conclusion needs a source

Supervisory inspections, credit-committee questioning and complaint handling all need to trace back to the original basis. In a bank, a conclusion without a source is no conclusion.

Why record “what was retrieved”

When a conclusion is wrong, you must tell “the material was not found” apart from “it was found but misused” — the fixes go in completely different directions.

Freshness and staleness

A change to a policy clause must trigger targeted rebuilds of the related vectors and graph, and frequently cited clauses get version checks so agents do not keep citing repealed rules.

Tool Gateway Design

A7 Tool gateway · Access, authentication, rate limiting and change control

The tool gateway is the only way out from agents to the bank’s systems: one place for access, authentication, tiered control and the audit trail.

The gateway’s six duties
Protocol adaptation

Turns in-house APIs, stored procedures and batch jobs into standard tool descriptions, hiding back-end differences.

Authentication and tokens

Verifies the inbound identity and exchanges it for short-lived, audience-bound downstream tokens; credentials sit in a vault and are rotated.

Tiered control

Enforces approval, dual review or outright refusal by the five action levels (Pass to Final sign-off), stored in tool metadata.

Rate limiting and circuit breaking

Limits by tool, by agent and by customer; abnormal call patterns trip a breaker and raise an alert.

Logging and replay

Records caller, parameters, result summary, latency and cost, forming a replayable call chain.

Change control

Tool descriptions and schema versions are pinned and hash-checked; any change triggers re-approval.

Outbound authorization sequence (standard authorization protocol)
← Swipe to see the full content →
Role12345
AgentCalls a tool · presents its workload tokenWaitsWaitsWaitsReceives the result · adds it to context
Tool gatewayVerifies the inbound identityChecks level and rate limitsGets a downstream tokenCalls the downstream systemRecords the call chain
Identity serviceVerifies signature and audienceToken exchangeVault hit or new token · returns a short-lived tokenRecords credential use
Business systemExecutes as the token authorizesReturns result and receipt

Mandatory node (human / governance)

Gold marks the controls that cannot be bypassed: level and rate-limit checks, and token exchange.

Key design one: no token pass-through

The gateway must never forward an upstream token to a downstream system as is; it must exchange it and bind it to an audience, or the permission boundary means nothing.

Key design two: a changed tool is a re-approved tool

A tool description is part of the model’s behavior — change the description and you change the behavior. This is the most overlooked attack surface (see the C16 incident list).

Key design three: the gateway is the only exit

Applications may not bypass the gateway to reach business systems; bypassing it means no rate limits, no audit trail and no tiered control.

Identity & Tiered Approval

A8 Identity & approval · Give every agent an employee ID and carve permissions into actions

The agent, the accountable owner and the person it acts for are bound together — no agent goes to work without all three; actions are graded into five levels by how irreversible their consequences are, and the gateway enforces the level.

Agent identity model (three-way binding)
Agent identity

Employee ID · role · capability list; creation, change and retirement are all recorded

Accountable owner

Direct manager · business line; the subject of performance review and accountability

Customer or employee represented

Whom it acts for; the scope and validity of that authorization

All three, or no deployment

Tiered action-approval matrix (ready to annex to internal policy)
← Swipe to see the full content →
LevelExample actionsTechnical controlHuman requirementAudit requirement
PassKnowledge retrieval, drafting, internal queriesRead-only permissions, outbound allow-listNoneCall chain logged, sampled checks
NotifyWriting working papers, routing internal ticketsWrite access limited to a namespaceSystem notice, reviewable afterwardsFull logging + content snapshot
ApproveSending externally, changing customer information, starting outbound callsExecutes only with an approval tokenHandler confirms and records a reasonApprover, reason and time recorded
Dual controlAccounting entries, limit adjustments, parameter changesSegregated permissions + second confirmationTwo-person review, with doer and reviewer separatedBoth identities and any differences recorded
Final sign-offFund payments, credit approval, regulatory reporting and disclosureMandatory human gate that cannot be bypassedSigned by an authorized officer; major matters authorized by committeeFull decision basis + versions + sign-off chain
Levels are carved into tools

The approval level is tool metadata enforced by the gateway, not left to the prompt — prompts can be bypassed, metadata cannot.

Permissions follow the role, not the person

Agents receive least-privilege permissions by role; when a role changes, the old permissions must be removed from running instances immediately so that permissions do not pile up.

More autonomy needs evidence

Each time an agent is allowed one more level of action (for example from Notify to Approve, then to Dual control), three documents are required: a pass^k report that meets the bar, shadow-run comparison results and a red-team report.

SourcesNFRA guidance Jinfa [2026] No. 8 (human oversight at key points of high-risk applications; whoever uses it is responsible); enterprise agent-identity product capabilities (identity templates, conditional access, lifecycle governance).

Security in Depth

A9 Security in depth · From a risk list to controls, item by item

Each of the ten top agent risks maps to platform controls and an owning module; whether security works is proven by continuous red teaming, not a one-off benchmark score.

Ten agent risks and the platform controls for each
RiskPlatform controlsOwning module
Behavior hijacking / goal tamperingFixed red-line injection + human plan-confirmation point + goal-consistency checksContext service / approval center
Prompt injection (incl. instructions hidden in tool descriptions)Untrusted-content isolation + capability-label adjudication + hash-pinned tool descriptions with change approvalSecurity service / tool gateway
Tool misuseMinimal tool sets + tiered approval + bans on dangerous tool combinationsTool gateway
Identity and privilege abuseAgent employee IDs + token exchange with audience binding + namespace authorizationIdentity service
Insufficient guardrails and sandboxingA microVM per session + outbound allow-list + no network in code sandboxesRuntime
Sensitive-information leakagePermissions applied at retrieval + output masking + content-capture switchesKnowledge service / observability
Data and memory poisoningSource-credibility checks + external content marked untrusted + regular poisoning checksMemory service
Resource exhaustionThree-dimensional rate limits + cost quotas + circuit breaking and degradationTool gateway / runtime
Supply-chain riskPinned dependencies + source allow-lists + typosquatting monitoringDevelopment process
Over-trust and blurred accountabilityMandatory human sign-off for Final sign-off actions + three-way identity binding + decision basis recordedApproval center / observability
Red teaming and verification
Frequency

Routine red teaming every quarter, plus a dedicated test whenever the model version, harness, tool set or prompt template changes.

Method

Evaluations must be attacker-aware and allow repeated attempts. Public tests show generic injection succeeds about 11% of the time, targeted custom attacks up to 81%, and about 80% on average with repeated attempts.

Conclusions

A single benchmark run is not a security conclusion; reports must give the attack success rate, average number of attempts and the distribution of failure modes.

Closing the loop

Every successful attack must become a control or an evaluation case in the regression suite — otherwise red teaming is a drill, not governance.

Blast-radius isolation (a hard ban)

Never put shell execution, file-system read/write, browser access and outbound email in one unconstrained workflow. Any combination of the four can be chained by injected instructions into a complete data-exfiltration path; where a combination is truly needed, split it across different agents and permission domains with a human gate in between.

Mapping to the three lines of defense

The first line embeds guardrails and approval gates in the business process; the second runs red-team exercises, the policy engine and risk measurement; the third checks the completeness of behavior logs and the effectiveness of controls — agent security is not a new system, it is a new subject for the existing one.

SourcesOWASP Top 10 for Agentic Applications 2026 (2025-12-09); NIST CAISI agent-hijacking evaluation (629 test cases); compiled MCP-ecosystem security incidents.

Observability & Audit Trail

A10 Observability & audit · Four-level tracing and the fields you must record

Observability is the technical form of the supervisory requirement that every operation leaves a trace and can be checked afterwards: four tracing levels you can drill into, and eight classes of audit field produced automatically with every call.

Four-level trace structure (drill down level by level)
  1. SessionOne complete customer interaction; keyed by the channel-generated session ID and bound to customer and employee numbersAudit primary key
  2. TaskOne goal being achieved; records the goal, acceptance criteria, final state and durationGoal and acceptance
  3. StepAtomic steps such as planning, retrieval, generation and approval; records an input summary, model version and durationAtomic steps
  4. Tool callEvery external call; records tool name, call ID, parameter summary, result status and downstream spansDownstream spans
Mandatory audit-log fields (eight classes)
Identity triple

Agent employee ID + accountable owner + customer or employee represented, and the authorization scope in force

Decision basis

Knowledge entries retrieved and their sources, memory entries injected, rule versions in force

Execution record

Tool-call sequence, parameter summaries, result status, downstream receipt IDs

Human touchpoints

Approver, time of approval, reason, and whether and how the output was changed

Model and configuration

Model name and version, prompt-template version, tool-set version, harness version

Resources and cost

Input / output tokens, cache reads, latency, time to first token, estimated cost

Exceptions and interceptions

Guardrail hits, rate-limit trips, injection-detection alerts, compression-event markers

Content labeling

Whether external output carries the required AI-generated-content labels (explicit and implicit)

Instrument first

Adding instrumentation after go-live means history cannot be traced; design the trace structure and audit fields together, once.

Content capture needs a switch

Raw prompts and outputs are sensitive; they need their own compliance switch and masking — observability does not mean keeping every original text.

The trail must be replayable

Inspections and complaint handling need to reconstruct what the agent saw, what it relied on and who approved it — not just the outcome.

Regulatory alignment

NFRA guidance Jinfa [2026] No. 8 requires a complete record of the input, decision basis and result of every agent tool call; the Measures for Labeling AI-Generated and Synthetic Content (effective 2025-09-01) require explicit and implicit labels on generated content released externally — both must be built into the instrumentation design, not patched in after launch.

A one-line test

If a supervisor asks “why did the agent decide this case this way?”, can you reconstruct within five minutes the material it saw, the rules it applied and the approvals it went through? If yes, it passes; if not, it cannot go into production.

Evaluation & Metrics System

A11 Evaluation & metrics · Go-live gates and continuous monitoring

Going live is a staged pipeline with a pass condition at every stage; the gates differ by business risk level, and the evaluation set is an asset as important as the code.

Go-live pipeline and pass conditions
  1. Offline evaluationStratified evaluation set, pass^k meets the bar
  2. Shadow runOne full business cycle; every disagreement reviewed
  3. Staged rolloutOpened by percentage, with automatic rollback thresholds
  4. Full launchContinuous sampling, drift detection
  5. Online monitoringFour metric families on one screen; automatic degradation on anomalies
Suggested go-live gates (by business risk level)
← Swipe to see the full content →
Risk levelTypical scenariospass^k gateCost and latency constraintsHuman requirement
Low (Pass, Notify)Knowledge Q&A, drafting, internal searchpass^3 ≥ 90%Cost per task ≤ budget baselinePost-hoc sampling ≥ 5%
Medium (Approve)Outbound-call scripts, suggested customer-information changes, ticket handlingpass^5 ≥ 90%P95 latency ≤ service standardHandler confirms every case
High (Dual control)Suggested accounting entries, suggested limit adjustments, classifying suspicious transactionspass^8 ≥ 95%Both cost and latency constrainedTwo-person review + differences recorded
Very high (Final sign-off)Fund payments, credit approval, regulatory reporting and disclosureAutomation is not the goalNot applicableHuman final sign-off; the agent only supplies the basis
Evaluation samples needed per group to detect an improvement
Detect +2 pct6,300
Detect +4 pct1,580
Detect +7 pct480
Detect +12 pct165

Two-sample test of proportions, 80% power / 95% confidence.

Small samples prove no improvement

With 100 samples and an 80% pass rate, the confidence interval is roughly 72%–88% — a small set cannot support a claim of “a few points better”.

Use paired designs

A paired design (the same samples before and after) cuts the sample requirement by about an order of magnitude.

Always report by stratum

Report by query type so the overall number cannot hide a sharp deterioration in one kind of scenario.

Core position

The evaluation set is an asset as important as code — built per scenario, versioned, owned by named people, and re-run as a mandatory regression after every model or harness change.

Deployment Topology & Capacity Planning

A12 Deployment topology & capacity cost · Getting the numbers right

Keeping data inside the bank comes down to a three-domain hybrid deployment: sensitive data and every write stay in the production domain, inference runs in the in-bank AI domain, and only de-identified intents and entities ever leave.

Hybrid deployment topology (how data stays in the bank)
In-bank production domain

Core business systems · customer data · tool gateway · approval center · audit logs

All sensitive data and every write
De-identification gate
  1. Local transcription → tokenization
  2. Small-model residue detection
  3. Only “intent + entities” sent out
  4. Results restored inside the bank
In-bank AI domain

Private large-model cluster · small-model matrix · memory and knowledge stores · sandboxed execution

Inference on de-identified data, inside the bank
Controlled external domain

External data interfaces · allow-listed external models (receiving only de-identified intents and entities)

Sent out only after de-identification and filtering
Capacity estimation in four steps
StepMethodExample
① Estimate GPU memoryWeight memory ≈ parameters × bytes per parameter; total ≈ weights × 1.3–2.0 (including KV cache and activations)A 70B model at half precision needs about 140 GB for weights, i.e. 2–4 × 80 GB GPUs
② Estimate throughputConvert target concurrency and average output length into tokens per second, against measured single-GPU throughputReserve extra KV cache for long contexts under high concurrency
③ Set utilizationDerive target utilization from the service level: a low time-to-first-token target lowers the utilization ceilingInteractive workloads usually run at 50–70%, never full
④ Count GPUsGPUs ≈ request rate × average service time ÷ target utilization, plus redundancy and disaster recoveryLeave headroom for peaks and parallel staged rollouts
Four cost levers (in order of payoff)
Cache hit rate

A stable static prefix keeps hitting the cache, and hits cost about an order of magnitude less than misses — the first lever.

Routing and cascades

Send simple requests to small models; at equal quality cost can fall sharply (the range is wide, so measure on your own traffic).

Leaner context

Tool-set curation and compression thresholds directly cut the input tokens of every call.

Batching and quantization

Continuous batching raises throughput; 8-bit quantization saves memory almost losslessly, 4-bit saves more but its accuracy must be assessed.

Sourcescapacity formulas and quantization trade-offs from public engineering practice (values are orders of magnitude and must be replaced by in-house POC measurements); the privacy-pipeline approach draws on public disclosures by leading international banks.

Availability, Rollout & Continuous Improvement

A13 Availability, rollout & continuous improvement · Survive failures, control changes, keep the loop turning

Production-ready means more than running: the platform must survive failures, keep changes under control and get better through a monthly improvement loop.

Four layers of availability design
Model layer

Mutual backup across models + automatic degradation: if the primary model fails, fall back to a standby model or the rules engine — reduced capability is acceptable, an outage is not.

Session layer

Session state persisted externally so another instance can recover it after a failure (sessions cannot be live-migrated, so design for replay rather than resumption).

Data layer

Memory and knowledge stores run active-active within the city; because long-term memory extraction is asynchronous, the recovery point objective cannot be zero — design for replay, not resumption.

Business layer

Every agent process keeps a human fallback channel; if any step is unavailable, work falls back seamlessly to people without the customer noticing.

Change control: what counts as a “change”
ChangeControl requirement
Model versionMust re-test
Harness versionMust re-test
Prompt templatesMust re-test
Tool set and descriptionsMust re-test
Knowledge-base rebuildMust re-test
Memory policyMust re-test
Approval-level adjustmentNeeds approval
Routing policyNeeds monitoring

Changes marked “must re-test” go through offline evaluation and a shadow run again; they cannot enter staged rollout until they pass.

Continuous-improvement loop (runs every month)
  1. CollectHuman corrections · rejection reasons · complaints and errors
  2. AttributeClassify by failure mode, locate the stage
  3. CaptureWrite to the experience library, add evaluation cases
  4. FixAdjust prompts, tools or knowledge, or trigger fine-tuning
  5. RegressValidate on the evaluation set, compare in shadow
  6. Roll outWiden the staged rollout, or roll back
The core insight of change control

An agent’s behavior is jointly determined by the model, harness, prompts, tools and knowledge — change any one and you have a different system. That is why every “must re-test” change has to pass offline evaluation and a shadow run before staged rollout.

Whether the loop turns depends on step one

Human corrections and rejection reasons must be captured as structured tags, not free-text notes. This single choice decides whether continuous improvement is data-driven or gut-driven.

End-to-End Trace: Customer Service

A14 End-to-end trace one · A customer-service inquiry, step by step

A customer inquiry from start to answer: time runs left to right, responsibilities top to bottom, and gold cells are the human and governance nodes that can never be skipped.

Customer-service inquiry trace
← Swipe to see the full content →
Role123456
Customer / channelStarts an inquiry · session ID generatedReceives the answer · with content labels
Security & identityAuthentication · injection detectionPermissions loadedPermission-filtered retrievalOutput masking and script-compliance check
Context serviceSix zones assembled · red lines injected as fixedMemory and knowledge injectedCompression logged (if triggered)
Orchestration & modelsIntent recognition · small-model triageHard cases escalated to a large modelAnswer generated · with sourcesReviewed by an independent instance
Tools & dataAccount / product lookup · gateway-authorizedMemory written (asynchronously)
People & governanceLow confidence → human agentHigh-risk product → mandatory human handoverFull trace recorded for audit

Mandatory node (human / governance)

Gold marks the three mandatory nodes: low-confidence handover, mandatory handover for high-risk products, and labeling of generated content.

Three mandatory nodes

① Low confidence or unclear intent → hand over to a human agent, counted in the human-escalation rate; ② high-risk product recommendations → mandatory human handover, the agent may not perform suitability matching; ③ external output → explicit and implicit AI-generated-content labels as required.

Four observability anchors

① The session ID runs through the whole chain as the audit key; ② every model call records tokens and cache reads for costing; ③ every tool call records its call ID and downstream receipt; ④ compression events are tagged separately to diagnose “the rule got summarized away” problems.

Reading the diagram

The gold cells are not there to slow things down — they are what lets this chain go into production and pass an audit.

End-to-End Trace: Corporate Credit Review

A15 End-to-end trace two · A corporate credit review, step by step

A corporate credit review: the plan is confirmed up front, reads run in parallel while one writer writes, disagreements are flagged in red and final sign-off cannot be bypassed — the agents do the preparation, authorized people keep the decision.

Corporate credit review trace
← Swipe to see the full content →
Role123456
Relationship managerSubmits the file · starts the reviewConfirms the investigation planSupplies more materialReceives the review report
Planning & orchestrationParses task boundaries · registers acceptance criteriaGenerates the investigation plan and milestonesFans out parallel reads (five sub-agents)Consolidates and re-plansSingle writer · produces the reportArchives the case file
Sub-agentsDocuments / financials / credit bureau / related parties / industry in parallelEach returns a structured summary (conclusion + evidence + confidence)
Knowledge & toolsGraph look-through of related parties · external data checksFinancial cross-checks · sandboxed calculationsPolicy-clause retrieval · with sources
Verification & checksCross-validation · disagreements flagged in redIndependent review instance · scored against a rubric
People & governancePermission and identity checksHuman plan confirmationReviewer checks and editsCredit committee final sign-off

Mandatory node (human / governance)

Gold marks where regulatory requirements land: human plan confirmation, the single writer, reviewer checks and credit-committee sign-off.

Confirm the plan up front

Let the relationship manager approve “what to check and how deep” once before execution — far cheaper than rework afterwards.

Parallel reads, a single writer

The five sub-agents only read; one writer produces the report, so the audit trail has a single anchor.

Flag disagreements, don’t auto-resolve them

Cross-validation is valuable because it exposes disagreements; the reviewer keeps the power to decide.

Final sign-off cannot be bypassed

Credit approval is the highest risk level; the agent supplies only the basis and the draft.

Regulatory mapping

Due diligence must combine on-site and off-site work and produce a written report; loan origination and review are separated with tiered approval; key points of high-risk applications need human oversight — these three requirements land on plan confirmation, the single writer and final sign-off, in line with the Measures for the Administration of Working Capital Loans (NFRA Order No. 2 of 2024) and NFRA guidance Jinfa [2026] No. 8.

Related regulations

  1. [33] NFRA guidance Jinfa [2026] No. 8 on the safe development and application of AI in banking and insurance, 2026-06-18, 32 provisions.
  2. [34] Implementation Plan for the High-Quality Development of Digital Finance in Banking and Insurance, NFRA General Office, 2025-12-22, 33 tasks.
  3. [35] Measures for the Administration of Working Capital Loans (NFRA Order No. 2 of 2024), effective 2024-07-01.
  4. [36] Measures for the Administration of Fixed Asset Loans (NFRA Order No. 1 of 2024), effective 2024-07-01.
  5. [37] Measures for the Risk Classification of Financial Assets of Commercial Banks (Order No. 1 of 2023), effective 2023-07-01.
  6. [38] Anti-Money Laundering Law of the People’s Republic of China (2024 revision), effective 2025-01-01.
  7. [39] Measures for the Administration of Reporting of Large-Value and Suspicious Transactions by Financial Institutions (2025 revision), effective 2025-12-01.
  8. [40] Measures for Customer Due Diligence and the Retention of Customer Identity Information and Transaction Records by Financial Institutions, effective 2026-01-01.
  9. [41] Measures for the Suitability Management of Financial Institutions’ Products (NFRA Order No. 7 of 2025), effective 2026-02-01.
  10. [42] Measures for the Handling of Consumer Complaints in Banking and Insurance (CBIRC Order No. 3 of 2020).
  11. [43] Measures for the Protection of Consumer Rights and Interests by Banking and Insurance Institutions (CBIRC Order No. 9 of 2022), effective 2023-03-01.
  12. [44] CBIRC Notice Yinbaojiangui [2022] No. 13 on further promoting the well-regulated and sound development of credit card business.
  13. [45] National standard: Internet finance — Personal online consumer credit — Risk-control guidelines for post-loan collection, published 2025-02-28.
  14. [46] Measures for the Administration of Banking and Insurance Regulatory Statistics (CBIRC Order No. 10 of 2022), effective 2023-02-01.
  15. [47] Measures for Labeling AI-Generated and Synthetic Content (four ministries), effective 2025-09-01.
  16. [48] Interim Measures for the Management of Generative AI Services, effective 2023-08-15; UCP600 Articles 4, 5, 14 and 16.
Get started

Start with one business question. Let SAM deliver a model you can sign off.

Whether you are in banking, FMCG and marketing, pharmaceuticals or another industry, try SAM online first — or talk to our modeling experts about your use case.