Building bank-grade software with AI agents: the controls that make it auditable
AI agents can write code for regulated financial software if the controls sit around them: written rules, tests first, independent review, human approval of the exact version, and an audit trail of who did what. A control map and an honest account of what is in place.
- By
- Finray
- Published
- Updated
- Reading time
- 10 min
Can AI agents build software for regulated finance, and under what conditions?
AI agents can build software for regulated finance, but only when the controls that make the work auditable are built around the agents rather than delegated to them. A bank-grade process lets agents do bounded work inside written rules, and it keeps scope, review, approval and evidence with people and systems that the model cannot override. The question a supervisor or auditor will ask is not “was AI used?” but “who decided, what was checked, and can you prove it?”
This note sets out an example control map for that question, the outside frameworks it connects to, and what Finray actually does today. It is a description of mechanisms, not a promise that errors cannot happen.
What new risk do AI agents add to software delivery?
Agents add speed and breadth, and they add a specific failure mode: an agent can take an action nobody intended because its output was wrong, ambiguous or manipulated. OWASP’s guidance for LLM applications calls this excessive agency and traces it to three root causes: excessive functionality (tools the task does not need), excessive permissions (rights beyond the task) and excessive autonomy (high-impact actions with no independent check).
The triggers named there include hallucination and prompt injection. Both are well-documented, everyday risks with current models, not exotic edge cases. So the safe design assumption is that any single agent output may be wrong, and the process must be built so that a wrong output is caught before it matters.
OWASP’s mitigations map cleanly onto delivery pipelines: minimise tools, minimise permissions, require human approval of high-impact actions, and enforce authorisation in the downstream system rather than trusting the model. Logging and monitoring do not prevent the problem, but they limit the damage and help investigation.
What do regulators and standards already expect of software change?
None of the frameworks below was written for AI agents, and none of them says that AI must not write code. They set expectations about control, which apply whoever or whatever writes the change.
- DORA (Regulation (EU) 2022/2554), Article 9(4). Financial entities under DORA’s full framework (Article 16 sets a simplified one for some smaller entities) need documented ICT security policies, including access control that limits access to what legitimate, approved activities require (point c) and risk-based change management procedures so that changes to software, systems and security settings are recorded, tested, assessed, approved, implemented and verified in a controlled manner (point e).
- EBA guidelines on ICT and security risk management. From 2020 they set ICT governance, information security and change-management expectations for credit institutions, investment firms and payment service providers. After DORA set harmonised ICT risk rules, the EBA cut them back: the amended guidelines, applicable from 20 May 2025, keep only rules for payment service providers on relationship management with payment service users. For software change, the reference point is now DORA.
- FINMA Circular 2023/1, operational risks and resilience, for banks. In force since 1 January 2024, with transitional provisions for operational resilience. FINMA says the revision clarifies supervisory practice on operational risk, in particular ICT, critical data and cyber risks. We do not read it as prescribing a development method; we read it as raising the expectation that control can be evidenced.
- NIST SP 800-218, the Secure Software Development Framework. A core set of high-level practices that can be integrated into any development lifecycle, to reduce the vulnerabilities in released software, limit the impact of those not caught, and address root causes. It also gives suppliers and buyers a shared vocabulary.
- ISO/IEC 27001:2022. The management-system standard whose Annex A lists controls such as segregation of duties and change management. Certification against it covers an organisation’s management system, not a product.
- Internal control standards. The GAO Green Book, which follows the COSO approach, tells management to segregate incompatible duties and, where that is impractical, to add alternative controls, and it cautions that collusion can defeat segregation.
What is a control map for agent-built software?
The map below links each risk to a control that sits outside the model and to the person or system that owns the control. It is a thinking tool; each organisation will adapt it.
| Risk | Control outside the model | Typical owner |
|---|---|---|
| Agent changes more than intended | Written brief: scope, files, what must not change, how done is proven | Task owner |
| Wrong behaviour passes unnoticed | Failing test first, then implementation | Author and reviewer |
| Vulnerable or leaked material enters the code | Secret scanning, dependency and vulnerability scanning, static analysis on every change | Engineering |
| Author’s blind spots shared by reviewer | Independent review by a different AI model family | Review process |
| Fix does not fix | Findings go back to the same workspace and are re-checked in the next review | Review process |
| Works in review, fails in operation | Live test on a dedicated environment with synthetic data | Test owner |
| Reviewed code differs from merged code | Human approval of the exact version reviewed and tested; any later change restarts review | Approving person |
| Unknown who did what | Audit trail: person or model, action, version | Platform |
| Agent touches production or real data | Approved tools only; synthetic data only; no production access for browser-driving agents | Security |
| Tool upgrade changes behaviour silently | Upgrades that add hooks, network calls or telemetry need re-approval | Security |
What does the path of one change look like?
This is the sequence a change to Finray’s product code follows. Each step produces evidence, and each hands over to the next only when its output exists.
- Task and plan recorded in Pync, Finray’s project system, with acceptance criteria.
- A written brief: scope, files, what must not change, and how completion is proven.
- A failing test first, then the implementation, carried out by an AI agent working in an isolated workspace.
- Automated checks on every change to product code: secret scanning, dependency and vulnerability scanning, static analysis and tests.
- Independent review by a different AI model family than the one that wrote the change. Findings return to the same workspace and are re-checked in the next review.
- Live test on a dedicated test environment with synthetic data.
- Human approval of the exact version that was reviewed and tested. Any later change restarts review.
- Merge with a complete audit trail: who, person or model, did what, at which version.
The order matters. Tests come before code so that “done” is defined before the agent starts. Review comes before approval so that the person approving sees evidence, not just a diff. Approval binds to the exact version, which closes the common gap where something changes after review.
What is a worked example?
Illustrative example with synthetic values. A change request: show a validation message when a user saves an account with an empty name, nothing else.
The brief names the file, forbids changes to posting logic, and says the work is done when a test that saves an account named "" fails before the change and passes after it, and the existing tests still pass. The agent first writes the failing test. It then implements the change in an isolated workspace. The automated checks run; the dependency scan is clean, and the static analysis flags an unused variable, which is fixed.
The reviewing model, from a different family, notices that a name made only of spaces still passes, a case the brief did not mention. The finding goes back to the same workspace, is fixed, and is re-checked in the next review round. The test environment run uses synthetic data only. A person then approves the exact version that passed those steps, and the merge is recorded with who or what did each step.
Nobody in this story trusted the agent’s first answer, and nobody needed to.
Which rules apply to AI tools in development?
Process controls need rules about the tools themselves. Finray’s internal guidance, described here rather than quoted, covers:
- An allow-list of approved AI tools, plugins and extensions; anything not listed is not approved.
- One reviewed source for shared AI skills; no local, unreviewed variants.
- Browser-driving agents only against local, development or test systems, never production.
- Re-approval of tool upgrades that add hooks, network calls or telemetry.
- Synthetic data only: customer data, personal data and secrets are never given to an AI tool.
- Attribution: every action stays attributable to the real person or model that did it.
These are segregation-of-duties and least-privilege ideas applied to agents. See segregation of duties for the underlying design principle.
How do the same ideas show up in financial software?
The discipline for building software and the discipline inside financial software are the same pattern: separate maker from approver, record evidence, and keep the evidence checkable.
- Approval by a second person. The four-eyes principle is what Step 7 applies to code. In Corebanq Accounting it applies to postings and bills by default.
- Evidence that can be verified. A hash-chained journal records entries in an append-only sequence whose chain can be re-checked. It is tamper-evident, not tamper-proof.
- Safe retries. In payment systems, idempotency is the design property that keeps a repeated command, whether from a timeout or an agent that retries, from being applied twice. This note makes no claim about how any Finray product implements it.
- Outside the model. In Corebanq Accounting, approvals, the rule that a maker cannot approve their own change, and the audit trail are enforced by the system, not by the model. AI proposals need a person’s confirmation and carry no posting authority.
What is in place at Finray today, and what is not?
Today. The path above is how Finray changes its product code, with people owning every decision and agents doing bounded, reviewable work. Finray Technologies Ltd holds ISO/IEC 27001:2022 certification (NQA, UKAS-accredited, certificate No. 215646, valid 21 October 2025 to 21 October 2028), and the 2026 surveillance audit was passed. This is a company-level information security management system certification, not a product certification; the certificate is shown on the company page.
In Corebanq Accounting, postings and bills need a second person’s approval by default (a company policy whose change is itself approved), a maker cannot approve their own change (enforced in the database), and the journal, audit log and access log are append-only and hash-chained with a verification check. AI there is off by default. The assistant runs on a model the customer hosts locally and has no posting authority, cloud document reading stays off unless the company turns it on, and AI proposals need a person’s confirmation. Corebanq Accounting offers chat access from Claude or ChatGPT. Questions are read-only. Documents arrive as drafts; a person approves before anything posts. Chat access is off by default and activated per company.
Not claimed. We do not claim that software built this way is free of defects, that every change in every product always needs four eyes, that the journal is tamper-proof or externally anchored, or that agents act inside the ledger. We do not claim conformance to the NIST SSDF or compliance on a customer’s behalf with DORA, the EBA guidelines or the FINMA circular; those are frameworks a regulated customer assesses against its own obligations. This page is not legal or regulatory advice.
For the full description of the process and a conversation about it, see how we build, the Corebanq Accounting platform page and Corebanq.
What should a regulated buyer ask any vendor that uses AI agents?
- Who approves the exact version that ships, and is that a named person?
- Is review independent of the author, and how is independence achieved?
- Which AI tools are approved, who approves upgrades, and what do they see?
- Is customer data kept away from AI tools in development?
- Can the vendor show, for a given release, who or what did each step?
- What happens when a check fails; does the change restart review?
- Which controls live outside the model, and who can change them?
Vendors that answer these concretely, with evidence, give an auditor something to test. Answers that rest on “the AI is very accurate” do not.
Frequently asked questions
- Can AI agents write code for a bank or payments platform?
- They can, if the surrounding controls are strong enough that a wrong or manipulated output is caught before it reaches production. That means written scope, tests that fail first, automated scanning, independent review, a test environment with synthetic data, human approval of the exact reviewed version, and an audit trail. The agent is a contributor inside those controls, not a replacement for them.
- Who is accountable when an AI agent writes the change?
- People, not the model. The person who approves the exact version that is merged is accountable for that decision, and under DORA the management body bears ultimate responsibility for the firm's ICT risk (Article 5(2)(a)). The audit trail should also show which person or model did each step, so that every action is attributable and reviewable later.
- Why use a different AI model to review the code?
- A reviewer built the same way as the author tends to share its blind spots. Using a different model family for review is a low-cost form of independence. It does not replace human approval; it makes the evidence that reaches the human better.
- What does a regulator expect of change management in software?
- DORA Article 9(4)(e) expects documented, risk-based procedures so that changes to software, systems and security settings are recorded, tested, assessed, approved, implemented and verified in a controlled manner. Other frameworks address related ground in their own terms: ISO/IEC 27001:2022 lists change management as an Annex A control, and the NIST SSDF sets out secure development practices.
- Does ISO/IEC 27001 certification mean the product is certified?
- No. ISO/IEC 27001 certification covers an organisation's information security management system. It says the company runs a managed process, not that any particular product has been certified.
- What stays outside the AI in an agent-built system?
- Identity, permissions, limits, approvals, secrets and the audit trail. These are enforced by systems and people the model cannot override, so that a hallucination or a prompt injection has to get past a system check and a person before it can become a production change.
Sources
- NIST SP 800-218, Secure Software Development Framework (SSDF) Version 1.1
- OWASP Top 10 for LLM Applications 2025, LLM06 Excessive Agency
- Regulation (EU) 2022/2554 (DORA), Article 9(4) ICT security policies, access control, change management
- FINMA news release on Circular 2023/1 Operational risks and resilience - banks
- EBA, Guidelines on ICT and security risk management (EBA/GL/2019/04 as amended by EBA/GL/2025/02, applicable from 20 May 2025)
- GAO, Standards for Internal Control in the Federal Government (Green Book), 2025 revision (GAO-25-107721), Principle 10
- ISO/IEC 27001:2022, information security management systems
Discuss how this applies to your institution with the Finray team.
Book a briefing