Why AI Governance Is Becoming Core Bank Infrastructure

For banks, the key question about artificial intelligence is shifting from what a model can do to whether the institution can show that it stayed in control. That means tracking data access, assigning responsibility when automated systems act, and reconstructing how a recommendation or action came about. As AI moves beyond trials and into financial workflows, those capabilities become part of the operating architecture.

Nikhil Goel, a founding engineer and engineering manager at cloud data security and AI governance company Theom.ai, has worked on security infrastructure used in major U.S. financial institutions. He says the issue has moved quickly from theory to implementation. “Two years ago, ‘how do you secure an LLM’ was a theoretical question,” Goel said. “Today it is something every enterprise I work with is actively trying to solve.”

From principles to operational proof

Recent work by Treasury and the National Institute of Standards and Technology (NIST) reflects that transition. In February, Treasury released a Financial Services AI Risk Management Framework tailored to financial institutions. Two months later, NIST began developing a separate AI Risk Management Framework profile for critical infrastructure. NIST is also seeking input on a draft framework for testing, evaluating, verifying and validating AI systems; comments are open through October 6.

The initiatives point toward a new phase of enterprise AI. Early adoption focused on capability: what could a model produce? The harder question now is whether an organization can demonstrate that the system behaved as intended in the conditions where it was used.

That standard fits the financial sector, where institutions already manage operational, cybersecurity, privacy, model-risk and compliance controls. AI adds complexity to those responsibilities. An assistant could draw on several internal systems to summarize information. An agent could interact with those systems for an employee. A model might help flag suspicious transactions or support credit, fraud or compliance work. Each application may be useful, but each raises questions about access, identity, explainability and accountability.

Treasury’s framework adapts NIST’s existing AI Risk Management Framework to the operating realities of financial institutions. It emphasizes accountability, transparency, resilience and risk management throughout an AI system’s lifecycle. The aim is to give institutions a way to manage deployment, not to rule out adoption.

Identity and data lineage are practical controls

Conventional enterprise security is built around people and applications: a person signs in, receives defined permissions, and leaves a record of activity. AI agents complicate that setup because they can retrieve information and take actions across systems at speed. Six banks have recently published voluntary agentic commerce principles addressing trust, security and accountability as agents begin to influence transactions.

The design questions are concrete. Should an agent inherit all the permissions of the employee it serves? Do restrictions on source data still apply after the agent combines that data into a new output? If the agent acts without a person clicking each step, whose identity appears in the audit record?

“An agent shouldn't get a blanket copy of an employee's permissions,” Goel said. “It should carry that person's identity on every request, so the access decision and the audit record both point back to an accountable human.”

Goel’s work includes identity-aware controls for AI systems and data lineage: tracing where information came from, how it changed and where it went. Company materials name him as lead inventor on two patent applications, one concerning identity-aware security for LLMs and AI agents, and the other tracking data lineage through transformations in systems that do not naturally expose those changes.

Lineage matters because information rarely stays in one place. A customer record may begin in a regulated system, move to a cloud warehouse, pass through another application and then become context for an AI model. Institutions may still need to know what restrictions apply after those transformations. If the trail is lost, it becomes harder to establish how data was used.

Identity has a similar role. An AI system acting for an employee may work faster and touch more information than that employee would handle manually. Preserving the limits of the employee’s authority is therefore central to controlling the agent’s actions. Governance depends on connecting permissions, data context and an auditable record.

Testing at production scale

NIST’s developing critical-infrastructure profile is meant to translate broad principles for trustworthy AI into requirements that developers, vendors and internal teams can apply. Its concept note highlights guardrails for autonomous cybersecurity agents, traceable and auditable recommendations, human oversight, and reliability when operating conditions change.

NIST’s draft TEVV-Athlon framework puts more emphasis on testing. TEVV stands for testing, evaluation, verification and validation. The practical question is whether a system is fit for the environment where it will run—not simply whether it performed well in a demonstration. Banks already expect critical systems to withstand real transaction volumes and failure conditions, and to produce evidence that can withstand scrutiny. AI controls are beginning to face a similar bar.

Goel has seen how the distance between a working product and a production-ready system can widen at scale. During an earlier deployment at a large financial institution, Theom encountered data volumes far beyond those of its earlier customers. Queries that had worked at smaller scale began to time out. Company data says the processing engine had handled roughly 100,000 queries per day; as larger institutions joined the platform, it had to support more than 500 million daily queries while reducing average response times from roughly two seconds to about 200 milliseconds. Goel led a redesign of the query path, data partitioning and distributed processing architecture.

That was a scaling challenge, but it also illustrates a broader risk-management point: a control that works in a demonstration may not hold up across a bank’s systems and data volumes. A tool that identifies sensitive information in a small environment could behave differently across billions of data interactions. Access rules can break down if identity context disappears between systems, and test results may not reflect behavior under changed data or operating conditions.

Governance must connect existing teams

Financial institutions have established risk functions and experience assessing third-party technology, models, cyber threats and operational resilience. AI risk is likely to be handled through those disciplines, rather than by a standalone AI team. Treasury’s framework is designed to suit institutions at different sizes and stages of AI adoption while giving technical, legal, compliance and business teams a shared vocabulary.

That common language matters because each group sees a different part of the system. Data scientists assess model performance; security engineers focus on access boundaries; compliance teams need evidence that restricted information was handled properly; business leaders look at productivity. Effective controls have to connect those concerns.

There is a commercial tension as well. Banks want to capture productivity gains without letting risk go unmanaged, while excessive friction can keep useful systems in pilot programs. The institutions best placed to deploy AI responsibly will make governance part of the infrastructure: locating sensitive data, carrying its context through transformations, limiting what automated systems can do, and retaining evidence of what happened.

The frameworks are still developing. NIST’s critical-infrastructure profile is in progress, and its TEVV-Athlon framework remains a draft. Treasury’s resource is intended as a practical risk-management guide, not a replacement for existing requirements. Even so, the direction is clear: as AI enters systems that move money, flag fraud, assess risk and handle sensitive customer information, the ability to test and account for its actions will shape how far institutions can put it to work.