Enterprise AI in banking is stuck in a persistent gap between prototype and production. Fast prototypes now stand up in hours, and moving them into live systems still takes months. The sector's industry-wide transition from proof-of-concept to full-scale production has stalled against 1990s-era infrastructure that was not designed to carry generative workloads. Regulatory pressure adds a second layer of drag, since every AI decision now falls inside the heavy compliance and oversight regime governing modern financial services. Behind what looks like slow engineering is a structural feature of the governance model banks were built on.
Sai Santhosh Goud Bandari, Generative AI Developer at TCS, builds generative AI systems inside the deployment constraints that define regulated environments. His current work designs enterprise-grade AI solutions for major financial institutions, and it draws directly on his prior experience running production-scale systems inside the regulated insurance environment at MetLife. From that perspective, the gap between pilot and production comes down to a fundamental mismatch: banking systems were built around deterministic rules with predictable outcomes, and modern language models operate on probabilistic pathways that resist the guarantees regulated environments require.
"With traditional processes, we know how auditing, logging, and compliance work. You simply follow the regulatory requirements. With AI, we cannot predict how the system will behave." The unpredictability drives the deployment freeze in ways that traditional risk frameworks were never built to absorb. Auditing assumes a system will produce the same output given the same input, and compliance assumes an auditable trail from input to decision. Both assumptions collapse the moment a language model gets involved, because the trail bends unpredictably even when prompt orchestration is tight. Existing governance frameworks do not cover the gap.
The slow deployment pace is a deliberate output of the current environment. Customer-facing AI carries real risk, so many financial institutions are prioritizing internal demonstration to secure billing for new initiatives while the governance framework catches up. Engineering cycles compress accordingly, and Bandari says he has deployed only two applications in the last three months. "They might give me a plan for one day or half a day, and by the next evening, they are asking for an update on the minimum viable product. I have to go in and explain to stakeholders exactly what we built and how it works," he notes. "In one of the initial PMO meetings I joined, the expectation was clear: you do not need to completely scale up to production. The immediate goal is to showcase some AI capability within the organization so we can secure billing."
Compliance overhead can absorb the cost savings from replacing human workers, and the practical response many teams are landing on is developer augmentation. Engineering groups are building highly modular applications aimed at accelerating cloud migrations and reducing manual coding, with core deployment, database migration, and testing separated into distinct functions to isolate any single AI failure from the rest of the stack. "By keeping these tools for internal software purposes, the software teams can validate the output themselves," Bandari explains. "The users are aware of what is going on, and they know whether they can safely implement or deploy it."
Even internal deployments still have to run through a dense set of technical constraints before production. Legacy systems, heavily restricted API access, and strict data privacy and auditability requirements all compress the deployment path. The architectural limits of RAG at production scale sit on top of that, and the single most consequential risk sitting underneath all of it is prompt injection, which is why teams are now building strict prompt orchestration into the pipeline itself. "When someone manipulates the context in a chatbot to bypass orchestration rules, asking for something like complete SSN details, you can get duplicate content, document embedding drift, slow document updates, and slow retrieval from large vector indexes," notes Bandari.
Rigid legacy constraints are catalyzing a broader architectural change across the sector, pushing engineering teams away from monolithic systems and toward modular operating models where each layer of the pipeline is owned by a specialized team. Organizational fragmentation maps to specific technical infrastructure choices banks are now making about containerizing AI services rather than relying on legacy virtual machines. "In major financial institutions, different teams often own different parts of the architecture," Bandari explains. "Data engineering handles the data pipelines, platform and infrastructure managers handle Kubernetes and cloud resources, and security handles IAM, encryption, and compliance. As an AI/ML engineer, I develop most of the RAG pipelines, prompts, and model ingestion."
Kubernetes has become the technical foundation for that layered architecture, largely because it handles the heterogeneous resource demands of AI workloads at a level traditional infrastructure cannot. Bandari draws a direct parallel to Postgres in data management: just as Postgres organizes data storage, Kubernetes organizes application orchestration, letting teams allocate expensive GPUs to embedding services while reserving cheaper CPU capacity for the API layer. "We leverage Kubernetes as an orchestration platform for AI workloads rather than traditional virtual machines, because containers start faster and enable efficient CI/CD deployments, scaling, and self-healing in production systems," he adds. "It allows each component to scale independently, providing both performance and cost efficiency."
The next 6 to 12 months are already pointing toward multi-step sub-agent automation inside deployment pipelines, which will require deterministic guardrails to keep unpredictable model behavior inside acceptable bounds. The likely consequence for banking is more human oversight, not less, because governance is what makes safe deployment possible in the first place. "More security jobs will be rising, and more observability roles will be needed," Bandari concludes. "One or two days of testing is not enough for a single production-based AI system. There must be rigorous reviews and multiple loops of security that it needs to pass before going to production."
