Skip to main content

What happens after AI builds your prototype?

Blog article hero image
Ryan Wong
Ryan Wong
Head of Engineering @ Retool

2025 was the year of vibe coding. Almost overnight, workers who’d never written a line of code were shipping sophisticated prototypes that felt like magic.

That gap—between what AI can generate and what an enterprise can actually deploy—is the most important unsolved problem in software right now.

The answer is architectural. Governance needs to live at the resource layer, not the app layer. This post explains why app-level security breaks down when anyone can generate an app in five minutes, and what IT and security teams need to put in place instead.

Just because an app seems functional doesn’t mean it’s production-ready

At the heart of this disconnect is that a functional app a business user ships with AI is fundamentally not the same as a production-ready internal tool that can scale securely across a business. What they see looks like an app, but the hard part starts when the app they generated needs to connect to a real database, touch customer data, or run inside an environment with compliance requirements.

Who decides what data this app can access? Who reviews the security model? Who’s responsible when an LLM-generated app with overly permissive IAM roles quietly starts leaking data?

The answer at most organizations today is nobody. Because most AI-generated apps never go through the process that answers those questions. They either sit in a sandbox forever, get rebuilt from scratch by an engineer, or they increasingly get pushed to production without any governance review at all.

Why vibe-coded software breaks the trust model and adds risk

When anyone can generate an app, the accountability chain that enterprise security depends on disappears.

Enterprise software has always had a simple trust model: a known developer writes code, that code gets reviewed, and the reviewed code gets deployed with explicit permissions.

AI-generated code inverts every part of that model.

Now, the builder might not be a developer. The code might not get reviewed, because the happy path works, teams can’t possibly review code faster than it is being generated. And the permissions? It’s easier to test that it works than it is to verify it is secure.

Security researchers are already documenting the pattern. According to their research, the three most common vulnerabilities in AI-generated enterprise code are improper input validation, over-permissive IAM role assignments, and hardcoded credentials. These are the security fundamentals that AI models systematically skip because they optimize for “does it work,” not “is it safe.”

David Mytton, CEO of developer security provider Arcjet, predicts that the steady buildout of vibe-coded apps into production will lead to “big explosions”. He argues that AI should implement validated security patterns, not invent security from scratch. But that’s exactly what happens when every generated app ships with its own bespoke security logic.

Why app-level security isn’t enough

The traditional approach to securing internal tools is app-level governance. Each app defines its own permissions: who can see what, who can edit what, what data it can access. The app is the security boundary.

This model worked when a company had dozens of internal apps, each built by an engineering team over weeks or months. There was time to get the permissions right. There were human experts accountable for each decision.

It doesn’t work when anyone in the organization can generate an app in five minutes.

If you secure at the app level, then whoever generates the app is also generating the security rules. When the builder is an LLM acting on a natural-language prompt, the security rules are whatever the LLM decides they should be. As Samsung SmartThings executive Nikhil Jain wrote in Forbes: “AI can generate code. It does not automatically generate governance.”

Think about what this means at scale. An enterprise with 500 business users, each capable of generating apps with AI, is an enterprise with 500 potential sources of ungoverned data access. Each app has its own security logic. Each one is a potential misconfiguration. And there’s no central place to see what's been built, what data it touches, or whether any of it is safe.

Why AI app governance can’t be solved with process

Governance doesn’t fail because organizations lack the right process. It fails because process is the wrong solution. However, the instinct in most organizations is to respond to this increased risk with processes: review boards, approval workflows, security checklists. And those are fine, but only for the apps that go through the process. The problem is that AI-generated apps are multiplying faster than any review board can absorb. A procedural response to an architectural problem is a losing game.

The alternative is to move governance underneath the app layer. Instead of each app defining its own permissions, permissions live with the data. The app doesn’t decide what it can access, the platform does. A generated app can be as insecure as the LLM wants to make it, and it still can’t touch data it shouldn’t, because the security boundary is in the infrastructure the app runs on, no longer relying purely on application code.

When governance lives at the resource level—meaning the governance is enforced by the platform, not by the individual app—it doesn’t matter who built the app, what tool generated it, or whether anyone reviewed the code. The security model is independent of the app layer.

What governance-first building actually looks like

Moving from app-level to resource-level governance changes several things:

  • Permissions are configured per-resource, not per-app. An admin defines what data each role can access, at the resource level. Every app that runs on the platform inherits those rules. A new app generated by an LLM doesn’t need its own security review because it can’t access anything the platform hasn’t already approved.
  • Audit trails are automatic. When every data access goes through a single governance layer, every query is logged. You don’t need to instrument individual apps. You can answer “what data did this app access and who used it” for any app, including ones that were generated five minutes ago.
  • Governance boundaries apply to any builder. An app built by a senior engineer and an app generated by a prompt go through the same governance boundary. The platform treats them identically because it’s enforcing rules on the data access, not on the code. This is what makes it safe to let anyone build—the governance doesn’t depend on the builder being trustworthy.

The real risk of AI-generated apps isn’t that they’re insecure in isolation. It’s that they’re invisible. When generation is easy and deployment is ungoverned, apps proliferate outside IT’s awareness. Resource-level governance, by definition, makes all data access visible because there’s only one path to the data.

How Retool solves governance for enterprise-ready apps

Right now, every AI coding tool and app builder is focused on making generation faster. A ton of companies are already solving the code creation problem. None of them are solving the governance problem.

That’s the architectural problem Retool is building to solve. We’re building for a future where governance doesn’t degrade as AI gets better. In Retool, governance is automatic and resource-level, so any app running on the platform—whether built by an engineer or generated by a prompt—operates within the same security boundary.

Ryan Wong
Ryan Wong
Head of Engineering @ Retool
Copied