Code from AI Agents: Checklist Before Production
At a board meeting or with an investor someone says: "We are adding AI agents — code will ship 3× faster." Two or three months later the repo has hundreds of "agent" lines, prod has a silent payment bug or an API key in git history, and the team argues who reviewed that PR. The market promise delivered on speed. It did not deliver on control.
AI agents write code — a normal draft accelerator. Production should only get that code through the same gates as human code: tests, review, security, staging. Below — risks in rubles, four mandatory gates, and a CTO checklist before release.
Bottom line. The agent speeds up the draft, not review. Into main — only with tests, secret checks, and a PR owner. One hour of senior review is cheaper than a day of payment incident.
Generation speed without review gates is not "efficiency" — it is a deferred incident
The problem is not AI agents — it is release without an owner
Agents handle routine well: boilerplate, template migrations, tests for a known function, API draft. They fail when used to "close" the whole task: "build payments like Stripe," "fix prod," "add roles" — with no fixed scope and no human accountable for the outcome.
Typical failure after 60–90 days:
- PRs land in main with no clear "what changes for the user" description;
- tests are green but do not cover payment / permissions / ERP integration;
- a chunk of .env was pasted into a prompt once — and stayed in history;
- dependency or internet snippet license was never checked.
This is not "AI vs developer." It is "do you have a process like mature CI/CD for production," or generation speed simply outran discipline.
Three risks that most often surface with users, not on code review
Three risks of AI agent code in rubles
1. Secret and data leaks. Key rotation, access audit, integration downtime — often 100–300k ₽ direct cost plus reputation. If PII is involved — lawyers and regulators.
2. Silent bug in money or permissions. Wrong rounding, role bypass, broken offline flow. One day of triage by a 3–4 person team at senior rates — easily 150–400k ₽, not counting lost orders in peak.
3. Licenses and "someone else's" code. A rewritten GPL or unclear-source fragment in a commercial product — claim risk and module rewrite. Cheaper to check the dependency at PR time than to litigate or cut the feature from release.
Rule of thumb for leadership: one hour of meaningful senior review (5–10k ₽) vs one day of incident (from 150k ₽). AI cheapens the draft; without gates it also cheapens accountability — until the first outage.
Savings on review are eaten by one incident — especially in peak season or on payments
Four gates to production for AI code
Minimum without which an AI PR must not reach main. Same rules — for in-house and contractor (fix in the contract appendix, see IT contract checklist for CEO).
- Scope. The task states: what changes for the user, what we do not touch, done criteria. Prompt "make it pretty" is not scope.
- Tests. Autotest on the critical path (payment, login, entity creation) green on CI. "Agent wrote tests" is not enough if they assert a mock, not the scenario.
- Security review. Secret scan, dependencies, access rights. Ban: prod secrets and PII exports in prompts. Human confirms: no hardcoded keys, no "temporary" auth bypass.
- Staging + PR owner. Run on staging. PR lists a responsible engineer (not "agent"). No name in Assignees — no merge.
Scope → tests → security → staging. Skipping any gate = straight path to prod incident
When AI agents fit — and when it is too early
Fit when:
- the task is narrow and repeatable (rule-based refactor, CRUD, schema migration per spec);
- CI, staging, and review culture exist before agent experiments;
- critical paths (money, PII, access) are tagged and need manual approve;
- pilot metric: share of AI PRs with regression on staging / in prod — and it falls.
Too early or risky when:
- "agent pushes to main" or merges without a human;
- no tests on payment / roles / integrations — while the agent "accelerates" them;
- prod chunks and client data go into prompts;
- MVP still has no basic CI — build the process skeleton first, as in SaaS MVP in 2 months, then accelerators.
Checklist: AI agent code before production
- The task has scope and acceptance criteria — not only a chat with the model.
- The PR has a human owner and a description of business effect.
- CI green: linter, build, critical-path tests.
- Secret and dependency checks passed; no .env in history.
- Staging run signed off (screenshot / scenario checklist).
- Permission, payment, PII changes — separate senior / security approve.
- Contractor contract has the same gates as in-house (not "agent sped it up — delivered without review").
Bottom line
AI agents write code faster than humans on the draft. In prod that becomes value only if speed does not outrun control. The problem is not the model — it is missing PR owner, tests, and security gates. One hour of review is cheaper than one day of incident.
Need gates in CI and the dev team contract — we help design the process and strengthen review: AI agent development or senior outstaff with code review discipline. Deployment overview — in the AI agents for business guide.
Related services
Executive questions about code from AI agents
Yes — if the agent speeds up the draft and prod only gets code after the same gates as human work: tests, review, secret checks, staging. Without a PR owner and acceptance criteria, "agent code" is unrecorded debt that surfaces on users.
The risk is not that the model is "dumb," but that generation speed outruns control: secrets in the repo, third-party licensed code, checkbox tests, nobody understands the change end-to-end. The problem is the release process, not the tool name.
Typical range for a B2B service: 150–400k ₽ per day of triage (engineers + downtime + rollback) up to millions if payments, PII, or peak sales are affected. One hour of senior review before release is an order of magnitude cheaper than a day of emergency recovery.
Four minimums: fixed task scope; autotests on the critical path; security check (secrets, dependencies, permissions); staging run with a clear PR owner. Missing any one — not into main.
Pilot on non-critical tasks for 2–4 weeks, rule "agent does not commit to main," PR description template, ban on pasting prod secrets into prompts, metric "share of AI PRs with regressions." Then expand scope. Contractor and in-house — same rules in the contract and CI.
Want to apply this in practice?
Tell us about your system — we’ll propose a work plan and the metrics worth fixing in an SLA/SLO.
Related articles
Production monitoring: 4 metrics anyone on the team can understand
Production monitoring in plain language: site speed, errors, traffic, and server headroom. What to check before ads and how not to learn about outages from angry customers. DevOps, Grafana, Prometheus.
Read ArticleDevOps and CI/CD in Production: What to Set Up First
DevOps services for business: build pipeline, staging, zero-downtime deploy, monitoring and rollback — priorities for the first 4–6 weeks.
Read ArticleKubernetes in Production: A CTO Checklist Before Launching a Cluster
Production Kubernetes setup: RBAC, resources, Ingress, GitOps, monitoring, and common mistakes — a checklist before going live.
Read ArticleWhy Business Needs SRE? Translating Reliability into Money
Why businesses adopt SRE: SLIs, SLOs, error budgets, and tying reliability to money—without chasing vanity nines or drowning teams in process.
Read Article