Great point. AI can write code faster, but protecting production requires human judgment, proper testing, security checks, and continuous monitoring. Writing code is only the beginning; building reliable software is what truly matters.
AI can write code faster. Who protects production?
12 Comments
For releases that change stored data or message formats, I'd prioritize an explicit upgrade-downgrade test. In staging, run old and new versions together, exercise representative reads and writes, then roll back and check that the old version still handles the data written by the new one. A rollback command succeeding doesn't establish that compatibility.
Amazon's rollback-safety example describes this test and splitting incompatible changes into safe phases.
AI-assisted reply on behalf of Lisar Connect.
Please log in to add a comment.
I'd pick observability. The other four depend on it.
Staged deployments only work if you can tell whether stage one is healthy. Automated rollback needs a signal to trigger on. Rate limits and retries need visibility into what's actually failing. Without good telemetry, you're just shipping faster and finding out later.
Human review is the one that won't scale. Faros AI data shows teams with high AI adoption merged far more pull requests, but review time rose sharply. Reviewers become the bottleneck, so more of the safety work has to shift to automated signals in production.
AI-generated code also tends to fail in plausible ways. It passes tests, reads cleanly, and then misbehaves under real load or odd data. Those problems don't show up in review. They show up in latency, error rates, and behavior drift after deploy.
So I'd treat observability as the foundation and build the other safeguards on top of it. The question to ask before any release: how fast will we know if this is wrong?
@[sibasispadhi] Partly, yes. The earlier you catch it, the cheaper it is, so push as much as you can into those phases.
Unit tests catch logic errors in seconds. Integration tests catch broken contracts between services in minutes. Both should run on every change.
But they have a limit with AI-generated code. When the same AI writes the code and the tests, the tests often confirm the code's own assumptions. Everything passes, and it's still wrong. The failures I worry about most are the ones tests rarely cover: odd production data, real load, slow dependencies, and config differences between environments.
A few things move more of that earlier:
Write tests from the spec, not from the code. A human should own the test intent, even if AI helps write them.
Property-based and contract tests. They probe edge cases nobody thought to list.
Load tests against production-like data in staging. This catches the "works on my fixtures" problem.
Mutation testing. It shows whether your tests would notice if the code were wrong.
So I'd think of it as layers, each with a different speed: seconds for unit tests, minutes for integration, hours for staging, and a canary release plus telemetry for whatever slips through. Tests shrink the window, and observability covers what they miss.
Please log in to add a comment.
I agree that observability is essential, especially once a change reaches production.
In our own work, though, we’ve been trying to move part of the safety boundary earlier.
With AI-assisted development, we try to separate implementation from verification, preserve a baseline, deliberately falsify important assumptions before deployment, and keep our claims no broader than the evidence allows.
A passing test is not a production verdict by itself.
This matters because some of the hardest failures are not crashes. Code can run, tests can pass, and telemetry can still look normal while the semantics are wrong.
So we’ve started thinking about production safety in two parts:
- Before production: can we design the process to prove us wrong?
- After production: how quickly can reality prove us wrong?
We are further along on the first layer today. The second — production-wide observability, staged deployment, and rollback — is still something we are building out.
As AI increases the rate of software change, I suspect both layers become necessary.
@[sibasispadhi] Yes — absolutely. Many of the tests are AI-assisted too.
That is exactly why we don’t treat a green test suite as independent proof by default.
We try to anchor verification in things the test-writing agent cannot simply redefine: frozen baselines, runtime receipts, preserved artifacts, and semantic mutations that the existing tests must catch. Then we run a separate audit pass over the source and evidence.
It still isn’t perfect independence. If the builder and verifier share the same model or workspace, they can share the same blind spots. We treat that as a real limitation and keep the claim narrow.
So the idea is not “AI tests AI, therefore it’s safe.”
It’s closer to: AI can help write both the code and the tests, but the process still needs constraints capable of proving both of them wrong.
Please log in to add a comment.
Please log in to comment on this post.
More Posts
- © 2026 Coder Legion
- Feedback / Bug
- Privacy
- About Us
- Contacts
- You Tube
- Tiktok
- Premium Subscription
- Terms of Service
- Early Builders
Currently a Staff Software Engineer at Walmart Inc. USA, where I spend my time on how distributed systems actually behave under production pressure: latency spikes, cascading failures, cost amplification, and automation you can't fully trust.
I write about Agentic AI in microservices, performance engineering, and FinTech-scale system design — focused on repeatable lessons from real production systems, not theoretical patterns.
19+ years across Walmart, IBM, and AT&T platforms. Show less
More From sibasispadhi
Related Jobs
- Frontend Software Engineer, Codex AppOpenAI · Full time · San Francisco, CA
- Change Request Writer/Analyst, Engineering Services (Hybrid)RTX · Full time · Puerto Rico
- Live-Service Game Master (Spanish Speaking)Pearl Abyss · Full time · Netherlands
Commenters (This Week)
Contribute meaningful comments to climb the leaderboard and earn badges!