menu

Are orchestration frameworks for production or just for getting started?

●2 ●13 ●38
calendar_today ago • schedule1 min read

One thing I have been wondering recently:

Do AI orchestration frameworks actually survive in production as systems become more complex?

Most tutorials teach LangGraph, LangChain, CrewAI, Pydantic AI, etc.

But when I look at products like Claude Code, Cursor, Windsurf, OpenAI Codex or Devin I wonder how much of their orchestration is still framework-based versus custom-built.

For engineers working on production AI systems:

What does your orchestration layer look like?
Where do frameworks help?
At what point did you decide to build your own?

I am genuinely curious to hear how others approached this. If you have built AI agents in production, I would love to hear your thoughts and the reasoning behind your architectural decisions.

7 Comments

•

Interesting take. I think frameworks are great for getting started, but production is where the real trade-offs show. What's your go-to setup today?

1 vote
•

@[Mehadi Hasan] I have worked with Langchain, LangGraph, DSPy, the OpenAI Agents SDK but I am still exploring which approach scales best in production which is why I started this discussion.

1
•

Great perspective. Orchestration frameworks are excellent once your workflows become complex, but they’re not always necessary from day one. Starting simple and adding orchestration only when the complexity justifies it can save a lot of unnecessary overhead. Great read.

1 vote
•

@[unwind] Totally agree. Starting simple and introducing orchestration only when it is actually needed seems like the most practical approach.

2
•

Never used one. The agent layer in my game is an abstract class with two methods - give me structured output matching this schema, or give me plain text. Every provider implements those two, and that has carried 20-something models across 9 providers.

The frameworks abstract the part that was already easy. Calling a model is easy. The work is deciding what goes into the context at each step, and that is your domain, not theirs. I compute the exact slice of rules a given step needs and inject it as the last message, then validate the answer against an enum before it goes anywhere.

1 vote
•

@[hiper2d] Hey thats a great perspective actually I like the idea of abstracting only the provider layer and keeping the orchestration domain specific. The context injection point is especially interesting that seems to be where the real value is. Did you ever experiment with frameworks or has this custom approach worked well from the beginning? What was your approach when you start working on new agentic ai project?

2
•

@[Ayush_SIngh] Never adopted one, but I read through a couple before deciding. What put me off is that when you break them apart, every piece is something you already have. Memory is a file plus a step that decides what to load. The loop is a while statement. The provider call is one function. I would be trading readability for convenience I do not need.

Starting a new one, I write the dumbest possible version first - one script, one prompt, no abstractions - and run it until it breaks. What it breaks on tells you what to build. Mine broke on context: the model kept drifting off rules that were sitting right there in the system prompt. So what I built was the per-step context assembly. If it had broken on tool routing I would have built that instead, and I would have got there without guessing.

The one piece I would put in on day one regardless is validation at the boundary. Every model response gets parsed into a type before anything downstream sees it. Cheap to add early, miserable to retrofit.

0
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.
Join CoderLegion

More Posts

The Sovereign Vault — A Comprehensive Guide to Protocol-Driven AI

Ken W. Algerverified - Jun 4

AI Reliability Gap: Why Large Language Models are not for Safety-Critical Systems

praneeth - Mar 31

Helping Clients Move from Pilot to Production: The Agentic AI Governance Playbook

Tom Smithverified - Jun 8

Your AI Doesn't Just Write Tests. It Runs Them Too.

Kevin Martinez - May 12

Just completed another large-scale WordPress migration — and the client left this

saqib_devmorph - Apr 7
chevron_left
Ayush Singh
1.9k Points • 53 Badges
India • github.com/AyushSingh110
13Posts
22Comments
12Connections
AI and data science undergrad student exploring new technologies and doing research on the models to... Show more

More From Ayush_SIngh

Not All Repair Helps: What I Learned Trying to Fix a Failing AI Agent

Your LLM guardrail speaks English. Your attacker doesn't.

# The Hidden Failure Modes of AI Agents

Related Jobs

View all jobs →

Commenters (This Week)

3 comments
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!