menu

Are orchestration frameworks for production or just for getting started?

●2 ●13 ●39
calendar_today • schedule1 min read

One thing I have been wondering recently:

Do AI orchestration frameworks actually survive in production as systems become more complex?

Most tutorials teach LangGraph, LangChain, CrewAI, Pydantic AI, etc.

But when I look at products like Claude Code, Cursor, Windsurf, OpenAI Codex or Devin I wonder how much of their orchestration is still framework-based versus custom-built.

For engineers working on production AI systems:

What does your orchestration layer look like?
Where do frameworks help?
At what point did you decide to build your own?

I am genuinely curious to hear how others approached this. If you have built AI agents in production, I would love to hear your thoughts and the reasoning behind your architectural decisions.

10 Comments

•

Interesting take. I think frameworks are great for getting started, but production is where the real trade-offs show. What's your go-to setup today?

1 vote
•

@[Mehadi Hasan] I have worked with Langchain, LangGraph, DSPy, the OpenAI Agents SDK but I am still exploring which approach scales best in production which is why I started this discussion.

1
•

Great perspective. Orchestration frameworks are excellent once your workflows become complex, but they’re not always necessary from day one. Starting simple and adding orchestration only when the complexity justifies it can save a lot of unnecessary overhead. Great read.

1 vote
•

@[unwind] Totally agree. Starting simple and introducing orchestration only when it is actually needed seems like the most practical approach.

2
•

Never used one. The agent layer in my game is an abstract class with two methods - give me structured output matching this schema, or give me plain text. Every provider implements those two, and that has carried 20-something models across 9 providers.

The frameworks abstract the part that was already easy. Calling a model is easy. The work is deciding what goes into the context at each step, and that is your domain, not theirs. I compute the exact slice of rules a given step needs and inject it as the last message, then validate the answer against an enum before it goes anywhere.

1 vote
•

@[hiper2d] Hey thats a great perspective actually I like the idea of abstracting only the provider layer and keeping the orchestration domain specific. The context injection point is especially interesting that seems to be where the real value is. Did you ever experiment with frameworks or has this custom approach worked well from the beginning? What was your approach when you start working on new agentic ai project?

2
•

@[Ayush_SIngh] Never adopted one, but I read through a couple before deciding. What put me off is that when you break them apart, every piece is something you already have. Memory is a file plus a step that decides what to load. The loop is a while statement. The provider call is one function. I would be trading readability for convenience I do not need.

Starting a new one, I write the dumbest possible version first - one script, one prompt, no abstractions - and run it until it breaks. What it breaks on tells you what to build. Mine broke on context: the model kept drifting off rules that were sitting right there in the system prompt. So what I built was the per-step context assembly. If it had broken on tool routing I would have built that instead, and I would have got there without guessing.

The one piece I would put in on day one regardless is validation at the boundary. Every model response gets parsed into a type before anything downstream sees it. Cheap to add early, miserable to retrofit.

1
•

@[hiper2d]Oh that's really a nice approach I really like the idea of letting the failures drive the architecture instead of guessing upfront. The validation-at-the-boundary point also feels like one of those small decisions that pays off as systems grow.
I just want to ask one thing have you ever has there ever been a point where you felt a framework would have saved time or has the custom approach continued to scale well for you?
and one more thing has your custom orchestration held up as you have added more providers and features, or have there been parts you have had to redesign over time?
Other than i genuinely like the approach thank you for the insights it is really helpful for me as well.

1
•

@[Ayush_SIngh] I didn't find the right framework for my case. Usually, the orchestration logic assumes that different models have different roles, and the orchestrator picks one based on whoever shuts the task better. In my case, the orchestrator needs to pick one by name to keep the conversation going smoothly. It has 10+ AI players, it follows their conversation and based of that decides who speaks next.

And implementing my own orchestrator was an easy thing. I can add new providers by simply passing their documentation page to Claude Code. I used LangChain in the past, and it was a struggle - this things kept breaking and was not customizable enough. I had to wait when devs adds some new feature or fix a bug - now I can have the newest models or API features the same day they are released. With AI it's easy to maintain.

AI has changed my philosophy to having as least frameworks are possible.

1
•

@[hiper2d] That's a really good insight for me, especially the point about maintaining and debugging framework heavy pipelines. I think every framework has its limitations and sometimes the better approach is to build your own orchestration while taking the useful ideas and patterns from those frameworks. I really like your philosophy of keeping the framework layer as small as possible and owning the orchestration logic yourself. The ability to adopt new model/API features immediately is a big advantage too.

This was really helpful and honestly cleared up a lot of my confusion around when custom orchestration makes more sense than using a framework. Thanks for sharing your experience!

1
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.
Join CoderLegion

More Posts

The Sovereign Vault — A Comprehensive Guide to Protocol-Driven AI

Ken W. Algerverified - Jun 4

AI Reliability Gap: Why Large Language Models are not for Safety-Critical Systems

praneeth - Mar 31

Helping Clients Move from Pilot to Production: The Agentic AI Governance Playbook

Tom Smithverified - Jun 8

Your AI Doesn't Just Write Tests. It Runs Them Too.

Kevin Martinez - May 12

Just completed another large-scale WordPress migration — and the client left this

saqib_devmorph - Apr 7
chevron_left
Ayush Singh
1.9k Points • 54 Badges
India • github.com/AyushSingh110
13Posts
24Comments
13Connections
AI and data science undergrad student exploring new technologies and doing research on the models to... Show more

More From Ayush_SIngh

Not All Repair Helps: What I Learned Trying to Fix a Failing AI Agent

Your LLM guardrail speaks English. Your attacker doesn't.

# The Hidden Failure Modes of AI Agents

Related Jobs

View all jobs →

Commenters (This Week)

4 comments
3 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!