Introduction: The "Doing Problem"
We have officially exited the "Chat Phase" of AI. For the past two years, we treated LLMs like brilliant consultants trapped in a glass box—great at answering questions, writing poetry, or debugging snippets. But the moment we asked them to do something—log into SAP, update a PO, or restart a server—the architecture collapsed.
We hit the "Doing Problem."
LLMs are born context-blind. They don't know your schema, your security policies, or that your legacy mainframe requires a specific correlation ID header. To fix this, developers started hardcoding fragile "tools" into their agents, creating a maintenance nightmare.
The solution isn't better prompts; it's better architecture. We need a "TCP/IP moment" for AI—a standard way for intelligence to connect to infrastructure.
The Missing Piece: A "Global Catalog" for AI
If you want an agent to act safely, you can't just give it an API key and hope for the best. You need a Global Catalog—essentially a DNS for enterprise capability.
Instead of an agent guessing how to interact with your systems, it queries a centralized metadata layer.
- Agent: "I need to update a customer address."
- Catalog: "I have a Salesforce Update Action and an SAP Customer BAPI.
Here are the schemas and the required permission scopes for both."
This solves the N×M Integration Problem (N models connecting to M tools). By standardizing the metadata of actions and triggers, you create a unified interface. The agent doesn't need to know the specifics of the SAP BAPI; it only needs to know the Catalog Schema.
The "Immune System": Why You Need an AI Gateway
Deploying agents without a governance layer is a security disaster waiting to happen. This is the "Shadow AI" crisis: junior devs calling OpenAI directly from production services, potentially leaking PII.
A robust AI Gateway acts as a reverse proxy for intelligence, handling:
- Rate Limiting: Prevents "loops of death" where two agents argue and rack up a $50k bill.
- Response Caching: Serves repeated answers from memory to slash API costs.
- PII Redaction: Automatically detects and masks sensitive data (like SSNs) before it ever leaves your perimeter.
The Mechanics of Execution
To make agents actually work, we need to move beyond "text-in, text-out" and focus on the "hands" of the system:
Flow Services as APIs: Logic should be decoupled from the agent. The agent shouldn't write the SQL query; it should call a deterministic Flow Service that handles the query safely.
Private Network Connections: Your agent needs to reach "dark data"—resources sitting inside private VPCs or on-prem data centers—without traversing the public internet.
Conclusion: The Boring Revolution
The future of AI isn't just about smarter models; it's about boring, reliable plumbing. It's about LDAP integration, metadata synchronization, and private network links.
We don't need another chatbot. We need a nervous system. We need a way to tell an AI, "Fix the supply chain bottleneck," and have it strictly, securely, and correctly identify the endpoint, authenticate, and execute the fix—all while being governed by a central platform.
For a full technical deep dive into how IBM webMethods is implementing this architecture, read my original article: The Protocol of Doing.