Building ZORGAX: Turning a Local LLM into a DevOps AI Agent
What we learned connecting Ollama, Docker, Open WebUI, and real tool calls inside the MyZubster ecosystem.
What happens when you want your local AI to do more than answer questions?
That's the challenge we're exploring with ZORGAX, an AI assistant being developed within the MyZubster ecosystem.
Our goal is to move from a conversational language model toward a controlled infrastructure agent that can monitor servers, inspect services, diagnose failures, and eventually assist with approved maintenance operations.
We're not building an unrestricted autonomous system.
We're building something that must earn trust through testing, restricted permissions, and predictable behavior.
And during our latest experiments, we encountered a problem that illustrates exactly why that distinction matters.
The environment
Our current pilot runs on an Ubuntu VPS and brings together several components:
| Component | Purpose |
|---|
| Ollama | Local LLM inference |
| Qwen 2.5 3B | Foundation language model |
| ZORGAX | Custom AI assistant |
| Open WebUI | Conversational interface |
| Docker Compose | Service orchestration |
| N4K48 | Independent AI integration pilot |
| Qdrant | Vector infrastructure |
| systemd | Host-level service management |
ZORGAX currently uses a custom Ollama model built on qwen2.5:3b.
We chose to start with a relatively small model to explore what could be achieved with local inference and controlled tools.
Challenge 1: Connecting containers to Ollama
Our first problem was networking.
Ollama was listening on:
172.22.0.1:11434
However, Open WebUI was configured to connect through:
http://host.docker.internal:11434
Inside the container, host.docker.internal resolved to 172.17.0.1.
Requests to that address timed out.
After verifying connectivity from inside both the N4K48 API and Open WebUI containers, we updated Open WebUI to use:
OLLAMA_BASE_URL: "http://172.22.0.1:11434"
We also configured a systemd socket proxy for host-local access through 127.0.0.1:11434, while preserving the existing Docker-facing Ollama endpoint.
The result was successful:
HTTP: 200
MODELS:
- nomic-embed-text:latest
- zorgax:latest
- qwen2.5:3b
CONNECTION: OK
Open WebUI also passed its healthcheck:
running / healthy
HTTP 200
We had established reliable connectivity between the application containers and the local AI service.
Next, we tested ZORGAX's ability to interact with external functions.
We asked the assistant to identify available capabilities and distinguish verified integrations from planned ones.
ZORGAX invoked a function named:
list_automations
The tool returned:
{
"automations": [],
"total": 0
}
A reasonable next step would be to report that the query returned no active automations.
Instead, ZORGAX repeated the function call.
We observed a sequence of 28 tool invocations, followed by another test that reached 16 calls.
This was the most valuable finding of the experiment.
A model that can call a function is not necessarily an agent that can manage a function safely.
Tool calling involves much more than producing correctly formatted JSON.
The agent must understand when the task is complete, avoid unnecessary retries, and behave predictably when tools return empty or unexpected results.
Challenge 3: Isolating the root cause
Rather than immediately changing the model or restarting services, we tested the behavior directly through Ollama's API.
First, we asked ZORGAX how it should respond if an automation search returned zero results.
Without any tools enabled, the model provided an appropriate response and completed generation successfully.
Then we simulated a complete function-calling interaction.
Step 1 — User request
How many active automations are there?
Step 2 — Model requests a tool
{
"name": "list_automations",
"arguments": {}
}
Step 3 — Simulated tool response
{
"automations": [],
"total": 0
}
Step 4 — Final assistant response
There are currently no active automations.
No additional tool call was generated.
This controlled test demonstrated that ZORGAX could successfully complete a tool interaction.
The repeated calls seen in Open WebUI therefore remain an integration-level issue to investigate. They could involve the model's behavior under different prompts, tool execution configuration, or the way tool responses are handled.
We haven't established a definitive root cause yet.
Why this matters for DevOps agents
Imagine extending the same behavior to real infrastructure commands.
A repeated read-only status query might consume unnecessary resources.
But a repeated restart, deployment, configuration update, or database operation could have much more serious consequences.
That is why our design direction emphasizes several safeguards:
- Least-privilege access: tools receive only the permissions required.
- Read-only first: monitoring before operational control.
- Bounded execution: limits on calls, retries, and execution time.
- Duplicate prevention: repeated requests should not automatically trigger repeated actions.
- Human approval: consequential changes require explicit authorization.
- Auditability: operations need traceable inputs, results, and outcomes.
These protections must exist in the execution layer, not merely as instructions inside the model's prompt.
Next milestone: myzubster_server_status
Our next planned integration is a read-only infrastructure monitoring tool.
The proposed myzubster_server_status interface would collect a restricted set of system information, including:
{
"server": "myzubster",
"services": {
"ollama": "running",
"docker": "running",
"ipfs": "running"
},
"health": "ok"
}
This JSON is an illustrative target format, not a live measurement.
The first implementation will focus on retrieving selected health information without granting the model shell access, root permissions, or direct access to the Docker socket.
Once implemented and tested, it could enable prompts such as:
ZORGAX, check the MyZubster infrastructure and explain whether any critical services require attention.
Later, we want to expand toward bounded log diagnostics, repository health, test results, and controlled maintenance workflows.
What we've learned so far
Three lessons stand out from this development session.
1. Network connectivity is part of AI engineering.
An LLM can be perfectly healthy while the applications using it cannot reach its endpoint.
Testing from inside containers matters.
2. Tool calling requires workflow discipline.
Generating a valid function call is only the beginning. Reliable completion and retry behavior are equally important.
3. Agent safety is an architectural concern.
Prompts alone cannot enforce operational safety. Permissions, execution limits, validation, and approvals belong in the surrounding software.
Where ZORGAX stands today
ZORGAX can now communicate through Open WebUI, use the local Ollama inference service, and successfully demonstrate a controlled function-calling cycle.
The infrastructure connectivity is verified.
However, the repeated tool-call behavior observed in Open WebUI still needs investigation before broader operational tools are enabled.
Our next step is to build the monitoring layer and make the execution workflow reliable.
The long-term goal is not an AI that controls everything. It's an AI that understands what it is allowed to inspect, what it can recommend, and when it must ask for permission.
That's the direction we're taking with ZORGAX and MyZubster.
Project: MyZubster Ecosystem
AI Agent: ZORGAX
Integration Pilot: N4K48
Stack: Ollama, Qwen 2.5, Open WebUI, Docker, Python, Ubuntu
Work in progress. Current capabilities are experimental and should not be interpreted as production-ready autonomous infrastructure management.
