Configuration languages shape how developers think about their systems. We chose TOML because it brings object-oriented thinking to configuration—concepts with operations, not nested metadata.
Configuration Languages Are Contentious
Developers have strong opinions about configuration formats. YAML advocates appreciate the clean look and minimal syntax. JSON supporters like the explicit structure and universal tooling. INI users value simplicity. Each choice involves tradeoffs, and those tradeoffs matter when you're configuring something that needs to be both human-readable and machine-reliable.
We've all been burned by configuration. YAML turns no into false. JSON strips trailing commas. INI has no nesting. This is cognitive overhead that distracts from what you're actually trying to configure.
What if configuration could think like code?
TOML Keeps Configuration Explicit
When we needed a configuration format for a system with complex validation rules and nested test cases, we evaluated the usual suspects. YAML would give us clean syntax but unpredictable parsing. JSON would be explicit but verbose and unforgiving. We needed something developers could read, write, and trust without fighting the format.
TOML stood out because it's explicitly declarative. What you write is what you get. No type coercion surprises, no invisible whitespace dependencies, no ambiguity about structure. But unlike JSON's verbosity, TOML has syntactic features that make configurations genuinely pleasant to write.
Dot Notation Without Explicit Nesting
Here's where TOML gets interesting. You can reference nested attributes using dot notation without having to explicitly define the parent structure first:
[eval]
description = "Test response accuracy"
type = "accuracy"
targets.agents = ["*"]
targets.tools = []
Notice targets.agents and targets.tools? We didn't have to write:
[eval]
description = "..."
type = "accuracy"
[eval.targets]
agents = ["*"]
tools = []
We just used dot notation inline. The structure is implicit but unambiguous. TOML creates the targets object automatically. This is a small thing that adds up when you're writing dozens of configs. It allows you to think in terms of attributes, not nesting depth. You are, however, free to construct the full nested structure if you prefer that style; in cases where we have this flexibility, we prioritize grouping similar concepts together for clarity.
Array of Tables: Repeatable Structures
Configuration often involves repeating the same structure with different values—test cases, server definitions, deployment targets. TOML's [[double bracket]] syntax creates arrays of tables cleanly:
[[eval.cases]]
prompt = "What is the capital of France?"
output.contains = "Paris"
[[eval.cases]]
prompt = "Explain what HTTP is"
output = {
similar = "HTTP is a protocol for transferring data over the web.",
threshold = 0.8
}
[[eval.cases]]
prompt = "Get weather data"
[eval.cases.output.schema]
temperature = { type = "float" }
condition = { type = "str" }
humidity = { type = "int" }
Each [[eval.cases]] declaration adds another item to the array. The structure is visually distinct. The repetition is obvious. You can scan through entries without getting lost in nesting.
Dot Notation as Behavior Modifiers
Here's where TOML's dot syntax enables something clever: you can use it to attach behavior modifiers to fields without introducing extra nesting. Instead of wrapping configuration in container objects, you extend the field name itself:
[[eval.cases]]
prompt = "What is the capital of France?"
output.contains = "Paris"
[[eval.cases]]
prompt = "What's the weather like?"
output.startswith = ["It's", "Currently", "The weather"]
[[eval.cases]]
prompt = "Extract the user's email address"
output.match = "\w+@\w+\.\w+"
Each suffix—.contains, .startswith, .match—changes how the value gets interpreted. The field name carries the semantic meaning. You're not nesting objects or adding metadata layers. You're using the key itself to express intent.
This is object-oriented thinking applied to configuration. You have a concept. You apply operations to it. It's how developers already think—now it's how configuration works. When someone sees output.contains, they immediately understand the pattern. They can intuit what other operations exist. Compare this to YAML or JSON where you'd need wrapper objects:
{
"output": {
"strategy": "contains",
"value": "Paris"
}
}
Or nested structures:
output:
contains: "Paris"
TOML keeps it flat. The field name describes the behavior, and the value is just the value—no extra layers wrapping things up.
This pattern extends to schema definitions. Field validation rules attach directly to field names:
[[eval.cases]]
prompt = "Validate user profile"
[eval.cases.output.schema]
username = {
type = "str",
min_length = 3,
max_length = 20
}
age = {
type = "int",
min = 13,
max = 120
}
The validation constraints are readable and self-documenting. No wrapper objects, no indirection—just direct field definitions with their validation rules.
Inline Tables for Compact Structure
When configuration objects are small, TOML lets you write them inline:
latency = { max_ms = 3000 }
output = { similar = "expected text", threshold = 0.8 }
tools = [{ name = "add", args = [2, 2] }]
This is more readable than splitting every small object into its own [section] block, but it's still explicit. You can see exactly what's nested and what isn't.
What TOML Gets Right
Ecosystem adoption: Python's packaging ecosystem standardized on TOML. pyproject.toml replaced setup.py and became the official configuration format for Python projects. Poetry, Ruff, Black, pytest—all use TOML. When the Python community—known for pragmatism and careful design decisions—chooses TOML as the de facto standard, that's a signal worth paying attention to.
Explicit types: Strings need quotes. Numbers don't. Booleans are true or false, not yes/no/on/off. Dates are ISO 8601. No guessing.
Readable structure: Keys are unquoted unless they need special characters. Sections use [headers] that stand out visually. Comments use # like most scripting languages.
No invisible whitespace: Unlike YAML, indentation is cosmetic. Your config won't break because someone's editor uses tabs instead of spaces. We chose TOML knowing this would eliminate an entire category of bugs.
The Tradeoffs We Accepted
TOML isn't perfect. Deeply nested structures get verbose—you end up with long dotted paths or multiple [section.subsection.subsubsection] headers. If your config is a deeply nested tree, JSON or YAML might feel more natural.
TOML also doesn't support references or anchors like YAML does. You can't define a block once and reuse it elsewhere. For configurations where reusability matters more than explicitness, that's a real limitation.
And TOML is less universally supported than JSON. Most languages have libraries, but JSON is everywhere. If you need to parse configs in obscure environments or legacy systems, JSON's ubiquity wins.
Could TOML Be Better?
One pattern we noticed: when you're defining attributes deep inside an array of tables, you have to repeat the full path:
[[eval.cases]]
prompt = "Test prompt"
[eval.cases.output.schema]
field1 = { type = "str" }
field2 = { type = "int" }
[[eval.cases]]
prompt = "Another test"
[eval.cases.output.schema]
field1 = { type = "str" }
field2 = { type = "int" }
That [eval.cases.output.schema] is verbose. What if TOML supported a shorthand where a leading dot means "relative to the parent table"?
[[eval.cases]]
prompt = "Test prompt"
[.output.schema]
field1 = { type = "str" }
field2 = { type = "int" }
This would indicate "relative to the last table-array item." It's a small syntactic feature that would reduce visual noise in configs with deep nesting under array items. Maybe that's worth exploring as a TOML extension, or maybe it introduces ambiguity that breaks the explicit clarity that makes TOML work. Worth considering.
Why Configuration Format Matters
Configuration gets read and written constantly. Developers write it when defining behavior. They read it when debugging. They scan it during code review. They modify it when requirements change. The format needs to be immediately legible and predictable.
This matters especially when the system being configured is already complex or non-deterministic. Configuration shouldn't add another layer of cognitive overhead—that's the problem we started with. TOML's declarative clarity lets developers focus on what they're configuring, not how the format will interpret it.
For Agent CI, TOML is our primary integration point. Evaluation configurations live in .agentci/evals/ and define all testing behavior without requiring changes to application code. This separation means you can add comprehensive evaluation coverage to existing agents without modifying prompts, tools, or deployment logic. The configuration format becomes the interface. It needs to be readable, maintainable, and expressive enough to handle complex validation rules. TOML gives us that.
The Broader Pattern
This choice reflects a larger philosophy: developer tools should be predictable. Surprises belong in what you're building, not in the tools you're using to build it.
YAML's type coercion might save a few characters, but it creates confusion when Norway becomes true because it matches a boolean alias. JSON's strict structure eliminates ambiguity but makes you count brackets and remember commas. TOML finds middle ground—explicit enough to be predictable, expressive enough to be pleasant.
Configuration languages shape how developers think about their systems. A good config format makes structure obvious, reduces ceremony, and doesn't surprise you. TOML does that.
If you're curious how these patterns play out in practice, the evaluation configs we've been using as examples are open source: github.com/Agent-CI/client-config. If this resonates with configuration frustrations you've hit—or if you've found other patterns that work—we want to hear about it. Configuration design is iterative. The more real-world usage we can learn from, the better.
Travis Dent is CEO of Agent CI, bringing systematic software development practices to AI agent development.