Or: How I Learned to Stop Worrying and Love the Dot-Notation
The Setup: Two Nodes, One Mystery
Picture this: you've just spun up a Kubernetes cluster on your laptop. You've got two nodes talking to each other:
NAME STATUS ROLES AGE VERSION
k8s-worker1 Ready <none> 9d v1.36.3
osboxes Ready control-plane 9d v1.36.3
Everything looks calm. But beneath the surface, one node is about to go rogue, and you're about to learn more about JSON paths than you ever thought possible.
This is the story of how a simple assignment—"Check the number of schedulable nodes excluding tainted (NoSchedule) and write the number to a file"—turned into an epic journey through Kubernetes internals, JSONPath syntax, jq filters, and the subtle art of handling null.
Chapter 1: Peeking Under the Hood with JSON
When you run kubectl get pods, you get a nice table. But Kubernetes is holding so much more information than fits in a table. To see everything, you ask for JSON:
k get po -o json
What comes back is a massive JSON object with nested objects, arrays, and maps. It's intimidating at first, but think of it like a Russian nesting doll:
{
"apiVersion": "v1",
"items": [
{
"kind": "Pod",
"metadata": {
"name": "nginx",
"labels": { "run": "nginx" }
},
"spec": {
"containers": [
{
"name": "nginx",
"image": "nginx"
}
]
}
}
]
}
The key insight: Every Kubernetes object follows this pattern. There's always a top-level structure, and inside .items[] lives the actual object (Pod, Node, Service, etc.).
Chapter 2: JSONPath — Kubernetes' Built-in X-Ray Vision
JSONPath is like giving kubectl a treasure map. Instead of dumping the entire JSON, you say: "Give me exactly this field."
k get pods -o=jsonpath='{.items[0].metadata.labels}'
# Output: {"run":"nginx"}
Boom! We navigated: .items → first item [0] → .metadata → .labels.
Success: Finding the Container Image
k get pods -o=jsonpath='{.items[0].spec.containers[0].image}'
# Output: nginx
Notice how we chain deeper: .spec → .containers[0] → .image. It's like a file path, but for JSON.
Success: Volume Mounts
k get pods -o=jsonpath='{.items[0].spec.containers[0].volumeMounts[0].mountPath}'
# Output: /var/run/secrets/kubernetes.io/serviceaccount
We're getting good at this! But wait...
Chapter 3: The Node Expedition — Where Things Get Interesting
Pods were practice. Now let's look at nodes.
Experiment 1: Getting Node Names
k get nodes -o=jsonpath='{.items[*].metadata.name}'
# Output: k8s-worker1 osboxes
The [*] wildcard says: "For every item in the array, give me this field." It's the JSONPath way of saying "all of them."
Experiment 2: Adding a Newline
k get nodes -o=jsonpath='{.items[*].metadata.name}{"\n"}'
# Output: k8s-worker1 osboxes
# (now with a proper newline!)
JSONPath can inject literal strings. Handy for formatting.
Chapter 4: Custom Columns — Making Your Own Tables
JSONPath is great for single values. But what if you want a table with only the columns you care about?
Enter custom-columns.
Attempt 1: The Syntax Trap
k get nodes -o='custom-columns=Node:{.items[*].metadata.name}'
# Output:
# Node
# <none>
# <none>
What went wrong? With custom-columns, Kubernetes already iterates over .items for you. Using {.items[*]...} is redundant—it expects a path relative to a single item, not the whole list.
The Fix: Drop the .items[*]
k get nodes -o='custom-columns=Node:{.metadata.name}'
# Output:
# Node
# k8s-worker1
# osboxes
Aha moment: custom-columns iterates automatically. You write the path as if you're looking at one node, and Kubernetes repeats it for all nodes.
Attempt 2: The Unclosed Brace
k get nodes -o='custom-columns=Node:{.metadata.name},IP:{.status.addresses[*].address'
# error: unexpected path string...
What went wrong? Missing closing } brace. JSONPath expressions in custom-columns must be fully enclosed in {}.
The Fix: Close Your Braces!
k get nodes -o='custom-columns=Node:{.metadata.name},IP:{.status.addresses[*].address}'
# Output:
# Node IP
# k8s-worker1 10.0.2.4,k8s-worker1
# osboxes 10.0.2.15,osboxes
It works! But notice the IP column has two values per node (InternalIP and Hostname). That's because [*] grabs everything in the array.
Attempt 3: Maps Are Not Arrays
k get nodes -o='custom-columns=Node:{.metadata.name},Capacity:{.status.capacity[*]}'
# error: map[string]interface {} is not array or slice
What went wrong? .status.capacity is a map (key-value pairs: cpu: 2, memory: 2018648Ki), not an array. You can't use [*] on a map.
Attempt 4: The Map Dump
k get nodes -o='custom-columns=Node:{.metadata.name},CPUCapacity:{.status.capacity}'
# Output:
# Node CPUCapacity
# k8s-worker1 map[cpu:2 ephemeral-storage:242470332Ki hugepages-2Mi:0 memory:2018648Ki pods:110]
# osboxes map[cpu:2 ephemeral-storage:515464096Ki hugepages-2Mi:0 memory:4011360Ki pods:110]
It works, but it's ugly. We wanted just the CPU number, not the whole map printed as a Go-style map[...].
The Fix: Drill Into the Map
k get nodes -o='custom-columns=Node:{.metadata.name},CPUCapacity:{.status.capacity.cpu}'
# Output:
# Node CPUCapacity
# k8s-worker1 2
# osboxes 2
Aha moment: Maps are accessed by key, just like objects. .status.capacity.cpu says: "Go into capacity, then get the value for key 'cpu'."
Chapter 5: The Real Boss Fight — Counting Schedulable Nodes
Now for the main event. The assignment sounds simple: count schedulable nodes, excluding those with a NoSchedule taint.
But this is where we descend into the underworld.
Mistake 1: Wrong Resource Type
kubectl get pods -o json | jq -r '.items[] | select(.spec.taints[].effect == "NoSchedule")'
# (empty output)
What went wrong? We queried pods. Pods don't have .spec.taints—nodes do! This is like looking for a car's engine in the trunk.
Mistake 2: The Null Iteration Bomb
kubectl get nodes -o json | jq -r '.items[] | select(.spec.taints[].effect == "NoSchedule")'
# jq: error: Cannot iterate over null (null)
What went wrong? We switched to nodes, but k8s-worker1 has no taints at all. In JSON, a missing field is null. jq tried to iterate over null with [], and jq hates that.
Think of it like this: you asked jq to "open every box in this room," but one of the boxes doesn't exist. jq throws its hands up.
Attempt 3: Filtering for Null Taints
kubectl get nodes -o json | jq -r '.items[] | select(.spec.taints == null)' > schedulable-nodes.txt
This works for finding nodes with no taints, but it writes the entire JSON object of k8s-worker1 to the file. The assignment asked for a number, not a novel.
Also, what if a node has taints, but none of them are NoSchedule? This query would miss it.
The Working Solution: The Full jq Pipeline
After much trial and error, here is the command that actually solves the problem:
kubectl get nodes -o json | jq '[.items[] | select(.spec.taints == null or ([.spec.taints[] | select(.effect == "NoSchedule")] | length == 0))] | length' > schedulable-nodes.txt
Let's dissect this beast, piece by piece.
Chapter 6: The jq Command Autopsy
Stage 1: Get the Data
kubectl get nodes -o json
Produces the full JSON list of nodes.
Stage 2: Enter jq
| jq '...'
jq is a command-line JSON processor. Think of it as grep and awk had a baby that only speaks JSON.
Stage 3: Iterate Over Items
.items[]
This says: "Take the items array and process each element one by one."
Stage 4: The select() Filter
select(.spec.taints == null or (...))
select() keeps an item only if the condition is true. We have two conditions joined by or:
Condition A: .spec.taints == null
- Does this node have no taints at all?
- For
k8s-worker1 (when healthy): true → keep it.
Condition B: ([.spec.taints[] | select(.effect == "NoSchedule")] | length == 0)
This is the sophisticated part. Let's break it down:
.spec.taints[] — "Go through each taint on this node."| select(.effect == "NoSchedule") — "Keep only the taints whose effect is NoSchedule."[ ... ] — "Put those matching taints into an array."| length — "Count how many items are in that array."== 0 — "Is that count zero?"
If a node has taints but zero of them are NoSchedule, this returns true.
Why both conditions? Because if a node has no taints, .spec.taints is null, and you can't iterate over null. Condition A catches the "no taints" case. Condition B catches the "has taints, but none are NoSchedule" case.
Stage 5: Collect and Count
[ ... ] | length
The outer [ ... ] gathers all the nodes that survived the filter into a new array. Then | length counts them.
In our case:
k8s-worker1 (no NoSchedule taint) → KEPTosboxes (has node-role.kubernetes.io/control-plane:NoSchedule) → DISCARDED
Result: 1
Stage 6: Write to File
> schedulable-nodes.txt
Shell redirection writes the output (just the number 1) to the file.
Chapter 7: The Plot Twist — A Node Goes Dark
Remember when we said k8s-worker1 was healthy? At one point in the session, it wasn't.
The node JSON revealed:
"conditions": [
{
"type": "Ready",
"status": "Unknown",
"reason": "NodeStatusUnknown",
"message": "Kubelet stopped posting node status."
}
],
"spec": {
"taints": [
{
"effect": "NoSchedule",
"key": "node.kubernetes.io/unreachable"
},
{
"effect": "NoExecute",
"key": "node.kubernetes.io/unreachable"
}
]
}
The kubelet on k8s-worker1 stopped sending heartbeats. After ~55 minutes of silence, the control plane declared it unreachable and slapped NoSchedule and NoExecute taints on it.
This is why JSON exploration matters. If you only looked at kubectl get nodes, you'd see NotReady or Unknown. But the JSON tells you why: the kubelet died. Maybe the VM crashed. Maybe the battery died on the laptop hosting it. The JSON is the detective's notebook.
Later, the node recovered—its taints disappeared and it started reporting healthy status again. But for a while, the schedulable node count dropped to zero (because both nodes had NoSchedule taints: k8s-worker1 from being unreachable, and osboxes from being the control plane).
Chapter 8: Key Lessons — The JSONPath Survival Guide
| Concept | JSONPath Example | What It Does |
|---|
| Single item | {.items[0].metadata.name} | First item's name |
| All items | {.items[*].metadata.name} | All names, space-separated |
| Wildcard in maps | {.status.capacity.cpu} | Value for key cpu |
| Wildcard in arrays | {.spec.containers[*].image} | All container images |
| Literal string | {.metadata.name}{"\n"} | Add a newline |
| Custom columns | -o='custom-columns=NAME:{.metadata.name}' | Build your own table |
The Golden Rules
custom-columns iterates for you. Don't use .items[*] inside it. Use .metadata.name, not .items[*].metadata.name.
Maps ≠ Arrays. You can't use [*] on a map like .status.capacity. Use .status.capacity.cpu instead.
jq hates null iteration. Always guard against null when using []. Use == null or // [] (default empty array) as safety nets.
Know your resource anatomy. Pods have .spec.nodeName and .spec.tolerations. Nodes have .spec.taints and .status.conditions. Don't mix them up!
Count at the end. If the assignment asks for a number, wrap your filter in [ ... ] | length before writing to the file.
The Final File
cat schedulable-nodes.txt
1
One file. One number. Hours of learning compressed into a single digit.
Epilogue: Why This Matters
Kubernetes is an API-driven system. Everything you see in kubectl is a pretty-printed version of JSON traveling over HTTP. When you learn JSONPath and jq, you stop being a tourist and start being an explorer. You can:
- Build monitoring scripts that extract exact metrics
- Write admission controllers that inspect object specs
- Debug scheduling failures by reading taints and conditions
- Automate cluster reports without parsing human-readable tables