If you follow AI safety, you know the uncomfortable truth: modern LLM guardrails are failing not because of complex zero-day exploits, but because of sloppy heuristics, brittle filters, and naive prompt-matching.
We built AURA to be pragmatic: deterministic, testable, fully auditable, and completely immune to the hallucination of its own telemetry.
Today, we’re unpacking AURA v0.1.0 — explaining our architectural decisions, showing real code excerpts, diving into non-linear risk scoring math, and solving the silent problem of lost repository analytics on GitHub.
1. System Architecture at a Glance
Data flow in AURA is intentionally simple, pipeline-driven, and fully reproducible:
- Source Cases (
public_cases/*.json) → Cleaned, validated, and normalized via scripts/normalize-percases.ts
- Rule Extraction (
scripts/extract-triggers.ts) → Contextual regex and sliding-window token analysis.
- Weights & Signal Mapping → Derived from
config/signal-mapping.json and saved to config/trigger-weights.json.
- Scoring & Policy Enforcement (
scripts/recalc_confidence.ts) → Non-linear math transform + cross-check audit logic via scripts/policy/crossCheckAdapter.ts.
- Persisted Telemetry → Automated daily snapshots written to
analytics/traffic-history.json via GitHub Actions.
2. Non-Linear Risk Normalization: Why We Don't Just "Sum Things Up"
We compute a transparent raw evidence sum (confidence_raw) and convert it into a bounded score via a diminishing‑returns exponential:
\[
\text{confidence} = 1 - e^{-\alpha \cdot \text{confidence\_raw}}
\]
This keeps scores in [0...1] and avoids noisy amplification from many weak cues. Implementation (excerpt from scripts/recalc_confidence.ts):
// computedRaw is the honest sum of trigger weights + cross-check contributions
const alpha = (typeof cfg.normAlpha === 'number') ? cfg.normAlpha : 1.0;
const normalized = 1 - Math.exp(-alpha * computedRaw);
let newVal = Math.round(normalized * 100) / 100;
if (newVal < minFloor) newVal = minFloor;
e.confidence_raw = Math.round(computedRaw * 100) / 100;
e.confidence = newVal;
Tuning Sensitivity (\({\alpha}\))
- Lower \(\alpha\) (0.1 - 0.2): Conservative normalization; requires heavier evidence to push confidence towards 1.0.
- Higher \({\alpha}\) (0.5+): Aggressive sensitivity for high-security environments.
- AURA Baseline (\(\alpha = 0.3\)): Striking a balance where \(\text{confidence\_raw} = 3.0\) yields \(\approx 0.59\), requiring compounding signals for a hard policy block.
Keyword lists are a recipe for false positives. Matching a naive string like "generate 500" catches harmless test cases alongside malicious payloads.
To balance precision and recall, AURA v0.1.0 introduces ordered-within-window token matching.
Sliding Window Engine
Excerpt from scripts/extract-triggers.ts showing how AURA tolerates small syntactic noise without matching scattered words across an entire prompt:
function containsOrderedWithinWindow(haystack: string[], needle: string[], window = 5): boolean {
if (needle.length === 0) return false;
if (needle.length === 1) return haystack.indexOf(needle[0]) !== -1;
for (let i = 0; i < haystack.length; i++) {
if (haystack[i] !== needle[0]) continue;
let idx = i + 1;
let matched = 1;
for (let k = 1; k < needle.length && idx < Math.min(haystack.length, i + window + 1); idx++) {
if (haystack[idx] === needle[k]) { matched++; k++; }
}
if (matched === needle.length) return true;
}
return false;
}
Contextual Anchors vs. Co-Occurrence Cues
In config/trigger-extraction.json, bulk asset creation explicitly requires a deception target:
{
"trigger": "non-consensual pattern generation",
"pattern": "\\bgenerate\\s+\\d+\\s+(?:deceptive\\s+assets|phishing\\s+emails|fake\\s+documents|fake\\s+profiles|fake\\s+accounts|malicious\\s+payloads|spam\\s+emails|synthetic\\s+attacks)\\b",
"description": "Bulk-generation demand with deception-specific targets."
}
For generic terms like "audit", we mandate multi-token co-occurrence:
{
"trigger": "unauthorized audit camouflage",
"cues": ["audit", "unauthorized", "independent", "bypass", "without permission"],
"description": "Require co-occurrence of 'audit' with authorization‑bypass phrasing."
}
4. Solving the GitHub Traffic "Blind Spot" (Because 14 Days Is Not "Project Growth")
I don't know about you, but I got tired of seeing my project's growth through the lens of GitHub’s default 14-day window. GitHub silently wipes daily clone and view metrics after two weeks, leaving open-source maintainers completely blind to long-term adoption trends unless they buy third-party analytics dashboards.
To solve this, AURA v0.1.0 includes an automated, self-healing snapshot pipeline in .github/workflows/traffic-history.yml.
How the Analytics Snapshot Pipeline Works:
- Daily Ingestion: Executes a daily cron job via GitHub REST API.
- Deduplication: Merges metrics into
analytics/traffic-history.json while purging duplicate artifacts.
- Self-Healing Merge Logic: If concurrent workflow updates cause a direct git push to main to fail, the action politely opens a temporary PR, squash-merges it via actions/github-script, and cleans up after itself:
- name: Auto-merge PR and delete branch
uses: actions/github-script@v6
with:
github-token: ${{ secrets.TRAFFIC_TOKEN }}
script: |
const head = `auto/traffic-report-${process.env.GITHUB_RUN_ID}`;
const { data: prs } = await github.rest.pulls.list({ owner: context.repo.owner, repo: context.repo.repo, head: `${context.repo.owner}:${head}`, state: 'open' });
if (prs && prs.length > 0) {
await github.rest.pulls.merge({ owner: context.repo.owner, repo: context.repo.repo, pull_number: prs[0].number, merge_method: 'squash' });
await github.rest.git.deleteRef({ owner: context.repo.owner, repo: context.repo.repo, ref: `heads/${head}` });
}
One of the best parts of open-sourcing AURA is having sharp contributors look at the edge cases. Our contributor Amirhossein Agrest spotted a subtle async race condition in how case state is updated before disk serialization.
updateCase() awaits cross-check evaluation internally. However, in certain batch execution flows, calling scripts invoked it across collections without awaiting the return promise before writing JSON files to disk.
The result? CLI logs proudly claimed success, while disk artifacts still contained pre-audit state.
// ❌ Potential Race: Fire-and-forget async invocation
for (const entry of entries) updateCase(entry);
// ✅ Fix Pattern: Await all async mutations before serializing to disk
const promises = entries.map(entry => updateCase(entry));
await Promise.all(promises);
We’ve logged this issue (shoutout to Amirhossein!) and are pairing it with artificial network-delay adapters in our test suite to guarantee filesystem persistence never outruns in-memory state in the upcoming patch.
6. Roadmap (What Realistically Comes Next)
- v0.2: Programmatic prompt tokenization + TF‑IDF experiments for ranking weak cues (automated test-corpus generation).
- Tooling UI: Better cross-trigger clustering and a visual rule editor — because manually squinting at hundreds of lines of raw JSON is a fast track to eye bleed, and my laziness is a major driver for automation.
- ML Augmentation: Replace some heuristics with small supervised models for cue disambiguation — but only where deterministic rules fall short (no ML for the sake of ML).
Closing Notes
AURA is not magic, and it doesn't pretend to be. It's a pragmatic stack: deterministic rules, auditable math, and CI that refuses to forget its history.
If you're looking for a silver bullet, good luck! But if you want a system that is testable, inspectable, and built on sound software engineering principles, welcome aboard.
⭐ Check out the repository, inspect the code, and give us a star:
👉 GitHub: kate8382/AURA
Open an issue, or even better — submit a PR with a failing unit test to help us catch edge cases faster.
