menu
AI-SLOP-Detector

AI-SLOP-Detector

Leader posted 2 min read

Stop the Slop: A Dedicated Gatekeeper for Your AI-Generated Code

We’ve all been there. You ask an LLM to "refactor this function" or "add a quick utility," and it produces something that looks perfect. It’s clean, it’s idiomatic, and it runs... until it doesn't.

The issue isn't that AI writes bad code; it's that AI introduces a very specific class of defects: AI Slop. I'm talking about hallucinated package imports, unimplemented pass stubs, and buzzword-heavy comments that add zero value.

To solve this, I built AI-SLOP Detector. It’s designed to catch the "slop" before it ever hits your production branch.

What exactly is "AI Slop"?

AI slop isn't just a syntax error. It’s structural "noise" that includes:

  • Phantom Imports: Hallucinated package names that sound plausible but don't exist.
  • Placeholder Fatigue: Functions filled with TODO, pass, or NotImplementedError.
  • Cross-Language Pollution: Seeing java_equals or php_strlen logic accidentally leaking into your Python script.
  • Low Logic Density: Codebases that are 80% comments and "jargon" and only 20% actual executable logic.

Key Features

1. The "Phantom Import" Hunter

The detector identifies non-existent packages before you hit a ModuleNotFoundError. In the latest v3.0.2, it’s smart enough to understand your project's pyproject.toml and internal src/ layout, so it won't flag your own modules as hallucinations.

2. Geometric Mean Scoring (The "Vibe" Check)

In version 3.0.0, we moved from arithmetic to weighted geometric mean for scoring.

Why? Because in quality control, one zero should pull down the whole score. If your code has 0% used imports (DDC), it doesn't matter how "dense" the logic is—it’s probably slop.

3. Self-Calibration

The tool actually learns your coding style. If you ignore a warning and keep the code, the tool marks it as a potential false positive. If you edit a file after a high deficit score, it confirms the "slop". It then runs a grid search to optimize the weights specifically for your codebase.

4. Structural Coherence

Using AST (Abstract Syntax Tree) node distribution, it measures how "similar" your files are. If a new file looks structurally alien compared to the rest of your project, it flags it for a closer look.

️ Quick Start

You can run it without even installing it using uvx:

uvx ai-slop-detector mycode.py

Or add it to your CI/CD pipeline to fail builds that exceed a "slop threshold":

# Hard gate — fails build at deficit_score >= 70
slop-detector --project . --ci-mode hard --ci-report

I’d love to hear how you’re managing AI-generated code in your teams. Is "slop" becoming a bottleneck for your PR reviews? Let's discuss in the comments!

1 Comment

•

I ran your detector on your own repo since I saw Claude and Co-pilot listed themselves as co-authors.... By its own standards, stripped of the false positives, this codebase has real structural debt concentrated in its most important files. The measurement code is more complex than the code it would pass in review. The CLI decomposition is the kind of thing the tool was specifically built to catch and would call slop in someone else's repo. It rates itself CLEAN on aggregate but HALT on gate — which probably means the aggregate scoring is too lenient and the gate is the more honest instrument. That being said I ran it on my own repos and my computer blew up lol This is a fun and useful tool. I am running it on all my projects from this week and its genuinely Fun

2 votes
Show 3 previous replys
•

Haha, absolutely no need to apologize at all! Please don't be sorry! Honestly, we are just thrilled to get attention and raw, genuine feedback from a real user. Having someone dig in and have fun with it—even testing it on itself—is the highest compliment a dev tool can get.

I took a quick look at your GitHub and your site, and it looks like you are pushing incredibly hard toward some massive goals. Even if you say you were just having fun, I want to send a huge shoutout and encouragement to your vision. If there is ever any small way I can help out or collaborate, please don't hesitate to reach out.

Since you are already running it on your own repos, I wanted to share a few "secret tips" with you. The truth is, we didn't build ai-slop-detector for GitHub stars or to surprise other devs. We built it purely for our own survival, and we run it dozens of times a day.

Here is how we actually use it to supercharge our workflow:

  1. The Autonomous Patching Loop:
    If you use CLI agents (like Claude, Codex, or Antigravity), don't just use this as a passive scanner. You can prompt your AI agent to run the
    entire governance loop autonomously:
    • Instruct the agent: "Run ai-slop-detector [target_folder] --json."
    • Have the agent parse that JSON output to locate the exact structural deficits.
    • Command the agent to initiate surgical patching/refactoring based only on that JSON report.
    • Have it generate an MD patch report and re-scan automatically. If it hits ALL GREEN, it passes.
    • If it fails, the agent asks for your approval to run Loop 2 (deep scrutiny).
  2. MD Report Architectural Review:
    The .md report generated during this process is incredibly valuable on its own. Instead of feeding your entire raw codebase into an AI agent (which wastes context window and confuses the model), try isolating just the generated Markdown report. Handing that structured summary to an agent for
    high-level architectural review is a massive efficiency boost.
  3. The Self-Evolving Configuration (The LEDA Engine):
    You might have noticed the .slopconfig.yaml file. While developers manually craft the strict domain overrides and ignore lists, here is the craziest part: the very heart of that file—the mathematical weights (ldr, inflation, ddc) that dictate your final governance score—is entirely handed over to the machine. As you use the detector, it records your history. When the --apply-calibration command runs, the engine reverse-engineers your team's past behavior and Git history to calculate the mathematically optimal weights for your specific coding style. It then automatically evolves and overwrites the .slopconfig.yaml file. The more you use it, the smarter and more meticulously tailored the pattern inspection becomes for you.

If you integrate this loop into your workflow, the probability of ending up with spaghetti code drops to near zero. Code quality stays pristine, and bottlenecks are drastically reduced before they even merge. It’s an absolute game-changer.

Given your experience, I have no doubt you'll figure out even better ways to utilize it. Keep pushing forward with your projects—I’m rooting for your vision!

2
•

@[Flamehaven] Really appreciate the detailed workflow breakdown — the autonomous patching loop idea is exactly the kind of thing I was hoping to find uses for. Already running it on my own repos so having the agent parse JSON output and auto-patch is a natural next step.
Wanted to let you know I liked the tool enough that I turned it into a Claude Code skill. It's live now in my setup under ~/.claude/skills/slop/ — gives me slash commands like /slop (full project scan with interpreted results), /slop-file for single file deep scans, /slop-gate for CI pass/fail, and /slop-spar for adversarial validation via fhval. It auto-detects whether slop-detector is installed locally or falls back to uvx. The skill interprets the scores instead of just dumping raw output — explains patterns in plain English and surfaces fixes for CRITICAL/HIGH hits.
If that's useful to anyone else using Claude Code I'm happy to share the SKILL.md.

2
•

@[johnohhh1] That genuinely made our day. Building your own skill wrapper with /slop, /slop-file, /slop-gate, and /slop-spar — that's exactly the kind of organic integration we hoped someone would figure out. The uvx fallback is a smart touch; low-friction setup matters more than most people admit.

If you're open to sharing the SKILL.md, we'd love to link it from the repo's README as a community integration. No pressure — but if it saves even one other dev the setup time, it's worth it.

Also, just a heads up: v3.2.1 shipped overnight. The LEDA calibration loop is now fully wired in — the --apply-calibration flag actually rewrites your .slopconfig.yaml with the optimal weights derived from your own git history. After a few dozen scans it starts to feel like the tool was always tuned for your codebase specifically. Worth a pip install --upgrade ai-slop-detector the next time you run it.

Thanks again for digging in this deep. Feedback like yours is what keeps the math honest.

1

More Posts

Dashboard Operasional Armada Rental Mobil dengan Python + FastAPI

Masbadar - Mar 12

Each /slop Is a Calibration Signal — AI-SLOP Detector v3.6.0 and the Claude Code Skill

Flamehaven - Apr 30

I Wrote a Script to Fix Audible's Unreadable PDF Filenames

snapsynapseverified - Apr 20

The Hidden Program Behind Every SQL Statement

lovestacoverified - Apr 11

Python Code Reviews and Collaboration Best Practices and Tips

Abdul Daim - Oct 13, 2024
chevron_left

More From Flamehaven

Each /slop Is a Calibration Signal — AI-SLOP Detector v3.6.0 and the Claude Code Skill

How Do You Trust the AI Auditor? STEM-AI v1.1.2 and Memory-Contracted Bio-AI Audits

When an AI Pipeline Passes — But One Path Still Must Be Held: EXP-034

Related Jobs

View all jobs →

Commenters (This Week)

4 comments
2 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!