Spec-Kit's Extension Manager Deleted a File It Didn't Own

Spec-Kit's Extension Manager Deleted a File It Didn't Own

●4 ●54 ●130
calendar_today ago • schedule8 min read
— Originally published at flamehaven.space

On November 11, 2026, a contributor opened two reports against GitHub Spec-Kit's extension manager.

In the first, installing an extension overwrote a SKILL.md the user had written, and removing the extension deleted the replacement, so the user's original content was lost [1]. The second came with a reproduction small enough to run in a temporary directory: install a sample extension, put my-notes.md beside its installed skill file, remove the extension, and look for the directory.

It was gone because the removal path called shutil.rmtree() on a directory holding material the extension never created [2]. Both reports concerned specific development revisions and remained open when checked on November 17.

Installers have overwritten files and cleanup routines have deleted populated directories for decades. Spec-Kit helps developers define requirements and check implementation progress. Yet its extension manager can make unsafe assumptions about file ownership.

A completed task list says nothing about a file the extension manager treats as disposable. Comparable gaps appear when agents store observations and when analysis tools assign a status to incomplete evidence.


The part of a convergence report that nobody checked

Spec-Kit's /speckit.converge command reads a feature's specification and implementation artifacts, looks for unmet requirements, and appends follow-up tasks to tasks.md when it finds them.

If the review finds no actionable gap, the command reports convergence and leaves tasks.md byte-for-byte unchanged [3]. Its assessment is confined to the requirements and code areas named in the feature artifacts.

Neither extension report names /speckit.converge as a cause of the deletion. Issue #5381 instead shows a removal routine holding ownership information for the installed SKILL.md and applying deletion to the directory containing it.

A user file placed there enters the removal path without an ownership check of its own. The feature can satisfy every requirement available to the agent while that assumption remains outside the review.

Birgitta Böckeler tested specification-driven development tools on an existing codebase. She found that the agent misunderstood existing code, duplicated functionality, and created extra Markdown review work [4]. The generated documents had to be reconciled with the actual implementation, a task that remained with the engineer.

In SWE-Bench Pro V2, researchers audited 731 coding-benchmark tasks, repaired 391 and discarded 89 after finding defects in task instructions and evaluation conditions [5]. The 480 figure counts tasks whose assessment needed correction or removal; it provides no measure of how many agents failed. An automated evaluation can execute as written while its acceptance conditions remain inadequate.

One way to test a verifier is to introduce a known bug into code that previously passed its checks. The review is then run again. If it approves the defective code, that reveals a gap in the review's coverage.

The inserted bug must be independently confirmed first. Otherwise, the experiment could produce a misleading result.


The working directory that disappeared from memory

Everything Claude Code, or ECC, packages reusable agent workflows with a persistent store of observations from earlier sessions. Reuse depends on knowing which repository an observation belongs to.

In issue #2746, filed in August 2026, a contributor found that an observation hook receiving no usable current working directory, cwd, could fall back to the directory inherited by the hook process. An observer started under the resulting global scope could continue to operate there [6].

Four of 26 observations in the reporter's current cycle had landed in global storage when they belonged to a particular project. That count comes from one installation. The agent could then apply a rule from one project to another, without knowing where it originally came from.

The hook parsed cwd but did not save it. Investigators could not tell whether the original event lacked a working directory or the system had simply failed to record it. The issue was closed on November 16, 2026.

Command hooks can miss alternative ways to run a blocked command. In an independent DEV Community experiment, one developer tested 14 variations of git push against Claude Code's permission rules. Eight were blocked, five reached a local test remote, and one passed the permission check but failed in the shell [7]. These results apply to the configuration tested in September 2026.

The experiment did not test ECC's security features. The author also later corrected details about their regular configuration. The results show the limits of command-based restrictions.

A deeper problem arises when agents can modify their own permission settings. In that case, critical restrictions need protection at the operating-system level, beyond the agent's control.


Two errors REA classified incorrectly

Reverse Engineer Anything, or REA, connects AI coding agents to software-analysis tools through the Model Context Protocol (MCP).

In October 2026, issue #4714 revealed a problem with REA's XML parser. It accepted a malformed archive containing an undeclared entity. The tool returned ok: true and marked the recovered value as observed, without warning that the XML was invalid [8]. The test used a harmless, generated file. The issue was later fixed.

A separate bug affected error reporting in Ghidra, a software reverse-engineering tool. In issue #4682, a simulated Ghidra process generated about 10.6 MiB of progress logs before reporting a Java OutOfMemoryError [9]. REA kept only the first 8 MiB of combined output, so the memory error was discarded. Instead of reporting a memory problem, REA returned a generic execution_failure and advised the user to run rea doctor.

With shorter logs, REA correctly classified the same error as resource_constraint. The test used a simulated Ghidra process, so its frequency in real sessions remains unknown. Maintainers fixed the error handling on October 19, 2026.


A record the evaluator had to rebuild

Pablo Díaz Jorge's independent rea-lab project conducted five studies on Windows, covering 482 executions. Of those, 336 used REA. Three additional runs were interrupted by a machine restart and later repeated, bringing the total to 339 REA-invoking executions [10]. None saved a persistent Evidence bundle or snapshot [11].


Figure 1. Evidence persistence across rea-lab's REA-invoking executions. Sources: [10], [11].

The study instructions told agents to close their analysis sessions but did not ask them to save Evidence. REA had an export function, but none of the 339 runs used it. After the sessions ended, the evaluator had to rebuild the records from client transcripts.

This caused a problem in one study. The grader initially failed to authenticate two of 144 citations because some large tool outputs were stored in separate files. After the evaluator updated the grader to read those files, both citations were verified [11].

The client transcripts allowed those records to be recovered, provided the relevant outputs were still available. A reviewer examining the analysis later therefore depends on a record retained by another component and on a grader that knows how to recover file-backed output. Díaz Jorge proposed an optional journal written by the REA server as each Evidence record is created, removing the need for this reconstruction step [11]. The proposal leaves open how to handle a journal write interrupted midway.


The benchmark grader had a bug

The LeanREA benchmark evaluated three configurations on the same model and machine, with twelve runs per configuration and 36 runs in total [12]. The setups were an agent without a specialized reverse-engineering skill, a lightweight command-line skill, and a REA MCP integration with its skill.


Figure 2. Results from 12 runs per configuration. All tests used the same model and machine, without Ghidra, Hopper, or IDA. Source: [12].

The lightweight CLI configuration used 388 seconds in total against REA's 627, approximately 38% less time, with reported API costs of $2.17 and $3.36 respectively. Its score was 74/75, one point below REA's 75/75. The no-skill baseline finished in 479 seconds and scored 67/75 after a grading correction. The benchmark measured a narrow task set without exercising REA's full native analysis capabilities.

The grader also rejected a correct answer, Go 1.27.1, because it expected go1.27.1. The authors corrected the no-skill score in their published results but did not report the original total. This error did not affect the one-point difference between the lightweight skill and REA.


After the agent finishes

These failures can be tested with simple, controlled experiments. First, install an extension, add a user file, and remove the extension. Does the file survive? Next, save an observation without a project identifier and check where it goes. Feed malformed XML to a parser and see whether it reports the error. Finally, introduce a known bug into working code and check whether the verifier detects it.

Each test needs a documented setup and an independently confirmed expected result. These experiments are proposed, not completed.

Each experiment should produce one of three outcomes. Verified means the evidence supports the checked property under stated conditions. Failed means the evidence establishes a violation. Unknown means the available evidence cannot establish a result, and the reason must be recorded.

An Unknown cannot become Verified without additional evidence. A command blocked by security policy should be recorded separately because it never reached the execution or verification stage.

Figure 3. Proposed verification workflow. Policy decisions and test results are recorded separately. This design has not been implemented or validated.

Once an agent finishes its work, the next engineer needs to know what was tested and whether the evidence is still available. In the Spec-Kit case, the extension manager deleted a directory containing both its own SKILL.md and the user's my-notes.md. The user had created the notes, but the removal command deleted them anyway.


References

[1] markuswondrak. (2026, November 11). *extension add overwrites a pre-existing user skill; extension remove then deletes it*. GitHub Spec-Kit, Issue #5380. GitHub.

[2] markuswondrak. (2026, November 11). Removing supplemental extension skills deletes unrelated user files (rmtree). GitHub Spec-Kit, Issue #5381. GitHub.

[3] GitHub. (2026). Spec-Kit convergence command template (converge.md). GitHub Spec-Kit Source Documentation.

[4] Böckeler, B. (2025, October 15). Understanding Spec-Driven Development: Kiro, Spec-Kit, and Tessl. MartinFowler.com, Exploring Gen AI.

[5] Dan, S., Da, J., Bhutani, S., Romero Calvo, M., Guevara, I., Carabedo, F., Bernal, J., Reyna, J., Monsalve Mendez, L. L., Liu, Y., & Zhang, D. Y. (2026). SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark. arXiv, 2610.24206.

[6] Karick143. (2026, August 10). continuous-learning-v2: Observations routed to global scope when hook payload lacks cwd (silent fallback + observer pinned at launch). Everything Claude Code, Issue #2746. GitHub.

[7] Rulestack. (2026, September 24; updated October 2). Claude Code permission rules: Bash(git push:) stopped 8 of 14 ways to push, and 5 reached the remote*. DEV Community.

[8] morluto. (2026, October 17). Keyed archive inspection reports recovered malformed XML as observed values. Reverse Engineer Anything, Issue #4714. GitHub.

[9] RishTheFish21. (2026, October 17). Ghidra: An OutOfMemoryError is not recognized once the session's 8 MiB output budget is used up. Reverse Engineer Anything, Issue #4682. GitHub.

[10] Díaz Jorge, P. (2026). rea-lab: Reverse engineering with AI agents on Windows, verifying, measuring and visualising the tools agents use. Independent research repository and preregistered comparative studies. GitHub.

[11] Díaz Jorge, P. (2026). Opt-in journal of a session's Evidence ledger, written as records are added. Reverse Engineer Anything, Issue #4809. GitHub.

[12] AICodeKing. (2026, October 16). REA vs a lean skill: Benchmark. LeanREA benchmark report. GitHub.

Research disclosure: The reported issues concern specific revisions and reproduction conditions. Independent experimental results are attributed to their original authors and have not been reproduced separately for this article. Figures 1 and 2 visualize source-reported data; Figure 3 is a proposed architecture. The final-section fixtures have not been run as comparative experiments.

Want more #ai & #architecture deep dives?

Join 5,943+ developers sharing real benchmarks, code breakdowns, and projects. Free forever.

1 Comment

0 votes

More Posts

The Sovereign Vault — A Comprehensive Guide to Protocol-Driven AI

Ken W. Algerverified - Jun 4

Local-First: The Browser as the Vault

Pocket Portfolio - Apr 20

The Esoteric Realms of Code

nevmenandr - Nov 14

Your AI Doesn't Just Write Tests. It Runs Them Too.

Kevin Martinez - May 12

The Languages of Machines and Humans: A Developer's Perspective

nevmenandr - Nov 4
chevron_left
5.3k Points • 188 Badges
South Korea • flamehaven.space
70Posts
37Comments
33Connections
Founder designing Sovereign AGI & Scientific AI systems — governance, reasoning models, medical/phys... Show more

Related Jobs

View all jobs →

Commenters (This Week)

8 comments
4 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!