Skip to content

fix(skills): remove a refused launch's new evidence dirs and the proof's init evidence - #372

Merged
ThomasK33 merged 1 commit into
mainfrom
fix/vcn-evidence-dir-cleanup
Nov 1, 2026
Merged

ThomasK33 merged 1 commit into
mainfrom
fix/vcn-evidence-dir-cleanup

Conversation

@ThomasK33

Copy link
Copy Markdown
Member

Why

The operator goal is to clear the open follow-up issues in this repository (#362 and #363 now) with small reviewed PRs. This PR covers items 8 and 9 of #363. Both are about the verify-claudecode-nvim skill.

  1. A refused launch leaves its new evidence dirs behind. vcn.sh launch checks the evidence dir again after mkdir -p. When that check refused, the dirs that mkdir -p had just made stayed on disk. The harness runs with umask 077, but a default ACL on a parent dir still makes mkdir create a 770 dir. Such a parent triggers the refusal on every run, with no race. In the default layout, the leftover group-writable verify-claudecode-nvim dir also blocks every later run at the first check.
  2. Every proof run leaves the init case's evidence in /tmp. proof-launch-refusals.sh gives the init case a fresh mktemp -d evidence dir and never deletes it. SKILL.md does not say to delete it. It also does not say that the cases that start a real run need Playwright Chromium for the user who runs the proof.

What changed

  • vcn.sh: before mkdir -p, launch records the dirs that are missing on the evidence dir's path, deepest first. When the check after mkdir refuses, launch removes those dirs with rmdir, then exits 1 as before. rmdir removes only empty dirs, and a dir that existed before launch is never on the list.
  • proof-launch-refusals.sh:
    • Every refused case now lists the proof's temp dir before and after the launch, and fails when a path was added or removed.
    • Two new cases use a default ACL (setfacl -d -m u::rwx,g::rwx,o::---) to reach the check after mkdir. out-made-unsafe uses a new VERIFY_OUT. parent-made-unsafe uses the default parent under an existing XUM_SCRATCH_DIR, which must survive. Both print SKIP when setfacl cannot set a default ACL.
    • The init case's evidence dir is deleted on exit when every case passes. When the init case fails, the proof keeps the dir and prints init evidence kept in <dir>.
  • SKILL.md: the VERIFY_OUT entry says that a refusal after mkdir removes the empty dirs that mkdir made. The proof entry says that out-dash and init need Playwright Chromium for the user who runs the proof, that the ACL cases need setfacl, and how the init evidence is cleaned up.

Scope

The verify-claudecode-nvim skill only: one harness function, the proof script and SKILL.md. No plugin code, no tests and no CI change.

Blast Radius

Agents and people who run the skill. A launch that passes both checks behaves as before. A refused launch now removes only the empty dirs it made. A passing proof run no longer leaves a dir in /tmp.

Verification

I ran every case below for real on this host (Neovim 0.12.3, agent-tty 0.5.0, Node 26). Each proof run used a fresh private TMPDIR, so leftovers were easy to see and other agents' runs could not interfere.

Run vcn.sh Result
Manual launch, VERIFY_OUT new under a default-ACL dir main exit 1, refused, leaves 770 $R/acl/out
Manual launch, XUM_SCRATCH_DIR = a default-ACL dir main exit 1, refused, leaves verify-claudecode-nvim and verify-claudecode-nvim/<time>-<id>, both 770
Old proof main 14 PASS, exit 0, leaves the init evidence dir (session.webm, final.png, ws-*.jsonl, ...) in TMPDIR
New proof (red) main FAIL: out-made-unsafe: launch changed ... > .../acl/out
New proof without out-made-unsafe (red) main FAIL: parent-made-unsafe: launch changed ... > .../acl/verify-claudecode-nvim > .../acl/verify-claudecode-nvim/<time>-<id>
New proof (green), twice this PR 16 PASS, exit 0, TMPDIR empty afterwards
New proof with the init check edited to want -32601 this PR FAIL: init: ..., init evidence kept in <dir>, exit 1, only that dir left
New proof with a setfacl that fails this PR 14 PASS, SKIP out-made-unsafe, parent-made-unsafe, exit 0
New proof with PLAYWRIGHT_BROWSERS_PATH = an empty dir this PR FAIL: out-dash: launch failed: vcn: agent-tty doctor failed: browser_cache_accessible .... A copy without out-dash fails init at the same doctor step.

shellcheck --exclude=SC1091,SC2016 and shfmt -i 2 -d (the options in treefmt.toml) report nothing on both scripts. treefmt --ci reports 0 changes, prettier --check passes on SKILL.md, mise run check reports 0 warnings, and mise run test reports 810 successes and 0 failures.

Part of #363.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

…f's init evidence

`vcn.sh launch` checks the evidence dir again after `mkdir -p`. When
that check refused, the dirs `mkdir -p` had just made stayed behind. A
default ACL triggers it with no race: under umask 077, `mkdir` still
makes a 770 dir. Launch now records the dirs that `mkdir -p` will make
and removes them with `rmdir` when it refuses. `rmdir` removes only
empty dirs, and a dir that existed before is never on the list.

`proof-launch-refusals.sh` now fails a refused case that adds or
removes a path under its temp dir. Two new cases, out-made-unsafe and
parent-made-unsafe, use a default ACL to reach the check after mkdir.
They print SKIP when setfacl cannot set one. The init case's
`mktemp -d` evidence dir is deleted when every case passes. When the
init case fails, the proof keeps it and prints its path.

SKILL.md says that out-dash and init need Playwright Chromium for the
user who runs the proof, that the ACL cases need setfacl, and how the
init evidence is cleaned up.

_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T02:00:44.914823Z 1054808 Manual request
🔒 Security Review ✅ Completed 2026-11-01T02:01:21.854633Z 1054808 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 1054808778

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 1054808778

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit ea0551a Nov 1, 2026
5 checks passed
@ThomasK33
ThomasK33 deleted the fix/vcn-evidence-dir-cleanup branch November 1, 2026 02:13
ThomasK33 added a commit that referenced this pull request Nov 1, 2026
Re-recorded on the merge with main (#373, #367, #372). The facts and the
model are unchanged by the merge, and check.sh passes. The only covered
file main changed outside the record is selection.lua, which cannot reach
diff state.

Part of #363

_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant