sanitize-public

{
  "name": "sanitize-public",
  "displayName": "Sanitize Public: Leak Check Before Publishing",
  "description": "Check a repository, plugin or download for credentials and private details before it goes public. Scans the files and the full Git history, matches your own private names in any spelling, and walks a checklist for leaks a scanner can't see.",
  "version": "1.2.2",
  "author": {
    "name": "Ivan Misic",
    "url": "https://ivanmisic.net"
  },
  "homepage": "https://ivanmisic.net/toolshed/plugins/sanitize-public",
  "repository": "https://github.com/imisic/claude-marketplace",
  "license": "MIT",
  "keywords": [
    "secrets",
    "secret-scanning",
    "privacy",
    "leak-detection",
    "publishing",
    "git-history",
    "open-source"
  ],
  "documentationUrl": "https://github.com/imisic/claude-marketplace/tree/main/plugins/sanitize-public#readme",
  "privacyPolicyUrl": "https://github.com/imisic/claude-marketplace/tree/main/plugins/sanitize-public#privacy",
  "supportUrl": "https://github.com/imisic/claude-marketplace/issues",
  "termsOfServiceUrl": "https://github.com/imisic/claude-marketplace/blob/main/LICENSE"
}

# sanitize-public changelog

## 1.2.2

The skill now calls its scanner through `${CLAUDE_SKILL_DIR}`, so it runs from your project directory. The history section of `references/patterns.md` points at `publish_gate.sh` instead of a per-term `git log -S` loop, matching the skill itself. The README and storefront open with installation and requirements and separate the file scan from the history scan.

## 1.2.1

Clearer name, description and keywords for the plugin directory. The README now opens by saying plainly what the plugin does and for whom, and its privacy section separates what the plugin does from how Claude itself handles your conversation. Skill descriptions reworded for accuracy; no behaviour changes.

## 1.2.0

A new `publish_gate.sh` runs every mechanical check in one command: the tree you are about to push, every blob and commit message reachable from it, the author email on each commit, gitleaks as a second opinion, and a list of binaries to open by hand. Use `--also` for anything else that ships, such as an unzipped download bundle.

Private terms now match inside code. The scanner splits `BenchFooPanel` and `foo_bar` into words before applying your private rules, and the docs show how to write a rule that matches every spelling of a name.

The history sweep extracts every blob once instead of running `git log -S` per term, which never finished against a large private layer. The method also gained the author-identity check and the binary review.

## 1.1.2

The public guidance now makes the artifact-first gate and the history sweep clearer.

## 1.1.1

The README now says which command you get after installing.

## 1.1.0

The skills in this plugin now carry an `agents/openai.yaml` sidecar, so they show up in Codex's skill picker with a proper name and one-line description instead of a raw folder name. It stays implicitly invocable, so Codex can reach for it when a publish is near.

## Earlier

Versions before this predate this file. See the git history at https://github.com/imisic/claude-marketplace.

# sanitize-public

sanitize-public helps anyone publishing a plugin, repository or download check it for credentials and private details before it goes public. It scans the files and their full Git history for keys, passwords, internal hostnames, private IP addresses and home-folder paths, matches your own private names (employer, clients, internal projects) from a local list you keep, and walks a short checklist for the leaks a scanner can't see. Out of the box it knows only the generic patterns; add your own private terms to make it catch what matters to you.

Installing a Claude Code plugin **copies the plugin directory onto the installer's machine**, and relative-path plugins pull in the whole marketplace repo. There is no "delete it later" once something ships. This skill turns the pre-publish check into a deterministic scan plus a short manual-review checklist, instead of something you have to remember to do by hand every time.

Once it is listed, you can also add it from Anthropic's plugin directory: **Customize > Plugins** in claude.ai, or `/plugin` in Claude Code.

## Install

```
/plugin marketplace add imisic/claude-marketplace
/plugin install sanitize-public@imisic
```

Then the command is `/sanitize-public:sanitize-public`.

Needs Python 3 for the scanner. The full pre-push gate (`scripts/publish_gate.sh`, a set of checks that stops the push when any of them fails) also needs Git and either `gitleaks` or Docker; without one of them it reports the gitleaks layer as skipped and fails. Output is findings in the terminal.

The longer write-up lives on the storefront: [ivanmisic.net/toolshed/plugins/sanitize-public](https://ivanmisic.net/toolshed/plugins/sanitize-public).

## Quickstart

From a clone of this repo, in `plugins/sanitize-public/` (inside Claude Code, `/sanitize-public:sanitize-public` runs it for you):

```bash
python3 skills/sanitize-public/scripts/leak_scan.py <artifact> --severity medium
```

Point it at the artifact that will actually ship (a plugin directory, a
bundle folder, a repo root), not the wider tree you authored it in. Exit code
0 means clean at that floor; exit code 1 means findings to review. `high`
findings should never ship; `medium` findings almost never should.

```bash
python3 skills/sanitize-public/scripts/leak_scan.py <artifact> --severity high  # hard blocks only
python3 skills/sanitize-public/scripts/leak_scan.py <artifact> --json           # for a hook or CI
```

A clean scan means the text-pattern checks found nothing. It is not the same as "safe to
publish": run the manual-review section in
`skills/sanitize-public/references/patterns.md` for the contextual leaks a
regex cannot see (unreleased plans, a client named in prose, infra topology
described in words).

## The full gate

Run it from anywhere, pointing `--repo` at the repository you are about to publish (its root, not a subfolder, because the gate exports the whole tree at `--ref`):

```bash
bash /path/to/sanitize-public/skills/sanitize-public/scripts/publish_gate.sh \
  --repo /absolute/path/to/target/repository --ref <what-you-will-push> --identity <public-email>
```

For a Claude Code install, `/path/to/sanitize-public` is `~/.claude/plugins/cache/imisic/sanitize-public/<version>`; in a clone of this repo it is `plugins/sanitize-public`.

The scanner alone reads files. The gate also scans every commit reachable from the ref you are about to push, including old blobs and commit messages, checks that every commit carries the email you mean to publish under, runs gitleaks as a second opinion on secrets, and lists binaries for you to open. Add `--also <dir>` for anything else that ships, such as an unzipped download bundle. Any finding exits 1.

## What it runs

Everything happens on your machine. `leak_scan.py` is standard-library Python that reads the files you point it at and prints findings; it opens no network connection. `publish_gate.sh` runs `git` against your local repository and `python3`. If gitleaks is installed it runs that, otherwise it uses the `ghcr.io/gitleaks/gitleaks` Docker image with networking disabled, provided you have pulled it. Nothing is uploaded and no telemetry is sent.

## Local override (your own private identifiers)

Out of the box the scanner only knows generic shapes. Your own private
identifiers, an internal product name, a client codename, a work email
domain, a named infra host, never belong in a script that ships. Add them to
a local file instead:

- `$LEAK_SCAN_PRIVATE` if set, else `~/.config/leak-scan/private-terms` if it
  exists.
- One rule per line: `<low|medium|high> <regex>` (first whitespace splits
  severity from the regex). `#`-comments and blank lines are ignored.

That file lives only on your machine. The scanner reads it to do the matching; it is never copied into anything the plugin ships or publishes.

Write each term so it matches every spelling, because code spells a project name however its language prefers: `medium (?i)\bfoo[\s_\-.]*bar\b` catches `FooBar`, `foo_bar`, `foo-bar` and `Foo Bar`. The scanner also splits identifiers into words before applying your rules, so the same term inside `BenchFooBarPanel` is caught.

## Examples

- `/sanitize-public:sanitize-public plugins/my-plugin before I push it to a public repo.`
- `Run publish_gate.sh on the release branch I'm about to push, with my public email as the identity.`
- `Scan the unzipped download bundle in dist/bundle for credentials, internal hostnames and home paths.`

## Privacy

Everything runs on your machine. The scanner reads the files you point it at and prints findings; it opens no network connection. Your private terms stay in a local file the plugin never copies. The plugin itself collects nothing and sends nothing anywhere, and there is no telemetry. Your conversation with Claude is processed by Anthropic under its usual terms; that is separate from the plugin. Retention: the plugin's author never receives any of it, and the only thing kept is what the plugin writes in your own project, which you can delete at any time.

## Support

Questions, bugs or a security concern: open an issue at https://github.com/imisic/claude-marketplace/issues, or email hello@ivanmisic.net.

---
name: sanitize-public
effort: xhigh
description: >-
  Pre-publish privacy gate. Runs before anything crosses from a private working environment to a
  public one: a plugin marketplace repo, a downloadable bundle, a public GitHub repo or gist, a
  shared skill or command. Scans the exact files that will ship for key material, credentials,
  internal hostnames, LAN IPs, personal home paths, work-email domains, secrets plumbing and
  internal or client project names with a deterministic scanner, then does a human pass for the
  contextual leaks a regex cannot see. Fires on packaging, bundling, publishing, exporting or
  "shipping" content for other people, and on "is this safe to publish", even if the word
  "private" is never said, as long as the destination is public.
---

# sanitize-public

The gate between a private working environment and anything other people can
see. Most setups carry some private infrastructure worth keeping private (a
LAN host, machine names, encrypted secrets, work that cannot be named
publicly). Most of what gets authored is clean, but the cost of one leak
crossing into a marketplace others clone onto their own machines is high and
irreversible. This skill makes the check mechanical and repeatable instead of
a thing you have to remember.

## Why this exists (the marketplace fact that raises the stakes)

When someone installs a Claude Code plugin, the plugin directory is **copied
onto their machine**, and relative-path plugins clone the whole marketplace
repo. Whatever sits in that repo ships. The same is true of a downloadable
content bundle (a tar.gz someone fetches) and any public repo. There is no
"delete it later" once it is on other people's disks and in git history. So
the scan runs on the **artifact that will actually ship**, not the source
tree it came from.

## When this fires (and when it does not)

**Fires on:**
- Building or updating a plugin marketplace repo, or adding a plugin to one
- Packaging a downloadable bundle or exporting content for publish
- Pushing a new public repo or gist, or sharing a skill/command with someone
- Explicit ask: "is this safe to publish," "scan before shipping," "make sure
  nothing private leaks," "check this bundle"

**Does not fire on:**
- Work that stays inside a private surface (private repos, local-only files,
  a private Claude config repo). Those are allowed to contain private data;
  that is the point of them.
- Routine editing that is not about to be published. Do not nag mid-draft.

If the destination is ambiguous, ask one question: "is this going somewhere
the public can reach?" If yes, gate it.

## Rebuild, do not redact (the rule that actually prevents leaks)

**Never copy a private file into a public artifact and then remove the private parts.** Redaction can only remove what you thought to look for, so it fails on the thing you did not think of, every time. The scanner is a backstop for when this rule is broken, not a substitute for it. A scanner finding means the process already failed.

The failure is mechanical rather than careless. You build a substitution map from the identifiers you can name, run it over the copied file, and every identifier outside that map survives untouched. The map covers the people and terms you remembered; the file carries the ones you did not. A bare first name, an internal acronym, a market or product codename, a former employer: all of those read as ordinary words to you and to the scanner's generic layer alike.

**Port in two phases, with a hard break between them.**

1. **Read the private artifact and write a behaviour spec.** For a test, that is the function name plus what it asserts, in words. For a script, the mechanism, keyed to function and phase names. No fixture strings, no names, no subjects, no example data. If the spec contains something you would not publish, it does not belong in the spec.
2. **Close the private source. Write the new file from the spec alone,** inventing every fixture from the destination's own roster and vocabulary.

The assertions still match, because the code under test is identical. The content is authored rather than laundered, so there is nothing private left to find and remove.

**Fixtures and test data are the high-risk surface**, not prose. Nobody reads a test fixture the way they read a paragraph, so real names survive there. Treat any ported fixture as unsanitized until proven otherwise.

**Redaction is acceptable only for a file that is already public**, where you are editing in place rather than deriving a new artifact.

**A remediation commit must not quote what it removed.** Naming the value in the commit message, the changelog or the issue writes it straight back into history, in a place the working-tree scan will never look. Describe the shape and the location ("a first name in a fixture table"), never the value. The same applies to the report you hand someone: say what class of thing was found and where, and quote the string itself only when they need it to make a decision.

## Method

1. **Identify the shipping artifact.** Not the repo you author in, the thing
   that leaves. For a marketplace, that is the plugin directory (and the repo
   root if plugins use relative paths). For a content bundle, the folder that
   gets packaged. Point the scanner at that.
2. **Seed your own private terms first (once per machine).** The scanner
   ships knowing only generic shapes (key material, LAN IPs, home paths);
   it cannot know your employer domain, your client codenames, or your
   internal product names, and a scan blind to those returns a false
   "clean" that is worse than no scan. Check for the override file
   (`$LEAK_SCAN_PRIVATE`, else `~/.config/leak-scan/private-terms`); if it
   is absent, offer to create it now, seeded with the private identifiers
   already visible in this session or environment, before running the
   scanner. If you proceed without one, say plainly the run only catches
   generic patterns. Format is under "Running the scanner" below.
3. **Run the scanner at the gate floor.** `python3 "${CLAUDE_SKILL_DIR}/scripts/leak_scan.py"
   <artifact> --severity medium`. Exit code 1 means stop and review. `high`
   findings never ship. `medium` (internal host, LAN IP, home path, a private
   email domain) almost never ship; each needs a reason. Re-run until clean.
4. **Do the human pass.** The scanner is blind to contextual leaks. Walk
   `references/patterns.md` "manual review" section against the content:
   unreleased plans, a client or vertical named in prose, infra topology
   described in words, a private opinion about a named person, a screenshot
   or attachment with private data, git history baked into a bundle.
5. **Report, do not silently pass.** State what shipped clean, what was
   flagged, and what was allow-listed and why. A clean scan is "the lexical
   layer is clean," not "safe to publish." Say that distinction out loud.
6. **Sweep git history when the history itself will ship.** The scanner above only reads the working tree. If the shipping artifact is a git repo whose history goes public too (a marketplace repo pushed as-is, a tarball that still has `.git` inside), a term can be clean in the current tree and still sit in an old commit. Do not loop `git log -S` once per rule: with a generated private layer there are thousands of terms and the loop never finishes. Extract every text blob reachable from what you will push, plus commit messages and authors, into one file and scan that once. `scripts/publish_gate.sh` does exactly this. Then run `git log -S"<term>"` only for the few terms that hit, to find the commit. Note that `-S` also lists the commit that *removed* a term, so read the message before concluding it is still there. A real hit means a history rewrite (`git filter-repo` or equivalent) or, usually cheaper, a fresh-history branch with one clean commit. **Check whether the commit is already on a public remote before promising to remove it.** If it has been pushed, it may already be cloned, forked or cached; a rewrite narrows the exposure, it does not undo it. Say so plainly.
7. **Check the author identity on every commit you will publish.** A fresh clone silently inherits the machine's global git email, which is often a work address. `git log <ref> --format="%ae %ce" | sort -u` should return only the identity you mean to publish under. Fix it with a repo-local `git config user.email` and a rewrite of the affected commits before they are pushed.
8. **Open every binary that ships.** Images, PDFs and archives are skipped by the scanner on purpose and counted in its output. A screenshot with a real terminal, tab, path or calendar in frame leaks more than any string, and no regex will ever see it.

**One command for the mechanical layers.** `bash "${CLAUDE_SKILL_DIR}/scripts/publish_gate.sh" --repo <absolute path to the repository root> --ref <what-you-will-push> --identity <public-email> [--also <unzipped-bundle>]...` runs the tree scan, the history extract, the identity check, gitleaks (via a local binary or a network-less Docker container, with the reviewed allowlist in `references/gitleaks.toml`) and the binary listing, and exits 1 on any finding. It refuses to run without a private layer unless you pass `--no-private`, because without your own terms the scan passes everything that matters. Point `--ref` at what will be pushed rather than your working branch, and pass anything else that ships (a downloadable bundle, exported page copy) through `--also`: the same leak can sit in the repo, its history and a download built from an older tree at once. The contextual pass in step 4 is still yours.

## Running the scanner

```bash
python3 "${CLAUDE_SKILL_DIR}/scripts/leak_scan.py" <path>                       # scan, medium floor (the gate)
python3 "${CLAUDE_SKILL_DIR}/scripts/leak_scan.py" <path> --severity high       # hard blocks only
python3 "${CLAUDE_SKILL_DIR}/scripts/leak_scan.py" <path> --json                # for a hook or CI
python3 "${CLAUDE_SKILL_DIR}/scripts/leak_scan.py" <path> --require-private     # fail if there is no private layer
```

Exit codes: `0` clean, `1` findings at or above the floor, `2` bad invocation, `3` no private layer under `--require-private`.

Every run reports which layers it used, how many files it read, and which it skipped as binary. **A run with no private layer says so on its own output and sets `"trustworthy": false` in `--json`.** It still exits 0, because a fresh install has no private layer yet. Once you have one, add `--require-private` in CI so a layer that goes missing fails the build instead of quietly narrowing the scan.

The pattern set in `scripts/leak_scan.py` is the source of truth;
`references/patterns.md` mirrors it in prose and carries the manual-review
layer. Out of the box the scanner only knows generic shapes (key material,
credential assignments, internal-host and LAN-IP patterns, home paths,
secrets-plumbing references). Your own private identifiers, an internal
product name, a client codename, a work email domain, a named infra host,
never belong in the shipped script. Add them instead to a local override file
that stays on your machine and is never part of what ships:

- `$LEAK_SCAN_PRIVATE` if set, else `~/.config/leak-scan/private-terms` if it
  exists.
- One rule per line: `<low|medium|high> <regex>` (the first whitespace
  splits severity from the regex). `#`-comments and blank lines are ignored.
- Present -> the scanner catches your private terms too. Absent -> generic
  layer only (this is what installing this skill gives you by default).

## The private layer is generated, not hand-written

The section above says the file exists. This says how to keep it honest. A hand-written `private-terms` holds the two dozen terms you thought of on the day you wrote it. Whatever system already holds your identities (a contact directory, an org chart, a CRM export, a notes vault) holds every name that actually exists. The gap between those two counts is where a leak lives, and it widens every time the real system grows and the hand list does not.

So generate the file rather than typing it. A small script that reads your source of truth, batches the names into a few compiled alternations and emits the `<severity> <regex>` format above is enough. Run it before a publish gate, and make it idempotent so regenerating is never a decision.

Two inputs stay manual, and they are the only two you edit by hand:

- **A vocabulary file** for internal terms no directory holds: employers, org units, internal product and project names, market names, infra hostnames. Add a term the first time you catch yourself typing it into public content.
- **An allow file** for names that must never be flagged: your own public identity, and the fictional roster of each public destination. Every line here is a deliberate blind spot, so keep it short and know why each one is there.

**Write vocabulary rules to match every spelling.** Code names a project however its language prefers, so a rule for `FooBar` alone misses `foo_bar`, `foo-bar` and `Foo Bar`. Split the term into its words and allow any separator run between them: `medium (?i)\bfoo[\s_\-.]*bar\b`. The scanner also re-reads every line with identifiers split into words (`BenchFooPanel` becomes `Bench Foo Panel`) before applying your private rules, so a name inside a class or module name is caught too. Add the names of your private repositories to the vocabulary as well: no directory lists side projects, and a project that exists on your disk is a name that can leak.

Keep those two wherever your config syncs between machines. Do **not** track the generated output: it expands to every real name you know, and it is one regenerate away on any machine.

**A scan is only as good as the machine's private layer.** The generated file does not travel with the repo, so a fresh machine scans with the generic rules alone and will happily call an artifact clean while it is full of colleagues' names. Regenerate there before trusting a clean result.

## Escape hatch

A line containing `ship-allow` is skipped, for a value that is public on
purpose (a published contact address, a documented public repo URL). Use it
only after looking at the line. An unexamined allow-list defeats the gate.

## Not a secret scanner for code security

This is about **publication**, not vulnerability. It does not audit for
injection, weak crypto, or exposed endpoints in running code. It answers one
question: is there anything in this artifact that should not leave your
private environment.

interface:
  display_name: "Sanitize Public"
  short_description: "Leak scan before anything ships publicly"

# gitleaks config for publish_gate.sh: the default ruleset plus the false
# positives that were reviewed and found to be prose, not secrets. Each entry
# needs the reason, because an allowlist nobody can explain is a blind spot.
[extend]
useDefault = true

[allowlist]
description = "reviewed false positives"
regexes = [
  # A review checklist line naming password-hashing algorithms. generic-api-key
  # reads "bcrypt/argon2" as a high-entropy token.
  '''^bcrypt/argon2$''',
]

# Leak patterns and the manual-review layer

Two halves. The **scanned patterns** are what `scripts/leak_scan.py` catches
deterministically; this section mirrors the script in prose so a human can
read the intent. The **manual review** is what a regex is blind to and a
person has to look for. A clean scan covers only the first half.

## Scanned patterns (the script is the source of truth)

The shipped `RULES` are **generic**: they contain no one's private
identifiers, so the script is safe to publish (it ships inside this plugin).
Your own private identifiers load at runtime from a local override file that
never ships (see "Local override" below).

### high (never ships, hard block)
- **Key material**: PEM / OpenSSH / PGP private key blocks, age secret keys.
- **Provider keys**: AWS access key ids, Google API keys, OpenAI-style `sk-`
  keys, GitHub `ghp_`/`gho_` tokens, Slack `xox*` tokens.
- **Bearer / Authorization** headers carrying a real token value.
- **Credential assignments**: `password=`, `secret=`, `api_key=`,
  `access_token=`, `client_secret=` with a concrete value (placeholders like
  `changeme`, `<...>`, `$VAR`, `env`, `your-...` are ignored).

### medium (almost never ships, needs a reason each time)
- **Internal hosts**: any hostname ending in `.lan`, `.local`, `.internal`,
  or `.home` (one generic rule covers every internal host on your network;
  no per-host pattern needed).
- **RFC1918 private IPs**: `10.x`, `192.168.x`, `172.16-31.x`.
- **Personal home paths**: `/home/<user>/`, `/Users/<user>/`.

### low (context-dependent, manual call)
- **Secrets plumbing**: mentions of a secrets manager, key file, or
  bootstrap script name. Fine in a private runbook, a leak in a public
  README.

## Local override (your private identifiers, never shipped)

Internal product names, project codenames, a work email domain, named infra
hosts: these are private to you and must not be baked into a script that
ships. They load from a local file instead:

- `$LEAK_SCAN_PRIVATE` if set, else `~/.config/leak-scan/private-terms` if it
  exists.
- One rule per line: `<low|medium|high> <regex>` (the first whitespace
  splits severity from the regex). `#`-comments and blank lines are ignored.
- Present -> full coverage locally. Absent (the shipped default) -> generic
  layer only.

When a new identifier enters your private set (a new machine name, a new
client, a new internal host), add it to the **local override file**, not to
`RULES` in the shipped script. The prose here follows the script's generic
layer, never leads it.

## Git history (when the artifact's `.git` ships too)

The scanner only reads the current working tree. If the shipping artifact still has its `.git` directory (a repo pushed as-is, a tarball that forgot to strip it), a term can be gone from the tree today and still sit in an old commit. Run `scripts/publish_gate.sh` on the ref you will push: it scans every blob reachable from that ref, plus commit messages and authors, in one pass. Use `git log --all -S"<term>"` only afterwards, to find the commit behind a specific hit, and read it before concluding anything, because `-S` also lists the commit that removed the term. A real hit means a history rewrite or, usually cheaper, a fresh-history branch with one clean commit. If the commit is already on a public remote, a rewrite narrows the exposure; it does not undo it.

## Manual review (the scanner cannot see these)

Run these by eye against the shipping artifact. Every one has leaked from
someone at some point with zero keyword a regex could catch.

- **Unreleased or internal plans.** Roadmap detail, an unshipped feature, a
  dated internal decision. Public copy is work-focused and does not preview
  private plans.
- **Named people or clients.** A colleague, a client, a vertical named in
  prose. If you have a standing rule about which of your own details never
  appear in public copy, this is where it applies. No opinion about a named
  individual.
- **Infra topology in words.** Describing which machine talks to which over
  the LAN leaks the same map as a hostname would, with no string to match.
- **Absolute paths and usernames in examples.** Command examples that
  hardcode a real home path or machine instead of `~` or a placeholder.
- **Attachments and media.** A screenshot with a private tab, terminal, or
  path visible. A PDF/image the scanner skipped by extension.
- **Git history baked into a bundle.** A tar of a folder that still contains
  a `.git` with private commit messages or author emails. Ship the files,
  not the history.
- **Commented-out real values.** A disabled credential line the credential
  rule might miss if malformed, or a "TODO: remove before publish" that never
  got removed.
- **The over-share in a docstring or comment.** A comment that explains
  *why* using a private incident, a client's constraint, or an internal name.

## Reporting shape

After the gate, say plainly:
- what was scanned (the artifact, not the source tree),
- scan result at the `medium` floor,
- anything allow-listed and the reason,
- the manual-review outcome,
- and the honest caveat: clean scan = lexical layer clean, not "safe to
  publish." The human pass is what makes it safe.

#!/usr/bin/env python3
"""
leak_scan.py - scan a file or directory for private data before it ships to a
public surface (a plugin marketplace repo, a toolshed bundle, a public gist or
GitHub repo). Deterministic source of truth for the leak patterns; the
human-readable view is references/patterns.md. If the two disagree, this wins.

What ships vs what stays local. The RULES below are GENERIC: key material,
credential assignments, internal-host and LAN-IP shapes, personal home paths,
secrets-management plumbing. They contain no one's private identifiers, so this
script is safe to publish (it is itself shipped inside the sanitize-public
plugin). Your OWN private identifiers (internal product names, project
codenames, a work email domain, named infra hosts) load at runtime from a local
override file that never ships:

    $LEAK_SCAN_PRIVATE            if set, else
    ~/.config/leak-scan/private-terms   if it exists

That file is one rule per line: `<severity> <regex>` (first whitespace splits
severity from the regex; blank lines and #-comments ignored). With it present
you get full coverage locally; without it you get the generic layer only. Point
the scanner at the shipping artifact and it catches your private terms via the
local file; the copy someone else installs carries only the generic rules.

A MISSING local file does not make a scan pass, it makes it narrow. This layer
carries every real name, address and internal term, so without it the generic
layer alone will happily call an artifact clean while it is full of colleagues'
names. That is an observed failure, not a hypothetical. So a run without the
local file says so, loudly, on every line of output that reports a result, and
sets "trustworthy": false in --json. It still exits 0, because a fresh install
has no local file yet and a tool that refuses on first run reads as broken. Pass
--require-private to turn that warning into exit 3, which is what you want once
you have a local file and want CI to notice if it goes missing.

What it can and cannot see. The mechanical tells live here. The high-value
leaks are CONTEXTUAL and a regex cannot see them: unreleased plans, a client's
name, infra topology described in prose, a private opinion about a named
person. Those are in references/patterns.md for a human pass. A clean scan
means the lexical layer is clean, not that the content is safe to publish.
Plain Python, standard library only.

Escape hatch. A line containing  ship-allow  is skipped, for a value that is
public on purpose (the site's own public contact address, a documented public
repo). Use it sparingly and only after you have looked at the line.

Usage:
    python3 leak_scan.py <path>                  # scan a file or dir
    python3 leak_scan.py <path> --severity high  # only the hard blocks (gate tier)
    python3 leak_scan.py <path> --json           # machine-readable (for CI / hooks)

Exit codes: 0 = clean at the requested severity floor, 1 = findings at or above
the floor, 2 = bad invocation, 3 = no local private layer (scan not trustworthy).
Wire it as a gate: default floor is 'medium'.
"""

import argparse
import json
import os
import re
import sys

SEV_RANK = {"low": 0, "medium": 1, "high": 2}

# Each rule: (id, severity, regex, human note). Severity order:
# high  = near-certain secret or credential. Never ships. Hard block.
# medium = internal infra / personal identifier shape. Almost never ships; review.
# low   = a token that is often private in this context; manual call.
# These are GENERIC by design. Private identifiers live in the local override
# file (see load_local_rules), never here, so this script can ship as-is.
RULES = [
    ("private-key-block", "high",
     r"-----BEGIN (?:RSA |EC |OPENSSH |DSA |PGP )?PRIVATE KEY-----",
     "PEM/OpenSSH/PGP private key material"),
    ("age-secret-key", "high",
     r"AGE-SECRET-KEY-1[0-9A-Z]+",
     "age private key"),
    ("aws-access-key", "high",
     r"\bAKIA[0-9A-Z]{16}\b",
     "AWS access key id"),
    ("google-api-key", "high",
     r"\bAIza[0-9A-Za-z_\-]{35}\b",
     "Google API key"),
    ("openai-key", "high",
     r"\bsk-[A-Za-z0-9]{20,}\b",
     "OpenAI-style secret key"),
    ("github-token", "high",
     r"\bgh[pousr]_[A-Za-z0-9]{36,}\b",
     "GitHub personal access / OAuth token"),
    ("slack-token", "high",
     r"\bxox[baprs]-[A-Za-z0-9-]{10,}\b",
     "Slack token"),
    ("bearer-token", "high",
     r"(?i)\b(?:authorization|bearer)\b\s*[:=]?\s*['\"]?[A-Za-z0-9._\-]{20,}",
     "Authorization / bearer token with a value"),
    # `pass` on its own is ordinary English ("- Pass: document file list", "a
    # second pass") and fired far more often on prose than on credentials. A
    # high-severity rule that cries wolf is worse than no rule, because it is the
    # tier people are told never to ship past. password/passwd only.
    ("credential-assignment", "high",
     r"(?i)\b(pass(?:word|wd)|secret|api[_-]?key|access[_-]?token|"
     r"client[_-]?secret|private[_-]?token|db[_-]?pass\w*)\b\s*[:=]\s*"
     r"['\"]?(?!(?:null|none|true|false|changeme|<|\$|\{|env|your[_-])\b)[^\s'\"]{6,}",
     "credential assigned a concrete value"),

    # The trailing (?!\.\w) keeps ordinary filenames out: `settings.local.json`,
    # `config.local.yml` and friends are a widespread convention, not hosts, and
    # a rule that cries wolf on them teaches you to skim past its real findings.
    ("lan-host", "medium",
     r"\b[a-z0-9][a-z0-9\-]*\.(?:lan|local|internal|home)\b(?!\.\w)",
     "internal-only hostname"),
    ("private-ipv4", "medium",
     r"\b(?:10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}|"
     r"172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3})\b",
     "RFC1918 private IP"),
    ("home-path", "medium",
     r"/home/[a-z_][a-z0-9_\-]*/|/Users/[a-z_][a-z0-9_\-]*/",
     "absolute personal home path"),

    ("sops-age-ref", "low",
     r"(?i)\b(?:sops|age|keys\.txt|known_hosts|dev-bootstrap)\b",
     "secrets-management plumbing reference"),
]


# Populated by load_local_rules() so the report can state which layers actually
# ran. A scan that cannot name its layers is a scan you cannot act on.
LOCAL_LAYER = {"path": None, "rules": 0}

# What the scan actually looked at. A skipped file must be visible in the report:
# silent skips are how a gate reports "clean" on content it never opened.
SCAN_STATS = {"scanned": 0, "binary": 0, "unreadable": 0,
              "skipped_ext": 0, "skipped_names": []}


def load_local_rules():
    """Load private, machine-local rules that must never ship inside this
    script. Path: $LEAK_SCAN_PRIVATE, else ~/.config/leak-scan/private-terms.
    Format: one rule per line, `<severity> <regex>` (first whitespace splits).
    Missing file is normal (the shipped default) and yields no extra rules."""
    path = os.environ.get("LEAK_SCAN_PRIVATE") or os.path.expanduser(
        "~/.config/leak-scan/private-terms")
    # Recorded before the existence check: the "no private layer" warning has to
    # name the path it looked for, which is exactly the case where it is absent.
    LOCAL_LAYER["path"] = path
    if not path or not os.path.isfile(path):
        return []
    extra = []
    try:
        with open(path, "r", encoding="utf-8") as fh:
            for lineno, raw in enumerate(fh, 1):
                line = raw.strip()
                if not line or line.startswith("#"):
                    continue
                parts = line.split(None, 1)
                if len(parts) != 2 or parts[0] not in SEV_RANK:
                    sys.stderr.write(
                        f"leak-scan: {path}:{lineno} skipped (want "
                        f"'<low|medium|high> <regex>')\n")
                    continue
                sev, rx = parts
                try:
                    extra.append(("private-local", sev, re.compile(rx),
                                  "private identifier (local rule)"))
                except re.error as exc:
                    sys.stderr.write(
                        f"leak-scan: {path}:{lineno} bad regex: {exc}\n")
    except OSError as exc:
        sys.stderr.write(f"leak-scan: cannot read {path}: {exc}\n")
    return extra


COMPILED = [(rid, sev, re.compile(rx), note) for rid, sev, rx, note in RULES]
_LOCAL = load_local_rules()
LOCAL_LAYER["rules"] = len(_LOCAL)
COMPILED.extend(_LOCAL)

# Never descend into these; they are not the shipped artifact.
SKIP_DIRS = {".git", "node_modules", "vendor", ".claude", "storage",
             "__pycache__", ".idea", ".vscode"}
# Binary / non-text extensions we do not read.
SKIP_EXT = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".ico", ".pdf", ".zip",
            ".gz", ".tar", ".tgz", ".woff", ".woff2", ".ttf", ".mp4", ".mp3",
            ".wav", ".phar", ".lock"}


def iter_files(path):
    if os.path.isfile(path):
        yield path
        return
    for root, dirs, files in os.walk(path):
        dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
        for name in files:
            if os.path.splitext(name)[1].lower() in SKIP_EXT:
                # Counted, not silent: these are exactly the files that need a
                # human to open them (a screenshot with a real terminal in frame
                # leaks more than any string, and no regex will ever see it).
                SCAN_STATS["skipped_ext"] += 1
                SCAN_STATS["skipped_names"].append(os.path.join(root, name))
                continue
            yield os.path.join(root, name)


def split_identifiers(line):
    """The line with identifiers broken into words: `BenchFooPanel` becomes
    `Bench Foo Panel`, `foo_bar` becomes `foo bar`.

    Private rules are usually word-bounded, and `\\b` sees no edge inside
    camelCase or on either side of `_`, so a private project name inside a
    function or module name scans clean. Code is exactly where such names hide.
    Only private rules get this second look: the generic ones are shaped for
    raw text.
    """
    line = re.sub(r"(?<=[a-z0-9])(?=[A-Z])|(?<=[A-Z])(?=[A-Z][a-z])", " ", line)
    return line.replace("_", " ")


def scan_file(fpath, floor):
    """Scan one file. Decodes leniently on purpose.

    This used to open with errors="strict" and swallow UnicodeDecodeError, so a
    single stray byte anywhere in a file made the WHOLE file scan as clean with
    no message. That is the worst possible failure for a gate, and it was
    observed: a 4MB git-history extract containing two embedded PNG blobs
    reported zero findings while holding a real name in its text. Binary is now
    decided by a NUL byte in the first block (the standard heuristic), counted,
    and reported; everything else is decoded with errors="replace" and scanned.
    """
    findings = []
    try:
        with open(fpath, "rb") as fh:
            raw = fh.read()
    except OSError:
        SCAN_STATS["unreadable"] += 1
        return findings
    if b"\x00" in raw[:8192]:
        SCAN_STATS["binary"] += 1
        return findings
    SCAN_STATS["scanned"] += 1
    lines = raw.decode("utf-8", errors="replace").splitlines()
    for lineno, line in enumerate(lines, 1):
        if "ship-allow" in line:
            continue
        split = split_identifiers(line)
        for rid, sev, rx, note in COMPILED:
            if SEV_RANK[sev] < SEV_RANK[floor]:
                continue
            m = rx.search(line)
            if not m and rid == "private-local" and split != line:
                m = rx.search(split)
            if m:
                findings.append({
                    "file": fpath, "line": lineno, "rule": rid,
                    "severity": sev, "note": note,
                    "match": m.group(0)[:80],
                })
    return findings


def main():
    ap = argparse.ArgumentParser(description="Pre-publish leak scan.")
    ap.add_argument("path")
    ap.add_argument("--severity", choices=("low", "medium", "high"),
                    default="medium", help="minimum severity to report (gate floor)")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--require-private", action="store_true",
                    help="exit 3 when there is no local private layer (for CI)")
    args = ap.parse_args()

    if not os.path.exists(args.path):
        sys.stderr.write(f"path not found: {args.path}\n")
        sys.exit(2)

    # The local layer carries every real name, address and internal term. Without
    # it a "clean" result only means no key material and no LAN hostnames, which
    # is not what anyone reads it as. Say so up front rather than mislead.
    blind = LOCAL_LAYER["rules"] == 0
    if blind and not args.json:
        sys.stderr.write(
            "leak-scan: NO LOCAL PRIVATE LAYER. Generic rules only, so this scan\n"
            "cannot see names, employers, internal products or work addresses.\n"
            "  add one: " + LOCAL_LAYER["path"] + "\n"
            "  format:  one rule per line, `<low|medium|high> <regex>`\n"
            "  in CI:   --require-private to make this an error instead\n")

    all_findings = []
    for f in iter_files(args.path):
        all_findings.extend(scan_file(f, args.severity))

    all_findings.sort(key=lambda x: (-SEV_RANK[x["severity"]], x["file"], x["line"]))

    # Always state the layers. "Clean" is only meaningful next to what was checked.
    layers = (f"generic({len(COMPILED) - LOCAL_LAYER['rules']})"
              f" + private({LOCAL_LAYER['rules']} from {LOCAL_LAYER['path']})"
              if LOCAL_LAYER["rules"] else
              f"generic({len(COMPILED)}) ONLY - no private layer")

    if args.json:
        print(json.dumps({"findings": all_findings, "count": len(all_findings),
                          "layers": {"generic": len(COMPILED) - LOCAL_LAYER["rules"],
                                     "private": LOCAL_LAYER["rules"],
                                     "private_source": LOCAL_LAYER["path"]},
                          "read": dict(SCAN_STATS),
                          "trustworthy": not blind}, indent=2))
    else:
        print(f"leak-scan: layers = {layers}")
        skipped = SCAN_STATS["binary"] + SCAN_STATS["skipped_ext"]
        print(f"leak-scan: read {SCAN_STATS['scanned']} text file(s); "
              f"skipped {skipped} binary/media, "
              f"{SCAN_STATS['unreadable']} unreadable.")
        if skipped:
            print("leak-scan: OPEN THESE YOURSELF, a regex cannot read a "
                  "screenshot:")
            for name in SCAN_STATS["skipped_names"][:20]:
                print(f"           {name}")
            if len(SCAN_STATS["skipped_names"]) > 20:
                print(f"           ... and {len(SCAN_STATS['skipped_names']) - 20} more")
        if blind:
            print("leak-scan: WARNING, generic layer only. This scan cannot see "
                  "names, employers, internal products or work addresses.")
        if not all_findings:
            print(f"leak-scan: no findings at severity>={args.severity} ({args.path})")
            print("leak-scan: this is the LEXICAL layer only. Contextual leaks "
                  "(unreleased plans, a named client, infra described in prose) "
                  "still need the human pass in references/patterns.md.")
        else:
            for f in all_findings:
                print(f"[{f['severity']:>6}] {f['file']}:{f['line']} "
                      f"{f['rule']} -- {f['note']}\n         {f['match']!r}")
            print(f"\nleak-scan: {len(all_findings)} finding(s) "
                  f"at severity>={args.severity}. Review before shipping.")

    if all_findings:
        sys.exit(1)
    # Only the caller who asked for strictness gets it. Everyone else gets a
    # clean exit with the narrowness stated in the output and in `trustworthy`.
    if blind and args.require_private:
        sys.exit(3)
    sys.exit(0)


if __name__ == "__main__":
    main()

#!/usr/bin/env bash
# publish_gate.sh: every lexical leak layer in one run, before anything goes public.
#
#   publish_gate.sh --repo DIR --ref REF --identity EMAIL [--also PATH]... [--no-private]
#
# REF is what will actually be pushed, so the tree at REF and every blob and
# commit message reachable from it get scanned. --also adds shipped artifacts
# that are not that tree: an unzipped download bundle, an export of site copy.
#
# Why one script: each layer catches something the others cannot. The tree scan
# misses old commits, the history scan misses a bundle built from somewhere
# else, and neither sees a commit made under the wrong email. Layers that have
# to be remembered get skipped.
#
# The contextual pass (references/patterns.md, "manual review") still needs a
# reader. This script only makes the mechanical part impossible to forget.
#
# Exit 0 clean, 1 findings, 2 bad invocation.
set -euo pipefail

HERE="$(cd "$(dirname "$0")" && pwd)"
SKILL="$(dirname "$HERE")"
PRIVATE="${LEAK_SCAN_PRIVATE:-$HOME/.config/leak-scan/private-terms}"

repo="" ref="" identity="" no_private=0
also=()
while [ $# -gt 0 ]; do
  case "$1" in
    --repo) repo="$2"; shift 2 ;;
    --ref) ref="$2"; shift 2 ;;
    --identity) identity="$2"; shift 2 ;;
    --also) also+=("$2"); shift 2 ;;
    --no-private) no_private=1; shift ;;
    *) echo "unknown argument: $1" >&2; exit 2 ;;
  esac
done
[ -n "$repo" ] && [ -n "$ref" ] && [ -n "$identity" ] || {
  echo "usage: $0 --repo DIR --ref REF --identity EMAIL [--also PATH]... [--no-private]" >&2; exit 2; }
git -C "$repo" rev-parse --verify -q "$ref^{commit}" >/dev/null || { echo "no such ref: $ref" >&2; exit 2; }

work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
fail=0
step() { printf '\n== %s\n' "$1"; }
bad() { echo "FAIL: $1"; fail=1; }

step "1/6 private layer"
if [ -s "$PRIVATE" ]; then
  echo "$(grep -cvE '^\s*(#|$)' "$PRIVATE") rule(s) in $PRIVATE"
elif [ "$no_private" = 1 ]; then
  echo "WARNING: no private layer; only generic shapes will be caught"
else
  # Without your own names and terms the scan passes everything that matters.
  echo "no private layer at $PRIVATE; create one (see SKILL.md) or pass --no-private" >&2
  exit 2
fi

scan() {  # scan LABEL PATH
  if python3 "$HERE/leak_scan.py" "$2" --severity medium > "$work/scan.txt" 2>&1; then
    echo "$1: clean"
  else
    grep -A1 -E '^\[' "$work/scan.txt" | grep -v '^--' | sed "s|$work/||"
    bad "$1: findings above"
  fi
}

step "2/6 tree at $ref"
mkdir -p "$work/tree"
git -C "$repo" archive "$ref" | tar -x -C "$work/tree"
scan "tree" "$work/tree"
for p in "${also[@]}"; do scan "also $(basename "$p")" "$p"; done

step "3/6 history reachable from $ref"
# One extract of every text blob, rather than `git log -S` per rule: with a
# generated layer of thousands of names, a per-term loop never finishes.
: > "$work/history.txt"
git -C "$repo" rev-list --objects "$ref" | awk '{print $1}' | sort -u | while read -r o; do
  [ "$(git -C "$repo" cat-file -t "$o")" = blob ] || continue
  git -C "$repo" cat-file -p "$o" | grep -qIm1 . && git -C "$repo" cat-file -p "$o" >> "$work/history.txt" || true
done
git -C "$repo" log "$ref" --format='%B%n%an %ae%n%cn %ce' >> "$work/history.txt"
echo "$(git -C "$repo" rev-list --count "$ref") commit(s), $(wc -c < "$work/history.txt") bytes"
scan "history" "$work/history.txt"

step "4/6 commit identities"
others="$(git -C "$repo" log "$ref" --format='%ae%n%ce' | sort -u | grep -vxF "$identity" || true)"
if [ -n "$others" ]; then bad "identities other than $identity: $(echo "$others" | tr '\n' ' ')"; else echo "all commits by $identity"; fi

step "5/6 gitleaks"
if command -v docker >/dev/null && docker image inspect ghcr.io/gitleaks/gitleaks:latest >/dev/null 2>&1; then
  for target in tree history.txt; do
    if docker run --rm --network none -u "$(id -u)" -v "$work:/w:ro" -v "$SKILL/references/gitleaks.toml:/c.toml:ro" \
        ghcr.io/gitleaks/gitleaks:latest dir "/w/$target" -c /c.toml --no-banner --redact >"$work/gl.txt" 2>&1; then
      echo "$target: clean"
    else
      grep -E 'RuleID|File|Line:' "$work/gl.txt" | sed 's|/w/||'; bad "gitleaks: $target"
    fi
  done
elif command -v gitleaks >/dev/null; then
  for target in tree history.txt; do
    gitleaks dir "$work/$target" -c "$SKILL/references/gitleaks.toml" --no-banner --redact >"$work/gl.txt" 2>&1 \
      && echo "$target: clean" || { grep -E 'RuleID|File|Line:' "$work/gl.txt"; bad "gitleaks: $target"; }
  done
else
  echo "SKIPPED: install gitleaks, or docker pull ghcr.io/gitleaks/gitleaks:latest"
  fail=1
fi

step "6/6 binaries for a human to open"
find "$work/tree" "${also[@]}" -type f 2>/dev/null | while read -r f; do
  grep -qI . "$f" 2>/dev/null || [ ! -s "$f" ] || echo "  $f" | sed "s|$work/tree/||"
done

echo
if [ "$fail" = 0 ]; then
  echo "GATE: lexical, history, identity and gitleaks layers clean. The contextual pass is still yours."
else
  echo "GATE: FAILED"
fi
exit "$fail"