python-rules

---
paths:
  - "**/*.py"
---

# Python Security

The Python enforcement of the generic security discipline. Weighted toward the surfaces that data pipelines and scrapers actually hit.

## Subprocess
- Never `shell=True`. Never `os.system()`. Use `subprocess.run()` with a list of arguments.
- Never interpolate a variable into a command string (no f-strings, `.format()`, or `%` in the args list).
- Always set `timeout=` on `run()`, `call()`, `wait()`, and `communicate()`.

## Unsafe Deserialization
- YAML: `yaml.safe_load()` only. Never `yaml.load()` without `Loader=SafeLoader`.
- Never `pickle.load()` or `pickle.loads()` on data that could be untrusted; pickle executes arbitrary code on load.
- No `eval`, `exec`, or `marshal.loads` on external input. For literals, `ast.literal_eval()` is safe (it doesn't execute code).

## SSRF on Outbound Requests
- Validate every URL before an HTTP request when the URL came from anywhere untrusted (scraped content, search results, sitemaps, user input): resolve the host and reject private, loopback, link-local, reserved, multicast, and unspecified addresses.
- Re-validate the final URL after following redirects. A safe first hop can redirect to an internal address.
- URLs from operator config or environment are trusted; a scraped URL becoming part of a config-derived URL is not. Keep that boundary.
- Bound request volume (crawl and page budgets), set timeouts, and back off on retry.

## Regex on Untrusted Input
- Patterns that run over crawled or user-supplied text must avoid catastrophic backtracking: no nested quantifiers (`(a+)+`, `(.*)*`) or overlapping alternation inside repetition. Prefer several small anchored patterns over one greedy alternation.

## Secrets
- API keys, tokens, and passwords come from `os.getenv()` or an encrypted config, never hardcoded. `.env.example` files hold placeholders, not real values.
- Never log a secret. Redact sensitive fields before any logging.
- Key files on disk get `0o600` permissions.

## Files and Archives
- Validate archive member names before extracting a tar: reject entries containing `../` or starting with `/`. Validate `arcname` on `tar.add()` too.
- `os.walk(followlinks=False)`; remember `Path.rglob()` follows symlinks by default.
- Temp files from `mkstemp()` or `NamedTemporaryFile(delete=False)` get cleaned up in a `finally` block.

---
paths:
  - "**/*.py"
---

# Python Standards

Layers on the generic code-quality and architecture rules.

## Type Hints
- Annotate every public function's parameters and return type.
- Use `X | None` (3.10+), not `Optional[X]`. Use lowercase generics (`list[str]`, `dict[str, int]`), not `List` or `Dict`.
- Dataclasses for structured data, not ad-hoc dicts passed between functions.

## Naming
- `snake_case` for functions and variables, `UPPER_SNAKE_CASE` for module-level constants, `_prefix` for private or internal, `PascalCase` for classes and dataclasses.

## Logging
- `logger = logging.getLogger(__name__)` at module level. Never `print()` for anything that isn't a CLI's own stdout contract.
- Levels: DEBUG for detail, INFO for progress, WARNING for recoverable issues, ERROR for failures. Use `logger.exception()` to capture the traceback.

## Error Handling
- Catch specific exception types, never bare `except:`.
- Log and continue where a single item can fail without sinking the batch; let truly unexpected errors propagate to a top-level handler.

## Imports
- Import specific names; no `from x import *`.
- No circular imports. Keep config and constants in a leaf module that imports nothing local.

## Anti-Patterns
- No mutable default arguments (`def f(x=[])`).
- No hardcoded config. Values come from environment or a config module, not literals scattered through the code.
- No `print()` debugging left in committed code. Use the logger.
- Lint before committing (`ruff check` or your linter of choice).