Most of My AI Pipeline Isn't AI

Most of my "AI pipeline" isn't AI. By share of the code, and by share of the actual work, the part that turns a messy inbox into clean, linked notes is plain Python. The language model is one small, expensive component I call as rarely as I can manage. It didn't start that way. I built the expensive version first.

The expensive first draft

The first version let the model do everything. It read each email and parsed the From: and Subject: lines. It matched names to people. It scored how important the message was. It grouped replies into threads, caught duplicates, and decided what to write to disk. End to end, one long act of language-model judgment.

It worked. It was also slow, costly, and quietly unreliable. Feed it the same inbox twice and you'd get two slightly different results, because a model is a probability machine, not a parser. And an LLM that gets header parsing "mostly right" is worse than five lines of Python that get it exactly right every time, because now you're debugging a hallucinated field instead of a regular expression.

The line that matters: fuzzy versus settled

It started with one distinction. A pipeline like this is really two kinds of work tangled together:

• Settled work has exactly one correct answer. Parsing a header, looking a name up in a table, stripping a signature block, deciding whether two files are duplicates, writing YAML to disk. There is a right answer and a wrong one.
• Fuzzy work needs judgment. What is this email actually about? What did someone commit to? Is this borderline message worth a full note?

Code is perfect and nearly free at the settled work, and hopeless at the fuzzy work. A language model is the exact reverse: good at judgment, wasteful and unreliable the moment you make it do clerical work. So the whole game is to split the task along that line and give each half to the tool that's good at it.

Let the script settle everything it can

In my pipeline, one Python script does all the settled work in a single pass before the model is ever called. Given a fresh inbox, it:

• detects what each file is (email, transcript, document),
• parses the headers, including the two awkward formats from the capture flow,
• strips the boilerplate (footers, disclaimers, signatures, link wrappers) with plain regex,
• swaps email addresses and phone numbers for tokens, so the model never sees raw contact details,
• resolves every name and address to a person through a registry, not a guess,
• scores each message's relevance by rule,
• groups replies into threads and drops duplicates,
• and pre-writes the filename and metadata for every note.

The output is one clean, structured file describing exactly what is in the inbox. By the time the model wakes up, there are no From: lines left to parse, no names left to guess, no duplicates left to catch.

What's left is judgment

Stripped of the clerical work, the model's job gets small and sharp. For each cleaned message it does four things: write a one-line summary I'll actually read, pull out the real action items, flag any committed decisions, and settle the rare "is this high or medium relevance" call the rules couldn't. That's it. Judgment, and nothing else.

It's better at that job because it isn't also parsing headers. A model asked to do one fuzzy thing on clean input is faster, cheaper, and steadier than the same model asked to do everything on raw input. You didn't make it dumber. You stopped distracting it.

Two moves that made it cheap

Two less obvious decisions did most of the work on cost and speed.

The first is keeping the heavy data out of the conversation. The preprocessing writes a full manifest, which can run to tens of kilobytes, to a file, and prints only a tiny summary to the part of the system that coordinates everything. That coordinator never reads an email body or the registry. It works from a two-kilobyte overview and hands the detailed file only to the workers that need it. Context you don't load is context you don't pay for.

The second is not making the model repeat itself. The worker agents write their results straight to a file and hand back a one-line pointer, instead of re-emitting thousands of lines of structured output back through the conversation. A plain script reads that file and does the writing. Pushing a big result back through the model costs you twice, in tokens and in wall-clock, for output you already had.

What it bought

Put together, a normal morning, a dozen emails and a few transcripts, finishes in a few minutes. The Python part takes seconds. The model only ever touches pre-chewed data, so it stays quick and consistent, and the bill stays small because most of the work never became tokens at all.

There is a quieter payoff too. The settled work is deterministic, so when something breaks it breaks the same way every time, and a reproducible bug is a fixable bug. It is also testable in the ordinary way: I have unit tests on the header parser and the dedup logic, and they catch regressions before a single token is spent. You cannot write a unit test for "summarize this thread well," which is exactly why you want as much of the pipeline as possible living on the side you can test.

For something I run on my own machine, the money was never really the headline. It rounds to nothing either way. What I wanted was the same inbox producing the same result twice, and something to point at when it doesn't. My scoring rules are not the best possible rules. They're written down, which means when they call a message important and it plainly isn't, I change one line and every run after that is better. A prompt that felt about right last month gives me nothing to grab.

The smartest part is how little of it is smart

The instinct, when you have a model this capable, is to hand it the whole problem and admire the result. The discipline is to hand it only the part that is actually ambiguous, and to push everything else down into code where it is exact and free. Most complaints about AI being too slow, too expensive, or too flaky are really one complaint underneath: someone asked a language model to do a job a function should have done.

Find the seam between the fuzzy and the settled, and put as little as you can on the expensive side of it. Across the whole pipeline, the best design decision was how little of it is the clever part.

Next: the layer that ties all of this together, the handful of commands that actually run the thing.