When not to use an LLM

LLMs are useful when the input is messy and the output can be approximate. They are a poor default when the work is already a function.

Prefer deterministic software when

• The output must be exact. Totals, access decisions, schema migrations, compliance attestations.
• The rules are known. If you can write a policy, write a policy. Do not ask a model to remember it.
• The action is irreversible and you do not have a separate approval gate.
• You need the same answer tomorrow. Models, prompts, and hidden system text drift.
• You cannot afford a silent miss. Classification into a legal or safety bucket is a classifier problem first.

An LLM can draft. A rule engine should decide.

Prefer search, retrieval, or a form when

The user needs a known record, not a generated paragraph. “What is the SLA for vendor X?” is a lookup. Generating a confident paragraph from a stale chunk is how RAG demos embarrass themselves in production.

If the interface can be a filter, a query, or a workflow button, start there. Add language only where the input is unstructured.

Prefer a human when

The decision is rare, high-stakes, and poorly specified. Automating the 12th such decision this year is usually more expensive than routing it.

Hybrid is the normal architecture

Most production systems that “use AI” are not “an agent.” They are:

1. Deterministic intake and authorization
2. Optional retrieval
3. A model step with a schema
4. Validators
5. A human or a policy check
6. A write to a system of record

The model is one box. If you cannot draw the other boxes, you are not ready to pick a model.

A practical test

If you deleted the LLM and the workflow still has a correct, cheaper path — even if it is uglier — do not put the LLM on the critical path. Use it for draft, rank, or extract, behind checks.

“When not to use an LLM” is not anti-AI. It is how you keep the AI you do ship from inheriting work it cannot own.