Context

Many LLM calls share the same shape: a large, fixed instructional prompt (rules, taxonomies, output format, examples) wrapped around a tiny variable input. Across Zeeguu’s fourteen LLM calls the template runs from a few hundred to several thousand characters, and it is re-sent in full on every call (Figure 1).

Fixed prompt-template size for the fourteen LLM calls in Zeeguu, in characters. Every call re-sends its whole template. The largest is 9,355 characters, the median 1,044, and the [`COMBINED_VALIDATION_PROMPT`](https://github.com/zeeguu/api/blob/master/zeeguu/core/llm_services/prompts/translation_validator.py#L8-L70) discussed below is 3,065. The variable input is a word, a phrase, or a sentence, except for the two article-level calls, where it is the article itself. This fixed overhead is what the pattern amortizes.

The work is offline and batchable: nobody is blocked on any single result.

Examples

Several Zeeguu jobs have exactly this shape. Rather than pay the preamble once per item, they batch (i.e., pack) related items into a single call, in one of two directions: fan-in (many inputs, one call) or fan-out (one input, many outputs):

• Meaning classification (fan-in) sends ~15 word-meanings per call, sharing one frequency/CEFR-type taxonomy prompt across the whole batch.¹
• Example-sentence validation (fan-in) checks ~20 generated examples per call.²
• Article simplification (fan-out) produces every CEFR level simpler than the original in one call, one section per level, turning four or five requests into one, about 75% fewer calls for a typical article.³

The three are one move seen along two axes. The first two amortize the preamble across inputs, the third across outputs, and in both directions the batch size is set by whichever ceiling binds first, which is not the same ceiling in the two directions.

Problem

Sent one item at a time, that fixed preamble is re-paid on every call and dominates token cost and latency. How can it be paid once instead of once per item, without blowing past the model’s quality or context limits?

Forces

• Preamble overhead. The preamble is large by necessity: detailed rules, taxonomies, and examples are what make the output accurate and consistent, so quality pulls it up, and it cannot be trimmed without losing quality. Sent one item at a time, that fixed preamble is re-paid on every call, in tokens, cost, and latency. The only lever left is to amortize it. (pushes toward bigger batches)
• Quality ceiling. Accuracy and consistency degrade as more items share one call; past ~15–20 small items some models start dropping or muddling entries. (pushes toward smaller batches)
• Context ceiling. Input and output must fit the context window; for fan-out the output side binds first, since each result is full-length.

Solution

At its core, this is map for LLM calls: apply one shared, expensive prompt across a batch so its setup is paid once, not once per item. It takes two forms:

• Fan-in batching packs many independent inputs into one prompt, spreading a large instructional preamble across the whole batch.
• Fan-out batching produces many outputs from a single input, emitting one section per output and collapsing several requests into one.

Fan-in packs many inputs behind one preamble; fan-out asks for many outputs (here, one simplified article per CEFR level) from one. Either way the preamble is paid once per call.

Both combine naturally with Anticipatory Precomputation (computing likely-needed results ahead of time): because results are computed offline, there is the luxury of batching.

Consequences

• Cost and latency amortize with batch size. The fixed preamble is paid once per call instead of once per item, so per-item token cost and wall-clock latency fall roughly inversely with the batch size.
• Batch size is capped by the tightest ceiling, and which ceiling binds flips by direction. For fan-in (many small items) the quality ceiling binds first (~15–20 items before the model drops or muddles entries), far below what the token window allows; for fan-out (full-length outputs) the token ceiling binds first, on the output side, since each result is full-length. So the workable batch size is workload-specific and must be tuned, not maximized.
• Applies only to deferrable work. Fan-in must wait to accumulate items, so the pattern fits offline / pre-computed paths (composes with Anticipatory Precomputation) and is unavailable when a user is blocked on a single result.

Known Uses

• The fan-in direction is prior art. Packing many independent inputs under one shared prompt is the published batch prompting technique (Cheng, Kasai & Yu, EMNLP 2023), which cuts token and time cost roughly inverse-linearly with batch size. This pattern’s contribution is to add the fan-out direction and to frame both as a single move: amortizing one fixed preamble.
• The fan-out direction maps directly to structured-output calls that emit several keyed results at once, and to the OpenAI/Anthropic multi-output n parameter (which requests several completions per call).

Notes

• Both directions can be conceptualized as the functional programming concept map, over a different axis. Fan-in maps the prompt over inputs, e.g. map(validate, examples). Fan-out maps over outputs for a fixed input, e.g. map(level -> simplify(article, level), levels). Either way the shared prompt is the function whose fixed setup the batch pays for once.
• Two adjacent provider mechanisms are not this pattern. Provider batch APIs (OpenAI Batch, Anthropic Message Batches) give ~50% off large asynchronous jobs, but each request still carries and pays for its own full preamble; they amortize scheduling and rate-limit overhead, not the in-prompt instructional overhead this pattern targets. Prompt caching (e.g. DeepSeek) instead caches a repeated prompt prefix and discounts it, which does amortize the preamble’s cost, but not its latency or the per-call overhead of many round-trips. Batching still earns its keep alongside either.

---

Escalate to the LLM →

💬 Open an issue about this pattern

1. The batched meaning-classifier prompt, create_batch_meaning_frequency_and_type_prompt in the zeeguu/api repository. ↩

2. The batch example-sentence validator, validate_examples_batch in the zeeguu/api repository. ↩

3. The multi-level simplification prompt, get_adaptive_simplification_prompt in the zeeguu/api repository. ↩