Cost per inference
Cost per inference is the total cost of one complete model call or agent action — all input and output tokens across every step, plus any fixed per-request overhead — used to measure unit economics for AI features priced per action rather than per token.
Cost per inference vs. cost per token
Cost per token measures the price of a single unit of text; cost per inference measures the price of a single unit of product value — one chat turn, one document summarized, one agent task completed. A single inference can involve several underlying model calls (for example, a retrieval step followed by a generation step, or a multi-step agent loop), each with its own input and output token counts and potentially its own model and rate.
Teams that price a feature per action (per summary, per ticket triaged, per lead qualified) need cost per inference, not cost per token, to know whether that price actually covers the underlying cost.
What to include
A complete cost-per-inference figure sums every model call chained together to produce one user-facing result, plus any fixed costs that do not scale with tokens — retrieval or vector-search costs, function-calling round trips, or a flat per-request platform fee some vendors charge.
Worked example
An inference made of two chained model calls — a first call that retrieves and reasons over context, and a second call that generates the final answer — both billed at $3 per 1,000,000 input tokens and $15 per 1,000,000 output tokens.
Cost of a two-call inference
- Call 1 input cost = 1,500 tokens × ($3 ÷ 1,000,000) = $0.0045
- Call 1 output cost = 200 tokens × ($15 ÷ 1,000,000) = $0.0030
- Call 2 input cost = 800 tokens × ($3 ÷ 1,000,000) = $0.0024
- Call 2 output cost = 100 tokens × ($15 ÷ 1,000,000) = $0.0015
- Total cost per inference = $0.0045 + $0.0030 + $0.0024 + $0.0015 = $0.0114