Reducing the cost of re-sending context
The default way to ask several questions about one video is several requests. The Perceptron Files API took the byte transfer out of that loop: upload once, reference file-abc123 from then on. The prefill challenge remained unsolved:
- The model re-reads the media for every question. A file id saves the upload. It does not save the forward pass over the frames. Ten questions about a ten minute clip means ten prefills over the same frames.
The model re-reads the media for every question. A file id saves the upload. It does not save the forward pass over the frames. Ten questions about a ten minute clip means ten prefills over the same frames.
- Cost tracks question count, not media volume. A 60 frame clip carries roughly 8,600 tokens of context. The question you attach to it is a sentence. Adding a question adds a full context, not a sentence.
Cost tracks question count, not media volume. A 60 frame clip carries roughly 8,600 tokens of context. The question you attach to it is a sentence. Adding a question adds a full context, not a sentence.
Built for two specific workloads
Two patterns account for much of the media reuse we see in production. In both, the media is fixed and the questions are not.
- A fixed query set against one visual asset
A fixed query set against one visual asset
Query sets are often fixed. Every teleop episode gets the same twelve questions. Every inspection frame gets the same eight. Every social media clip gets a predefined moderation pass.
BASE ROUTE · FOUR REQUESTSMULTILOOK · ONE REQUESTvideo context8,624 tokquery 1answer 1video context8,624 tokquery 2answer 2video context8,624 tokquery 3answer 3video context8,624 tokquery 4answer 44 prefills · 4 admission slots35,612 tok at $0.15/Mvideo context8,624 tokprefilled oncequery 1answer 1query 2answer 2query 3answer 3query 4answer 41 prefill · 1 admission slot9,740 at $0.15/M + 25,872 at $0.0375/M46% of base input cost · 2.2× cheaperbilled at the input ratereused, billed at the cache-read rate
With Multilook, the first query pays for the context at the input rate, and the remaining P − 1 prompts pay for it at the cache-read rate. Nothing about the answers changes.
- A fixed query set against one visual asset
A fixed query set against one visual asset
Perceptron models support in-context learning. They can learn an image detection class (e.g., a niche brand logo, a car model) or a video detection class (e.g., OSHA violations, pre-recorded industrial SOPs) from visual examples, rather than from a text label. These visual examples are token-rich but remain byte-identical on every call.
Reuse is scoped to one call, so the saving comes from batching your targets into a single Multilook request. An identical call a second later prefills from scratch. Put the exemplar set in context and the targets in prompts, and the exemplars, which are the token-rich half, are computed once for the whole batch.
BASE ROUTE · SIXTEEN REQUESTSMULTILOOK · ONE REQUESTq1 · s1q1 · s2q1 · s3q1 · s4q2 · s1q2 · s2q2 · s3q2 · s4q3 · s1q3 · s2q3 · s3q3 · s4q4 · s1q4 · s2q4 · s3q4 · s4exemplar set re-sent 16 times16 prefills · 16 admission slots142,448 tok at $0.15/M3 exemplars8,624 tokprefilled oncen = 4 samplesquery 1query 2query 3query 4exemplar set sent once1 prefill · 1 admission slot9,740 at $0.15/M + 25,872 at $0.0375/M11% of base input costbilled at the input ratereused, billed at the cache-read rate
Put the exemplars in context and they are prefilled once for the whole batch. For example, drawing eight samples to vote over costs one prefill rather than eight, because n completions per prompt do not re-bill the prompt tokens.
Four prompts at four samples each bills about 11% of the input tokens the same sixteen requests would. The pricing follows how the call is processed: the shared context is billed at the full input rate once, each additional prompt reads it at the 25% cache rate, and extra samples cost only their output tokens.
How it works
- One shared prefix, prefilled once. context takes the same message format as a chat request. It is prefilled a single time per call, and every entry in prompts extends it.
One shared prefix, prefilled once. context takes the same message format as a chat request. It is prefilled a single time per call, and every entry in prompts extends it.
- Prompts are isolated. Each entry in prompts is the content of an implicit final user turn, either a bare string or a list of content parts. No prompt observes another prompt's text or completions.
Prompts are isolated. Each entry in prompts is the content of an implicit final user turn, either a bare string or a list of content parts. No prompt observes another prompt's text or completions.
- Reuse is scoped to the call. A per-request salt keeps the prefill private to the call that built it. An identical call a second later reports cached_tokens: 0. Nothing is shared across requests, across keys, or across organizations.
Reuse is scoped to the call. A per-request salt keeps the prefill private to the call that built it. An identical call a second later reports cached_tokens: 0. Nothing is shared across requests, across keys, or across organizations.
- Reuse is reported, not inferred. usage.prompt_tokens_details.cached_tokens is the subset of prompt_tokens served from the in-request prefill, and it is exactly what the cache-read rate is applied to.
Reuse is reported, not inferred. usage.prompt_tokens_details.cached_tokens is the subset of prompt_tokens served from the in-request prefill, and it is exactly what the cache-read rate is applied to.
Results come back in request order, one entry per prompt, with the reuse reported on the envelope:
Performance
If your workload asks several questions of the same image or video, or wants multiple samples per prompt, Multilook does in one API call what would otherwise take many parallel /v1/chat/completions requests, while delivering better latency and cost.
Per-unit wall time at concurrency=16
Shapep50 to p95, seconds
P=4, n=1
P=8, n=1
P=4, n=4
Base routeMultilookfilled marker p50, hollow marker p95, shared axis 0 to 44 s
Single Multilook call latency with number of prompts (P)
SamplesP=2P=4P=8P=16
n=1
4.0 s
2.00 s / prompt
P=2, n=14.0 s for 2 completions2.00 s per completion
3.7 s
0.93 s / prompt
P=4, n=13.7 s for 4 completions0.93 s per completion
4.5 s
0.56 s / prompt
P=8, n=14.5 s for 8 completions0.56 s per completion
8.6 s
0.54 s / prompt
P=16, n=18.6 s for 16 completions0.54 s per completion
n=4
3.5 s
1.75 s / prompt
P=2, n=43.5 s for 8 completions0.44 s per completion
6.1 s
1.52 s / prompt
P=4, n=46.1 s for 16 completions0.38 s per completion
10.0 s
1.25 s / prompt
P=8, n=410.0 s for 32 completions0.31 s per completion
18.3 s
1.14 s / prompt
P=16, n=418.3 s for 64 completions0.29 s per completion
bars share one scale, 0 to 20 s · per-prompt time falls from 2.00 s at P=2 to 0.54 s at P=16
On latency, adding prompts to a call costs far less than adding calls: the shared media is processed once, so per-prompt time falls as calls get bigger, multi-sample requests return in a fraction of the time the per-request pattern needs, and response times stay flat under load where the parallel pattern queues against its own duplicated work.
Input cost by prompts per call
PromptsAt input rateAt cache-read rateInput cost vs separate requests
P=29,1828,624
64% · 1.6× cheaper
P=49,74025,872
46% · 2.2× cheaper
P=810,85660,368
36% · 2.7× cheaper
P=1613,088129,360
32% · 3.1× cheaper
Separate requestsMultilookdashed line marks the 25% floor · input cost only
On cost, the shared context is billed at full rate exactly once and every additional prompt reads it from cache at a fraction of the price, and each prompt is billed once no matter how many samples you draw. This creates savings that grow with the number of prompts and samples. In terms of implementation, it replaces a client-side orchestration problem. Now, with a single request, the media is sent once, and one response is grouped per prompt.
Built for production
- Partial failure is per prompt. A prompt that fails returns an error object in place of its completions, and the others are unaffected. The call returns 200 if at least one prompt succeeded. If all of them fail, the request returns the first error's status.
Partial failure is per prompt. A prompt that fails returns an error object in place of its completions, and the others are unaffected. The call returns 200 if at least one prompt succeeded. If all of them fail, the request returns the first error's status.
- A separate rate bucket. 150 requests per minute, independent of /v1/chat/completions, so a Multilook batch job does not eat your interactive budget.
A separate rate bucket. 150 requests per minute, independent of /v1/chat/completions, so a Multilook batch job does not eat your interactive budget.
- Bounded by design. Up to 16 prompts, 64 completions, and 256 image or video inputs per call, inside a 20 MB body and a 300 second budget. On timeout, retry with fewer prompts, a lower n, or a smaller max_completion_tokens .
Bounded by design. Up to 16 prompts, 64 completions, and 256 image or video inputs per call, inside a 20 MB body and a 300 second budget. On timeout, retry with fewer prompts, a lower n, or a smaller max_completion_tokens .
Applications
Robotics and automation. Teleop annotation against a standing annotation set: subtask boundaries, grasp attempts, success or failure, and an episode quality score for filtering.
Manufacturing and industrial. Continuous procedure and safety monitoring against an SOP, leveraging in-context visual examples, rather than fine-tuning.
Media and content. Content moderation where every upload requires a full policy pass (e.g., brand safety, age rating, caption).
Geospatial and critical infrastructure. One aerial view of a corridor queried for serveral characteristics, for example vegetation encroachment, hardware condition, and access road state.
Security and surveillance. Event monitoring and triage where every detection requires detailed analysis: what happened, is this actionable, and does it match exemplars of past false alarms.
Retail. Operational monitoring where a single shelf capture requires analysis of multiple attributes, for example stockouts and price tag legibility.
Getting startedMultilook is available today.
EndpointPOST api.perceptron.inc/v1/chat/completions/multilook
Model idperceptron-mk1
Input$0.15 / M tokens
Cache read$0.0375 / M tokens
Output$1.50 / M tokens
Prompts per call1 to 16
Completions per call64 (prompts x n)
Media inputs per call256, each video frame counts as one
Rate limit150 / min, separate bucket
Request budget300 s
