Use a batch API for independent AI requests whose results can wait through the provider’s processing window, reconciliation and any recovery work. Keep time-sensitive interactions on a real-time path. The discount applies to inference; it does not establish that your finished business task will cost half as much.

OpenAI and Anthropic both advertise a 50% batch discount against their standard synchronous API pricing. Their documentation also describes requests expiring before processing finishes. A product owner choosing between these modes has to price the wait and decide what happens to unfinished records.

Reserve time after the provider’s processing window

OpenAI’s Batch API guide describes asynchronous processing with a 24-hour completion window. Its expiration rules matter: unfinished requests are cancelled, completed results remain available, and completed work is charged.

Anthropic’s Message Batches guide also describes a 24-hour expiration window. The provider reports that most batches finish sooner, but that statement cannot set your customer’s delivery promise. Capacity, request volume and the actual workload affect completion.

Ask the workshop or workflow owner to set the latest acceptable delivery time. Subtract the time the team reserves for import checks and exception handling, then allow for file retrieval and input preparation. Give the provider the remaining processing budget. If that budget cannot accommodate its stated window, submit earlier or keep the deadline-bound work on the synchronous path.

Consider a hypothetical agency preparing a searchable archive of completed campaign assets for a workshop. Independent descriptions can be generated offline. If the workshop starts shortly after the full processing window ends, there is little room to find missing results or correct descriptions. Submitting earlier creates recovery time; changing the workshop promise creates a different commercial decision.

Record the budget explicitly:

StageDecision to settle before submission
Input preparationWhen will the source snapshot be ready and checked?
Provider processingWhat window does this endpoint specify, and what happens at expiry?
Result reconciliationWho checks that every submitted record has a known outcome?
RecoveryWhich failures can be corrected before the output is needed?
ConsumptionWhen must validated output be available in the actual destination?

A daily trigger alone answers none of these questions. Batch suits work with slack between source readiness and consumption. A customer waiting for an answer during a live support exchange has a different requirement.

Treat the submission as a cohort of records

A batch identifier tells you which job to retrieve. The consuming workflow still needs to know what happened to each record inside it.

OpenAI requires a unique custom_id for each request and warns that output order can differ from input order. Anthropic likewise recommends matching results by custom_id because order is not guaranteed. Joining the first result to the first source row risks attaching a valid description to the wrong asset.

Keep a submission ledger with a stable source identifier, source version, request identifier and batch identifier. When results arrive, match by identifier and record the request outcome before importing content. The source version matters when a record changes while the batch is running: a correct answer to yesterday’s input can be stale for today’s destination.

For the agency archive, the importer should distinguish a returned description awaiting checks from an accepted description attached to the intended asset. That separation makes missing records visible before the workshop owner assumes the archive is ready.

This ledger is an implementation recommendation, not a feature automatically supplied by either provider.

Recover individual failures without repeating accepted work

Anthropic exposes per-request outcomes including succeeded, errored, canceled and expired. OpenAI provides output and error files, including errors for requests that expire. Inspect those outcomes rather than treating a terminal batch status as evidence that every business record is usable.

Define recovery by failure type. Correct invalid inputs before retrying them. For expired requests, decide whether another batch fits the remaining time. Content failures need their own exception queue, because a response can complete successfully at the API and still produce an unusable description.

Keep the original source identity when creating a new attempt. Before applying the replacement result, check whether that record already has an accepted result for the relevant version. This protects the destination from duplicate imports when a recovery attempt overlaps work already processed by your application.

For a deadline-sensitive subset, a synchronous recovery path is a design option. It needs an explicit spend limit and an owner authorised to use it. Automatically resubmitting the whole cohort can erase part of the discount while making the completion ledger harder to interpret.

Calculate savings on the eligible work

The OpenAI Batch API FAQ states the 50% discount against synchronous APIs. Anthropic’s guide states a 50% reduction against standard API prices. Check model and endpoint eligibility before treating either rate as available for your workload.

Here is an illustrative calculation, using assumed spend rather than vendor model prices or measured performance. Suppose the same requests would cost $100 synchronously and all qualify for the advertised half-price batch rate. Initial batch inference costs $50. If failed or rejected cases require another $10 of synchronous inference, the combined inference bill is $60: a $40 reduction from the assumed baseline.

That leaves reconciliation, engineering maintenance and human correction outside the calculation. Their incremental cost determines how much of the $40 becomes an operating saving. A changed model or output length also changes the comparison, so retain actual usage rather than forecasting everything from request counts.

The implementation-cost guide covers the wider delivery budget. If your alternative is a cheaper model rather than a slower execution mode, use the smaller-first routing break-even test. Batching and model routing change different parts of the bill.

Check the storage boundary before moving sensitive work

Asynchronous processing requires the provider to hold inputs and results while the job runs. Retention eligibility must be checked for this endpoint, even when the organisation already uses the provider elsewhere.

OpenAI’s FAQ explicitly says zero data retention does not apply to its Batch API. Anthropic’s batch guide lists the feature as not eligible for zero data retention and describes result availability for 29 days after batch creation. These documented boundaries are procurement inputs, not a legal conclusion about your deployment.

For a consultancy handling client records, a lower token rate does not resolve whether the batch storage terms are acceptable. Confirm the permitted data, endpoint terms and retrieval/deletion responsibilities before submission.

Run a bounded rehearsal before switching the workload

Choose a representative, permitted dataset and keep the current path available. Compare the two modes against the same acceptance criteria. Record when inputs became ready, when results could be retrieved and when the destination contained usable output.

Deliberately include a missing-result case in the importer test, shuffle result order and replay a previously accepted record. Check that the reconciliation process exposes the gap, preserves source association and avoids a duplicate application. These are proposed acceptance tests; this article does not report an executed provider benchmark.

The adoption decision should name which records can wait, how unresolved cases are handled and the latest time the completed output can reach its consumer. A successful rehearsal supplies evidence for that decision. The advertised discount supplies only one input.

Model Operator is Alexander Chrisostomou’s Dubai-based AI product studio. Through AI Initiative Consulting, Alexander helps teams choose consequential AI workflows and define their implementation and adoption path. Bring one offline workload and its delivery deadline to establish whether batch processing is worth testing within that scope.