Direct vs async

There are two ways to run inference on inference.club. The direct path is a normal API call: you send a request and the response comes back on the same connection. The async path queues the work and returns immediately with a job you poll. Same models, same providers, same results — different control flow.

The direct path

This is the default and what every OpenAI client does. The platform authenticates you, picks an online provider, forwards the request over the tailnet, and streams the answer straight back. For chat it streams token by token; for media it returns the bytes (or a URL) when generation finishes.

Use it for anything interactive — chat, quick images, transcriptions — where you want the result now and the call finishes in a reasonable time.

The async path

Add "async": true to any JSON-bodied generation request and instead of waiting you get a job back immediately:

{ "id": "42", "object": "inference.job", "status": "QUEUED" }

A background worker picks the job up, runs it against a provider, and stores the result. You poll GET /v1/jobs/42 until the status is terminal (PROCESSED, FAILED, or CANCELED), then read result_url / result. Jobs can be listed, canceled, and retried. An Idempotency-Key header deduplicates retries so a network hiccup never enqueues the same work twice.

Why this exists: long-running generations (video especially) outlast a comfortable HTTP request, and async lets you fire many of them without holding connections open. It also gives the platform a place to retry failures and to respect capacity instead of overwhelming a GPU.

Batches

A batch submits up to 256 async requests as one atomic unit — handy for bulk generation (a hundred thumbnails, a dataset of transcriptions). Every item is validated up front: one malformed entry rejects the whole batch before anything is created. Each item becomes a normal job you can inspect individually, and the batch aggregates their statuses.

Workflows

A workflow is a DAG of steps — the right tool when one step's output feeds the next. Steps reference each other with {{ steps.<id>.output }} templates, and the engine queues each step as its dependencies complete. Step kinds:

• inference — call one model.
• map — fan out over a list, one job per item.
• transform — a pure data step (split, pluck, join) with no model call.
• collect — gather a fan-out's outputs back into one list.
• gate — pause and wait for a human to approve (or edit) before continuing.
• prompt — a meta-prompting LLM step that writes the prompt for a later step.

Each step runs as a queued job, so it inherits the same capacity limits, retries, and durability as a standalone async job. The live run renders as an SVG graph in Dashboard → Queue, and you can author workflows visually in the Workflow Studio.

Which should I use?

For request shapes and every endpoint, see the async jobs, batches, and workflows references.