关于分布式、自托管 LLM 推理的指南、动态与思考。
最新 EN
2026年6月12日 · @briancaffey
A 'healthy' mesh-generation service sat wedged for three days while my agent.yaml described services that didn't exist. So I moved four GPU boxes — three RTX 4090s and a DGX Spark — onto k3s and taught the inference-club-agent to discover services from the Kubernetes API instead of a config file. Health checks lie; queues don't. Config is fiction; clusters are testimony.
2026年5月30日 · @briancaffey
From GitHub sign-in to your first chat completion: a step-by-step quickstart for inference.club's OpenAI-compatible API, plus how to wire it into Open WebUI and the OpenAI SDK.
2026年5月30日 · @briancaffey
Local models keep getting better while the grid can't build centralized data centers fast enough. Span and NVIDIA's new XFRA puts Blackwell GPUs inside homes to tap idle power — strong validation for distributing AI compute to the edge, which is exactly the bet inference.club is making.
2026年4月26日 · @briancaffey
Why we're building inference.club — a self-hosted, OpenAI-compatible LLM inference network where you bring your own GPU or use someone else's, all behind one API key.
2026年4月26日 · @briancaffey
How inference.club uses an embedded Tailscale stack on the agent and a userspace sidecar on the server to route LLM traffic from a public OpenAI-compatible API into a private GPU on someone's home network — with no port forwarding, no public callback URLs, and no shared secrets per device.