← use cases · Latency-focused operators
Latency-sensitive serving
Remove the scheduler, the syscall trap, and the userspace boundary so the CPU spends its cycles on the inference path and nothing else.
The problem
On a host OS, an inference process shares the machine with a scheduler, other processes, and a kernel that traps on every syscall. For a workload whose steady state is "do as much math as possible without being interrupted," those interruptions are exactly the tail-latency budget you are trying to protect.
Why cllm fits
- ▸No scheduler dispatching between unrelated processes — the main loop owns the CPU.
- ▸No syscall trap and no kernel/userspace transition on the hot path.
- ▸Deterministic boot and a flat memory model with no page faults.
- ▸A tractable place to apply ring-0 tricks to the inference path.
Honest caveats
- ▸Request handling is single-threaded today — one request at a time.
- ▸Inference itself is a stub until the engine integration lands, so end-to-end latency is not yet measurable.
- ▸We publish no latency benchmarks because there is no working inference path to benchmark.