vLLM + SGLang Inference Engine Deep Dive

From PagedAttention to RadixAttention, from scheduling preemption to structured output — a systematic guide to modern LLM inference engine algorithms and design philosophy.

1. 1

   LLM Inference Engine Landscape: vLLM, SGLang, Ollama, and TensorRT-LLM

   Intermediate

   #inference#vllm#sglang#ollama#tensorrt-llm
2. 2

   PagedAttention and Continuous Batching

   Advanced

   #paged-attention#continuous-batching#vllm#memory-management#kv-cache
3. 3

   Scheduling and Preemption: The Inference Engine Scheduler

   Advanced

   #scheduling#preemption#chunked-prefill#vllm#inference
4. 4

   Prefix Caching and RadixAttention

   Advanced

   #prefix-caching#radix-attention#sglang#vllm#kv-cache
5. 5

   SGLang Programming Model and Structured Output

   Advanced

   #sglang#structured-output#constrained-decoding#fsm#dsl