SlaveCode
.
Academy
DSA
Resources
System Design
More
Toggle theme (Current: Custom)
Inference Engineering Roadmap | SlaveCode
Inference Engineering
Step-by-step guide to becoming an Inference Engineer in 2026
Back
Roadmap
Vedios and Other Resources
Projects
AI Tutor
Inference Engineer
Models & Performance
Hardware
Software
Inference Techniques
Inference Beyond LLMs
Production
Basic Terminology
Latency Metrics
Application Context
Inference Phases
Model Strategy
Quantization Approaches
Speculation Algorithms
Embedding Architectures
Also visit the following related roadmaps
Learn more through this book
C++
AI Engineer
Machine Learning
Machine Learning
AI Engineer
C++
Docker
Kubernetes
Kubernetes
Docker
Inference Engineering
Related Roadmaps
AI Engineer Roadmap
ML Roadmap
C++ Roadmap
Docker Roadmap
Kubernetes Roadmap
Tokens Per Second
Latency vs. Throughput
Latency Percentiles
End-to-End Metrics
Real-Time Factor
AI-Native Applications
Online vs. Offline Inf.
Consumer vs. B2B
Model Selection
Model Evaluation
Fine-Tuning
Distillation
Reasoning Models
Shared vs. Dedicated Inf.
Encoder-Decoder Arch.
Neural Networks
Transformer Blocks
Attention Mechanism
KV Cache
Mixture of Experts (MoE)
Large Language Models
Diffusion Models
Image Generation Arch.
Few-Step Image Generation
Video Generation Models
Image & Video Models
Ops:Byte Ratio
Roofline Model
LLM Bottlenecks
Image & Video Bottlenecks
Bottleneck Analysis
FlashAttention
PagedAttention
Attention Variants
Attention Optimization
Compute
Memory
Instances & Interconnects
Sizing & Procurement
GPU Architecture
Hopper
Blackwell
Grace & Vera
GPU Models
Desktop Inference
Mobile Inference
Local & Edge Inference
Writing Custom Kernels
Kernel Selection & Fusion
CUDA
PyTorch
Transformers & Diffusers
Frameworks & Libraries
vLLM
SGLang
TensorRT-LLM
NVIDIA Dynamo
Inference Engines
Performance Benchmarking
Profiling Performance
Benchmarking & Profiling
Post-Training Quantization
Quantization-Aware Training
Measuring Quality Impact
Quantization
Draft-Target Decoding
EAGLE
N-gram
Speculative Decoding
Caching Approaches
KV Cache Storage
Cache-Aware Routing
Long Context Handling
Caching
Tensor Parallelism
Expert Parallelism
Pipeline Parallelism
Multi-Node Inference
Model Parallelism
NVIDIA Dynamo
Disaggregation
Video Processing for VLMs
Omni-Modal Models
Vision Language Models
LLM-based models
Embedding Model Inference
Embedding Models
Voice Activity Detection
Auto. Speech Recognition
Text-to-Speech (TTS)
Streaming TTS
Speech-to-Speech Models
Speech
Image Generation Inference
Video Generation Inference
Image & Video Generation
Dependency Management
NIMs
Containerization
Continuous Batching
Cold Starts
Routing & Load Balancing
LoRA-Aware Routing
Request Queueing
Scale to Zero
Indep. Component Scaling
Autoscaling
GPU Procurement
Capacity Management
Geo-Aware Load Balancing
Security and Compliance
Multi-Cloud
Zero-Downtime Deplo.
Cost Estimation
Observability
Testing & Deployment
Introduction
Inference vs. Training
Time to First Token
Prefill & Decode Phases
Inter-Token Latency
The Inference Stack
Asynchronous Inference
Open vs Closed Models
Arithmetic Intensity
NVIDIA
Other Accelerators
Model Formats & Runtimes
safetensors
ONNX Runtime
TensorRT
SGLang Genai-bench
AIPerf
Locust
ModelOpt
BERT-style models
Structured Outputs
Constrained Decoding