Role
Optimize machine-learning models for high-performance production inference, focusing on latency, throughput, GPU utilization, memory efficiency, and serving performance.
Responsibilities
Optimize inference performance of LLM, vision, and speech models.
Benchmark latency, throughput, concurrency, and GPU utilization.
Optimize models using quantization, compilation, batching, caching, and other techniques.
Work with inference frameworks such as vLLM, TensorRT, ONNX Runtime, TorchServe, or Ray Serve.
Optimize GPU memory usage and model execution.
Design efficient batching and request scheduling strategies.
Investigate performance bottlenecks across model, GPU, network, and serving layers.
Build reproducible inference benchmarks.
Work with ML engineers to make models production-ready.
Work with MLOps engineers on production serving infrastructure.
Requirements
Strong Python and deep learning knowledge.
Strong understanding of ML inference and model architectures.
Experience with PyTorch inference and GPU optimization.
Experience with at least one production inference framework.
Understanding of batching, concurrency, latency, throughput, and GPU memory.
Strong performance profiling and debugging skills.
Experience with Linux and Docker.
Nice to have
Experience with CUDA, TensorRT, Triton, or CUDA profiling.
Experience with vLLM and LLM serving.
Experience with quantization such as FP16, INT8, or MXFP4.
Experience with distributed inference.
Experience optimizing models on NVIDIA GPUs.