ML Inference & Optimization Engineer

ML Inference & Optimization EngineerML Inference & Optimization Engineer
Location: MumbaiExperience: 2-4 yearsYou will be responsible for deploying and scaling domain and task-specific LLMs and deep learning models for real-time and batch inferenceYoull work on quantization, model optimizations, runtime tuning, and performance-critical serving.What Youll DoIntegrate models into containerized services and APIs, and build high-performance inference pipelines optimized for latency, concurrency, and costDeploy and optimize LLMs using vLLM, TGI, SGLang, Triton, TensorRT etc.Implement model quantization, speculative decoding, KV cache optimization, dynamic batching etc.Benchmark model throughput and latency across cloud VM configurationsDebug performance bottlenecks: VRAM usage, token sampling speed, latency, instabilityCollaborate with infra team for scaling and observabilityMonitor and troubleshoot inference performance, ensuring system reliability and efficiencyStay abreast of advancements in model inference technologies and best practicesYou Bring3+ years of experience in deploying and optimizing machine learning models in production, with 1+ years of experience in deploying deep learning modelsExperience deploying async inference APIs (FastAPI, gRPC, Ray Serve etc.)Understanding of PyTorch internals and inference-time optimizationFamiliarity with LLM runtimes: vLLM, TGI, TensorRT-LLM, ONNX Runtime etc.Familiarity with GPU profiling tools (nsight, nvtop), model quantization pipelinesBonus: prior work on ElasticSearch, distributed KV cache, or custom tokenizers