2 часа назад
Software Engineer, LLM Inference
- Slovenia
- Удалённо
AILLMDistributed systemsPyTorchGPUCUDAInferenceTriton
Описание
Responsibilities
- Build and optimize our vLLM-based inference stack for low latency, high throughput, and maximum GPU utilization.
- Optimize continuous batching, scheduling, prefill/decode, prefix caching, and KV-cache allocation and reuse.
- Optimize or implement CUDA and Triton kernels for attention, GEMMs, sampling, normalization, and other critical model operations.
- Evaluate and integrate technologies such as FlashAttention, FlashInfer, CUDA Graphs, torch.compile, speculative decoding, and quantization.
- Optimize distributed inference using tensor, data, and expert parallelism, NCCL, NVLink/NVSwitch, and InfiniBand.
- Work closely with researchers to bring new dense and MoE model architectures into production quickly and efficiently.
Requirements
- Have deep hands-on experience with LLM inference, ideally working inside vLLM, SGLang, TensorRT-LLM, or similar systems, not just deploying them.
- Understand TTFT, inter-token latency, throughput, continuous batching, PagedAttention, KV caching, and prefill vs. decode performance.
- Are comfortable profiling GPUs and reasoning about compute, memory bandwidth, kernel launches, synchronization, and communication bottlenecks.
- Have experience with CUDA, Triton, PyTorch, NCCL, and modern NVIDIA GPU architectures.
- Understand Transformer internals including MHA/GQA, ROPE, KV cache, quantization, and MoE.
- Have experience optimizing distributed, performance-critical systems in production.
- Care deeply about performance, simplicity, and reliability, and take ownership from profiling through production deployment.
We offer
- You'll help build one of the most technically advanced voice AI platforms in the world, and push LLM inference performance at every layer of the stack.
- You'll work directly with a world-class team of engineers and researchers on hard, measurable problems spanning models, GPU kernels, distributed systems, and production infrastructure.
- You'll have a voice in how our technology evolves, how our company grows, and how AI transforms human communication.