Associate AI Inference Engineer (m/f/d/x)

About the position

T-Systems is building Europe's next generation of AI infrastructure with the AI Industry Cloud powered by thousands of NVIDIA GPUs, including the latest generation of NVIDIA DGX B200 and B300 systems. Our platform enables companies to deploy, scale, and operate cutting-edge large language models (LLMs) and generative AI applications on a secure and high-performance infrastructure.

We are looking for an Associate AI Inference Engineer to join our AI platform team. In this role, you will support the provision, operation and optimization of productive LLM inference services based on the T-Systems AI Foundation platform, which is operated on the Industrial AI Cloud. Working closely with platform engineers, AI researchers and software developers, you will help provide high-performance and reliable AI services for enterprise customers.

This position is particularly suitable for young professionals with practical experience in LLM Inference who want to work on a large scale with modern NVIDIA GPU infrastructure and productive AI systems.

tasks

• Provision, configuration and operation of productive LLM inference services based on the T-Systems AI Foundation platform.

• Optimize inference workloads on NVIDIA GPU infrastructure for latency, throughput, and GPU utilization.

• Benchmarking and evaluation of LLM serving frameworks, inference configurations, and model deployments.

• Implementation and optimization of modern LLM serving techniques, such as kV cache optimization, continuous batching, and distributed inference.

• Working with platform engineers to deploy and scale AI workloads on Kubernetes

• Analyze and fix performance issues in inference operations across software, network, and GPU infrastructure.

• Involvement in automating deployment, benchmarking, and operational processes.

• Monitor inference services using metrics, logging, and dashboards to ensure reliable production operations.

• Working closely with AI engineers and researchers to onboard new foundation models and inference technologies.

• Maintaining technical documentation and participating in the continuous development of the AI inference platform.

Must-Have Skills

• Bachelor's degree in computer science, electrical engineering, information technology or a comparable technical field. A master's degree in artificial intelligence, machine learning, or a related field is an advantage.

• Relevant practical experience in AI engineering, machine learning engineering, or AI infrastructure engineering, including experience as a working student or research engineer.

• Very good programming skills in Python as well as experience in developing AI applications and corresponding tools.

• Practical experience deploying, benchmarking, or optimizing large language model (LLM) inference workloads.

• Practical experience with modern LLM serving frameworks such as vLLM and sGLang.

• Practical experience optimizing LLM inference workloads on NVIDIA GPU infrastructure and points of contact with NVIDIA DGX/HGX systems or next-generation GPU platforms such as B200 and B300.

• Experience with evaluating or optimizing advanced inference techniques such as KV caching, disaggregated prefill/decode inference, or comparable LLM serving optimizations.

• Knowledge of modern NVIDIA inference technologies such as NVIDIA Dynamo, LMCache, Tensorrt-LLM, Triton Inference Server, or comparable GPU inference frameworks.

• Experience benchmarking AI models and analyzing inference performance, latency, throughput, and serving efficiency.

• Experience with Docker, Kubernetes, Bash, and Linux-based compute environments.

• Strong analytical and problem-solving skills, as well as the ability to collaborate within multi-disciplinary engineering teams.

• Willingness to travel of up to 50%.

• International experience, for example through study, work or extended stays abroad.

Nice-to-have Skills

• Knowledge of CUDA, NCCL, NVLink, GPUDirect RDMA, or InfiniBand Networking.

• Experience running productive AI services on Kubernetes.

• Knowledge of observability tools such as Prometheus, Grafana, and OpenTelemetry.

• Experience with Helm, Terraform, GitHub Actions, or Infrastructure-as-Code.

• Experience with enterprise AI platforms or multi-tenant inference environments.

• Knowledge of retrieval-augmented generation (RAG), vector databases or embedded retrieval systems.

What we offer

• The opportunity to work on one of Europe's largest enterprise AI platforms.

• Work with cutting-edge NVIDIA AI infrastructure, including DGX B200 and B300 systems.

• Hands-on experience with advanced LLM inference technologies, including AI Foundation, vLLM, sGLang, Tensorrt-LLM, NVIDIA Dynamo, and lmCache.

• Work with large clusters of GPUs for enterprise AI workloads.

• Collaboration with experienced AI researchers, platform engineers and software developers.

• Opportunities for continuous education as well as professional and personal development.

• A collaborative work environment that promotes innovation, technical excellence and knowledge sharing

About T-Systems International GmbH

At T-Systems, we offer our business customers the right system solutions for their digital business. With our portfolio, we ensure that digital transformation reduces complexity, saves costs and makes everyday work easier. Our focus is connectivity, digital, cloud & infrastructure as well as security - Let's power higher performance!

Is this job made for you? Then take advantage of this opportunity! Please apply online. Severely disabled people are given priority if they are equally qualified. We are looking forward to seeing you!