AI Engineer – System Z Infrastructure

Introduction IBM Infrastructure is looking for an entry-level AI Engineer to join our System Z mainframe team. If you're interested in bringing modern AI capabilities to enterprise systems that process billions of daily transactions for major banks, healthcare providers, and government agencies worldwide, this could be the role for you. You'll work with Spyre hardware accelerator cards, specialised hardware we've built to run Large Language Models efficiently on System Z architecture. This is a unique chance to work across the full technology stack, from low-level hardware interfaces up to application APIs. Most teams don't get to touch this many layers. As an entry-level engineer, you'll work alongside senior engineers and architects who know System Z inside and out. You'll gain hands-on experience with performance optimisation, debugging complex systems, and building the monitoring infrastructure that keeps enterprise AI running reliably. This isn't just about deploying models; it's about making them work at enterprise scale on one of the most demanding platforms in computing. We're looking for someone who's genuinely curious about how things work under the hood and wants to understand systems at multiple levels. Recent grads with strong fundamentals and a willingness to learn are encouraged to apply. Your Role And Responsibilities LLM Integration and Deployment on System Z You'll help develop, profile and integrate Large Language Models on System Z using Spyre acceleration cards. This means working at multiple stack levels; from firmware interfaces to application-level APIs. You'll collaborate with senior engineers to determine the best approaches for model loading, inference optimisation, and resource allocation specific to our architecture. The goal is to make LLMs accessible to both traditional mainframe applications and newer cloud-native workloads. Performance Profiling and Optimisation Profile LLM workloads running on Spyre cards to find bottlenecks. You'll use profiling tools to measure inference latency, throughput, memory usage, and power consumption. Then work with that performance data to identify where we can optimise, whether that's hardware utilisation, kernel efficiency, memory management, or batch processing. Document what you find and help build best practices guides. Failure Analysis and Debugging Investigate problems when things go wrong; anything from inference errors to system-level failures. Work across the stack (firmware, drivers, runtime, applications) to figure out root causes. Partner with hardware engineers, firmware developers, and system architects on the trickier issues. Keep detailed records of failure patterns and resolutions, and help build automated tests to catch regressions. Observability and Telemetry Infrastructure Build monitoring solutions so we can see what's happening with LLM workloads in production. Deploy monitoring agents, add instrumentation points, and set up logging to capture metrics like model performance, hardware utilisation, error rates, and system health. Create dashboards and alerts that help operations teams spot issues quickly. Work with time-series databases and visualisation tools to enable both real-time monitoring and historical analysis. Cross-Stack Collaboration Work with teams across hardware engineering, firmware, system software, AI research, and application development. Participate in architecture reviews and design discussions. Write technical documentation and contribute to knowledge sharing within the team. Learning and Growth Keep up with new developments in LLMs, hardware acceleration, and AI deployment practices. Try out new optimisation techniques and tools. Contribute ideas for making our LLM infrastructure faster and more reliable. Take advantage of training programs and mentorship to build your expertise in both mainframe systems and AI engineering. Preferred Education Bachelor's Degree Required Technical And Professional Expertise - Strong Python programming skills with some experience in C/C++ and Linux-based development environments. - Understanding of machine learning fundamentals, particularly neural networks and transformer-based models, alongside an interest in AI inference and performance optimisation. - Basic knowledge of computer architecture, including memory hierarchies, parallel processing and I/O systems, with an interest in profiling and performance analysis. - Strong problem-solving, debugging and analytical skills, with good technical communication and documentation abilities. - A collaborative mindset, curiosity about complex systems and enthusiasm for learning from experienced engineers in AI profiling and hardware acceleration. Preferred Technical And Professional Experience - Exposure to PyTorch, TensorFlow, Hugging Face Transformers or other machine learning frameworks. - Familiarity with profiling tools, performance monitoring, GPU programming, CUDA or AI model optimisation techniques. - Understanding of Git, CI/CD, Docker or Kubernetes, with exposure to enterprise computing, IBM Z or z/OS considered an advantage.