Lead AI Platform Engineer (Python, Cloud & GenAI)

Job description

TransUnion's Job Applicant Privacy Notice Team Overview Our team is at the forefront of building next-generation AI-powered platforms and cloud-native solutions that enable the organization to accelerate innovation at scale. We develop reusable engineering capabilities, intelligent services, and enterprise-grade platforms that empower product teams, data scientists, and engineers to build, deploy, and operate secure, scalable, and reliable applications. We work across the full technology stack, combining modern software engineering, cloud infrastructure, data platforms, and Generative AI to solve complex business challenges. Our focus areas include Python engineering, cloud-native architectures on Google Cloud Platform, AI/ML platforms, Retrieval-Augmented Generation (RAG), agentic AI systems, platform reliability, and developer productivity. The team values technical excellence, innovation, ownership, and continuous learning. Engineers are encouraged to experiment with emerging technologies, influence architectural direction, contribute to platform strategy, and collaborate closely with product, security, data science, and business stakeholders. We foster a culture where strong engineering fundamentals, operational excellence, and practical AI innovation come together to deliver meaningful business impact. As part of this team, you will have the opportunity to work on cutting-edge technologies, solve large-scale engineering problems, and help shape the future of intelligent systems across the enterprise. You'll collaborate with highly skilled engineers, architects, and AI practitioners while helping establish engineering standards, reusable frameworks, and best practices that drive success across multiple teams. This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week. Role Overview And Core Responsibilities Role Overview: We are seeking an experienced and highly skilled Lead Engineer to design, build, and operate production-grade systems that span software engineering, cloud-native infrastructure, and AI-powered applications. The ideal candidate combines deep Python engineering expertise with strong cloud architecture skills on Google Cloud Platform (GCP) and proven experience building and deploying Generative AI at enterprise scale. This role will define reusable engineering patterns and platform capabilities that enable multiple product, data science, and engineering teams to safely build, deploy, evaluate, and operate intelligent systems. The Lead Engineer will remain deeply technical while providing architecture leadership, mentoring engineers, and influencing engineering standards across the organization. Core Responsibilities: Production Python Engineering: Design and develop high-performance, production-grade Python services and applications following modern software engineering best practices. Establish coding standards, design patterns, and reusable frameworks that enable teams to ship reliable software at velocity. API Design & Microservices Architecture: Architect and build scalable microservices and RESTful/GraphQL APIs using Python frameworks such as FastAPI and Flask. Define service boundaries, communication patterns, and data contracts that support distributed system reliability and independent team ownership. System Design & Distributed Systems: Lead the design of distributed, event-driven, and asynchronous systems that handle high throughput and low-latency requirements. Apply principles of concurrency, fault tolerance, and horizontal scaling to deliver resilient production architectures. Testing Strategy & Engineering Quality: Define and drive comprehensive testing strategies including unit, integration, contract, and end-to-end testing using pytest and related tooling. Champion code review practices, static analysis, and automated quality gates to maintain high engineering standards. Reusable Libraries & SDKs: Identify repeated engineering patterns across teams and convert them into standardized Python libraries, SDKs, templates, and self-service platform capabilities that accelerate development and reduce duplication. Cloud-Native Architecture (GCP): Design and implement cloud-native architectures on Google Cloud Platform using services such as GKE, Cloud Run, Cloud Functions, and App Engine. Establish patterns for containerization, orchestration, autoscaling, and resilient infrastructure that support production AI and software workloads. Data Infrastructure & Pipelines: Architect scalable data infrastructure using BigQuery, Pub/Sub, Cloud Storage, Dataflow, and Cloud Composer. Design data pipelines that support both real-time streaming and batch processing for analytics, ML training, and AI application data needs. Infrastructure as Code & DevOps: Establish infrastructure-as-code practices using Terraform and CI/CD pipelines with Cloud Build or equivalent tooling. Drive automation of build, test, deployment, and rollback workflows to ensure repeatable, auditable, and fast delivery cycles. Observability & Site Reliability: Implement end-to-end observability using Cloud Monitoring, Cloud Logging, Cloud Trace, and custom metrics. Define SLOs, SLIs, and error budgets for production services and establish incident response practices that minimize mean time to recovery. Cloud Security & Cost Optimization: Establish security best practices including IAM policies, VPC design, secrets management, and workload identity federation. Drive cloud cost optimization through resource right-sizing, committed use discounts, and efficient architecture patterns. LLM Integration & GenAI Applications: Design and build production applications powered by large language models, integrating commercial and open-source models through well-defined abstraction layers. Establish patterns for prompt engineering, context management, structured outputs, and token optimization. RAG & Knowledge Architecture: Design scalable Retrieval-Augmented Generation pipelines using embeddings, vector databases, semantic and hybrid retrieval, reranking, and metadata-enriched knowledge stores. Implement GraphRAG and semantic models where appropriate to improve retrieval quality. AI Evaluation & Quality: Define engineering standards for evaluating LLM applications and AI agents. Establish golden datasets, automated regression testing, retrieval evaluation, groundedness measurements, task-success metrics, and structured-output validation frameworks. Model Serving & Inference: Design model integration and inference architectures on Vertex AI and equivalent platforms supporting commercial, private, and open-source models. Establish patterns for model selection, routing, fallback, versioning, latency optimization, and cost management. Agentic AI & Orchestration: Define patterns for production agentic systems including tool calling, workflow orchestration, state and memory management, planning and execution, human-in-the-loop controls, retries, recovery, and long-running workflows. Technical Leadership: Lead architecture and design reviews, resolve complex technical problems, establish engineering standards, and make pragmatic trade-offs across quality, speed, scalability, security, operational complexity, and cost. Mentoring & Engineering Excellence: Mentor junior and mid-level engineers, review technical designs and code, raise engineering quality across the team, and help engineers adopt modern Python, cloud, and AI engineering practices. Cross-Team Collaboration: Partner with data scientists, platform teams, security, architecture, product management, and business stakeholders to translate enterprise requirements into scalable, production-ready solutions. Technical Communication: Communicate technical strategy, architecture decisions, risks, trade-offs, and capabilities to engineering leaders, architects, product teams, and senior stakeholders with clarity and precision. Required Knowledge And Experiences Qualifications: Education: Bachelor's or Master's degree in Computer Science, Engineering or a related technical field, or equivalent practical experience. Experience: Typically 7+ years of software, platform, cloud, data, or AI engineering experience, including significant hands-on ownership of production systems and technical architecture. Demonstrated experience building production cloud-native applications on GCP and working with Generative AI or ML systems is required. Experience may span traditional software/platform engineering and newer AI engineering; candidates are not expected to have spent an unrealistic number of years specifically working with Generative AI. Technical Leadership: Demonstrated ability to independently design complex systems, lead technical execution across multiple engineers or teams, influence architecture, and convert emerging technologies into reliable enterprise capabilities. Technical Skills: Expert-level proficiency in Python and strong production software engineering fundamentals. Experience with FastAPI, Flask, asyncio, pytest, and modern Python tooling. Familiarity with Java, Go, TypeScript, or another production language is advantageous. Strong knowledge of data structures, algorithms, API design, distributed systems, concurrency, asynchronous programming, software architecture, automated testing, debugging, Git, and engineering best practices. Deep hands-on experience with Google Cloud Platform (GCP) including GKE, Cloud Run, BigQuery, Pub/Sub, Cloud Storage, Dataflow, Vertex AI, Cloud Build, IAM, and Cloud Monitoring. Experience with Terraform, Docker, Kubernetes, and CI/CD pipelines. Strong understanding of cloud-native architecture patterns including containerization, serverless, event-driven design, service mesh, autoscaling, and infrastructure as code. Experience designing for high availability, disaster recovery, and cost efficiency. Strong working knowledge of deep learning and Transformer architectures, including tokenization, embeddings, attention, inference, and modern foundation-model architectures. Experience with PyTorch, Hugging Face, LangChain, LlamaIndex, or comparable modern ML/AI frameworks. Deep hands-on experience designing and building production applications using LLMs, Generative AI, RAG pipelines, vector databases, and agentic AI patterns. Experience with observability and monitoring tools such as Prometheus, Grafana, OpenTelemetry, Cloud Monitoring, and Cloud Logging for production systems. TransUnion Overview: At TransUnion, we encourage and are committed to creating a real, positive impact and shared sense of purpose within our Workforce for Good , which empowers our people to grow, innovate and contribute to a better future for our communities and customers. We strive to build an environment where our associates are in the driver’s seat of their professional development— while having access to help along the way. We recognize that success comes when our associates thrive both professionally and personally; that’s why we prioritize work/life flexibility and offer resources for our teams across the globe to collaborate and drive excellence. Be a part of our Workforce for Good – you’ll work with great people, pioneering products and cutting-edge technology. TransUnion Job Title Sr Developer, Software Development