Principal Architect - Reinforcement Learning (Physical AI)

Job description

Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it. Help Build the Future of Caterpillar – At Caterpillar, technology always has a purpose, which is to solve our customers’ toughest challenges. Through Cat Technology, we are solving problems by building the intelligence layer that connects machines, data, and people to make jobsites safer, more productive, and more sustainable. By combining deep domain expertise in physical systems with software, connectivity, autonomy, and AI, we deliver solutions that work in the real world—on real jobsites, at global scale. You’ll build and deploy against one of the most unique data foundations—over 1.6 million connected assets generating real-world data daily. These data and platform capabilities are enabling the development of AI models, edge computing architectures, and software systems that scale across fleets, products, and industries. The result will be a new generation of machines that continuously learn, improve, and deliver performance at scale. Be Part of What’s Next in Autonomous Construction Sites Construction autonomy is one of the most complex challenges in applied AI, and at Caterpillar, advancements in physical AI, simulation, sensing, and edge computing are turning things that once felt impossible—intelligent machines operating in dynamic jobsites—into reality. Our connected ecosystem brings together massive volumes of high-quality data to create a foundation where engineers like you can build and deploy against. If this work motivates you, we invite you to join our team. In these roles, you’ll work at the intersection of the physical and digital worlds. You’ll help design and deliver intelligent systems that enable machines to perceive their environment, make informed decisions, and support safer, more productive operations. Apply today to build the new era of construction autonomy at Caterpillar. Job Summary: The Learning Systems Architect owns how the machine learns — the end-to-end system that turns simulation, digital twins, and fleet data into high-performance, field-proven policies and models. The Principal Architect – Reinforcement Learning (RL) Systems is responsible for the architecture of the learning system that produces Caterpillar’s Physical AI models: the reinforcement learning, imitation learning, and foundation-model post-training pipelines that train perception, prediction, planning, and control policies for excavators, dozers, loaders, and haul trucks. Where the Simulation Engine architect owns the runtime the policy trains in, the Digital Twin architect owns the fidelity of what it trains against, and the AI Factory architect owns the compute and data platform it trains on, this role owns the learning problem itself: how tasks are formulated as observations, actions, rewards, and curricula; which policy and model architectures are used; how large-scale closed-loop training in simulation is structured; how sim-to-real transfer is achieved and measured; and how learned policies are evaluated, made safe, and deployed onto machines. The architect defines the reference architecture for learning-based autonomy at Caterpillar: Gym-style task and environment specifications on top of Isaac Lab / Isaac Sim, Omniverse, and hybrid neural simulation (NVIDIA Cosmos-class world models); massively parallel GPU-based RL (PPO, SAC, GRPO and successors) and imitation learning (behavior cloning, distillation, offline RL) at fleet-data scale; vision-language-action and foundation-model post-training with RL; domain randomization, curriculum, and system-identification strategies for sim-to-real transfer; reward and metric design tied to productivity, safety, machine health, and operator-quality outcomes; and the policy evaluation, regression, and safety-case methodology that gates deployment. The role sets standards for reproducibility, experiment rigor, and benchmark hygiene, and partners with the Experimentation & Operations architect to run training and evaluation at scale. This role that turns research into repeatable engineering, and is measured by field task performance of deployed policies — not by simulation benchmarks alone. What You Will Do: Developing detailed architecture deliverables to solve business problems. Designing an application's technical infrastructure, such as specific databases, programming languages, utilities, and testing approaches. Leading the evaluation and deployment of new technologies to add or enhance existing digital technical capabilities. Participating in addressing business requirements for applications and collaborating with cross-functional teams to deliver digital solutions that meets business results. What You Will Have: Requirements Analysis: Knowledge of tools, methods, and techniques of requirement analysis; ability to elicit, analyze and record required business functionality and non-functionality requirements to ensure the success of a system or software development project. Analytical Thinking: Knowledge of techniques and tools that promote effective analysis; ability to determine the root cause of organizational problems and create alternative solutions that resolve these problems. Platform Architecture: Knowledge of technologies and methods to design processing mechanisms and roadmaps to execute business application systems; ability to design these roadmaps and deploy supportive interfaces for end-users to access related systems, in accordance with standards and processes. Target Architecture: Knowledge of target architecture; ability to develop the IT blueprint and roadmap while aligning the architecture and processes with business strategies and objectives. Effective Communications: Understanding of effective communication concepts, tools and techniques; ability to effectively transmit, receive, and accurately interpret ideas, information, and needs through the application of appropriate communication behaviors. Top Candidates Will Have: Reinforcement Learning for Physical Systems: Typically 8+ years (or PhD plus 5+) in robot learning with RL taken from algorithm to deployment on real hardware: PPO, SAC, GRPO, model-based / world-model RL, reward shaping, curriculum, and compounding-error handling in closed-loop training. Imitation Learning and Foundation-Model Post-Training – Behavior cloning, offline RL (IQL, GAIL), distillation, diffusion policies; vision-language-action or robot foundation models with RL-based post-training (RLHF / DPO / GRPO). Large-Scale Closed-Loop Training in Simulation – Architected RL frameworks on GPU-parallel simulators (NVIDIA Isaac Lab / Isaac Sim, MuJoCo, Brax) with Ray, Kubernetes, or Slurm; used neural / hybrid simulation (NVIDIA Cosmos, neural reconstruction) for training or augmentation. Sim-to-Real Transfer – Closed sim-to-real gaps via domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction, measured by field task success; hydraulic or heavy-machinery dynamics highly valuable. Policy Evaluation and Safety – Metrics for learned policies (safety, progress, productivity, comfort), statistically rigorous policy comparison, continuous evaluation and regression, adversarial / long-tail scenario testing, runtime safeguards; safety-case thinking (ISO 26262 / SOTIF / ISO 13849). Controls and Robotics Foundations – MPC, impedance control, trajectory optimization; judgment on learned versus classical approaches; perception-to-action pipelines under real-time constraints. Data Flywheel Literacy – Shapes demonstration and fleet data collection, auto-labeling, and synthetic data strategy for imitation and offline RL. Software Engineering – Strong Python and PyTorch (JAX a plus), working C++ for real-time integration, CUDA familiarity, experiment tracking, Bazel or monorepo tooling, CI/CD. Architecture Leadership – Turns research into repeatable engineering; sets reproducibility and benchmark standards; mentors ML engineers and researchers; led a shipped learning-based capability. Domain Exposure – Off-highway, construction, or mining autonomy (Bedrock Robotics, Built Robotics, Applied Intuition), or robot learning at NVIDIA (Isaac / GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus. Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or related field preferred; research or publication record a plus. Additional Details: This position requires the candidate to work full-time at the Irving, TX office (Dallas) Domestic relocation assistance is available for this position. Visa sponsorship is available with this position. Why Caterpillar You'll work on some of the most challenging applications of AI in the physical world, helping build systems that transform vast amounts of machine and sensor data into the intelligence that powers autonomous and AI-enabled equipment. If you're excited about applying AI to real-world machines operating in complex environments, we'd like to hear from you. Summary Pay Range: $147,760.00 - $240,110.00 Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar. Benefits: Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits. Medical, dental, and vision benefits* Paid time off plan (Vacation, Holidays, Volunteer, etc.)* 401(k) savings plans* Health Savings Account (HSA)* Flexible Spending Accounts (FSAs)* Health Lifestyle Programs* Employee Assistance Program* Voluntary Benefits and Employee Discounts* Career Development* Incentive bonus* Disability benefits Life Insurance Parental leave Adoption benefits Tuition Reimbursement * These benefits also apply to part-time employees This position requires working onsite five days a week. Relocation is available for this position. Visa sponsorship is available for eligible applicants. Posting Dates: Any offer of employment is conditioned upon the successful completion of a drug screen. Caterpillar is an Equal Opportunity Employer, Including Veterans and Individuals with Disabilities. Qualified applicants of any age are encouraged to apply. Not ready to apply? Join our Talent Community .