Staff Software Engineer, Replication Foundations

Job description

# **Role Summary** We’re hiring a Staff Software Engineer to join the Replication Foundations team within Temporal’s Cloud Global Services (CGS) organization. Replication Foundations owns and evolves Temporal’s core replication stack in Temporal OSS—the distributed systems backbone behind key Temporal Cloud capabilities such as High Availability namespaces, cross-cluster and cross-region failover, and migration products that enable customers to move workloads between self-hosted Temporal and Temporal Cloud. The team also builds foundational scalability and reliability mechanisms that support Temporal at scale. In this role, you’ll help set the technical direction for Temporal’s distributed replication systems. You’ll lead complex, correctness-critical initiatives spanning architecture, design, implementation, rollout, and operations. You’ll work across teams to evolve reliable and scalable replication capabilities that support both the open source project and Temporal Cloud. # **What You'll Do** - Set the technical direction and evolve the architecture of Temporal’s OSS replication stack, from problem definition through rollout and operational support. - Lead the design and implementation of replication protocols that power: - High Availability namespaces - Cross-cluster and cross-region replication - Migration between Temporal clusters, including cloud-to-self-hosted and cloud-to-cloud scenarios - Drive scalability and reliability initiatives such as: - Multi-cell namespaces - Enabling a namespace to span multiple clusters - Improving load distribution and handling hot spots - Define and communicate system-level guarantees, including consistency models, ordering, idempotency, failure recovery, performance, and operational behavior. - Identify architectural risks and opportunities, and shape the technical roadmap for replication capabilities that support current and future cloud products. - Partner with Cloud Enablement, CGS, Product, and other engineering teams to align OSS replication foundations with customer and product needs. - Lead design reviews, raise the quality of implementation and testing practices, mentor engineers, and provide technical guidance across the organization. - Lead or contribute to debugging complex production issues, incident response, and follow-up improvements related to replication and core system behavior. # **What You'll Bring** - A track record of designing and delivering complex production distributed systems, including systems where correctness, availability, and performance are critical. - Deep understanding of distributed systems fundamentals such as replication, partitioning, consistency, fault tolerance, durability, concurrency, and failure recovery. - Experience defining system architecture, protocol behavior, invariants, and trade-offs in ambiguous or evolving problem spaces. - Experience debugging complex production issues, including concurrency bugs, data inconsistencies, partial failures, and performance bottlenecks. - Proficiency writing production-quality concurrent code in Go; experience with Java, C++, or similar systems languages is also welcome. - Strong written and verbal communication skills, including the ability to explain complex designs and trade-offs to both technical and cross-functional audiences. - Demonstrated ability to influence technical direction across teams, build alignment without direct authority, and mentor engineers. - A thoughtful and curious approach to understanding how systems behave under load, failure, and changing workload conditions. # **Nice to Have** - Experience designing or maintaining replication protocols or data-plane infrastructure. - Experience with multi-cluster or multi-region architectures, including active-active or active-passive systems. - Familiarity with database internals, log-based replication, or event-sourced systems. - Prior contributions to large open source projects or distributed systems infrastructure. Temporal Technologies is an Equal Opportunity Employer. Temporal Technologies does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need. We embrace and celebrate differences and diversity. Temporal is committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. If you need to request a reasonable accommodation, please let your Recruiter know so we can assist.