Staff Infrastructure Engineer - Data Streaming

Job description

Accountabilities: Lead the architecture, deployment, and operation of distributed data services, including self-hosted Kafka and Redis, across large-scale Kubernetes clusters and multi-cloud environments. Build highly automated, self-service infrastructure that enables services to operate consistently across AWS, GCP, and air-gapped on-premises environments. Manage data infrastructure supporting more than 5 PB of daily ingestion, optimizing for high throughput, low latency, reliability, scalability, and cost efficiency. Consolidate and optimize multi-tenant Kafka environments to improve resilience, operational efficiency, and infrastructure economics. Drive lifecycle automation for Kafka and Redis using Infrastructure as Code and GitOps practices, including Terraform and ArgoCD, to reduce operational toil and pager fatigue. Establish and enforce platform standards for observability, high availability, backup, disaster recovery, and lifecycle management of stateful Kubernetes workloads. Own the end-to-end platform experience for mission-critical open-source data systems serving hundreds of internal product teams. Partner with FinOps and engineering stakeholders to continuously identify opportunities to improve performance, reduce costs, and streamline operations. Collaborate with globally distributed engineering teams, particularly colleagues in Europe and India, with a preference for candidates able to work effectively within U.S. Eastern Time Zone collaboration windows. Work with modern orchestration and delivery technologies including Kubernetes, EKS, GKE, Jenkins, GitHub Actions, ArgoCD, and Terraform. Provide technical leadership in designing reliable, scalable infrastructure and help establish engineering standards and best practices across the data platform. Requirements 8+ years of experience in infrastructure, platform, or systems engineering, with a demonstrated history of operating stateful distributed systems at significant scale. Deep hands-on expertise with self-hosted Kafka and Redis in Kubernetes environments, including performance tuning, scaling, partitioning, persistence, and operator-based lifecycle management. Strong understanding of Kubernetes internals and production best practices for both stateless and stateful workloads. Experience delivering Database-as-a-Service, Messaging-as-a-Service, Platform-as-a-Service, or comparable infrastructure capabilities to internal engineering teams or external customers. Experience working in multi-cloud environments, with strong expertise in at least one major cloud provider such as AWS, GCP, or Azure. Strong experience with Infrastructure as Code and GitOps methodologies, particularly technologies such as Terraform, ArgoCD, or Pulumi. Familiarity with modern deployment approaches including blue-green, canary, and rolling deployments. Strong scripting or software development capabilities using Python, Go, or a comparable language. Solid understanding of CI/CD pipelines and workflow automation, including technologies such as GitHub Actions or Argo Workflows. Strong problem-solving skills and the ability to make sound architectural and operational decisions in complex production environments. Excellent communication and collaboration skills, with the ability to work effectively across engineering, infrastructure, FinOps, and globally distributed teams. A continuous-learning mindset and willingness to explore emerging technologies, particularly developments in cloud infrastructure, distributed systems, automation, and AI-enabled engineering. U.S. citizenship is required due to Federal Government contract requirements. Depending on responsibilities and customer requirements, the role may require additional background checks or security clearance, including Secret Clearance. Benefits Base salary: $156,000–$215,000 USD annually, with the applicable range varying by candidate location. Equity: Restricted Stock Units (RSUs). Employee stock program: Employee Stock Purchase Plan (ESPP). Time off: Flexible paid time off, company holidays, and paid sick time. Family support: Gender-neutral parental leave and grandparent leave. Healthcare: Medical, dental, and vision coverage. Retirement: 401(k) plan with company match. Financial protection: Life and disability insurance. Flexible spending: Health and dependent care FSAs. Additional insurance: Voluntary hospital, accident, and critical illness coverage. Wellbeing support: Employee Assistance Program (EAP) and wellness resources. Legal support: Pre-paid legal services. Pet benefits: Nationwide pet insurance. Specialized health support: Cancer Care program and global business travel medical insurance. Remote-work support: Home office allowance and mobile phone reimbursement. Wellness: Wellness coaching and gym/wellness reimbursement. Family-building benefits: Fertility coverage plus adoption and surrogacy reimbursement. Work environment: An opportunity to contribute to large-scale, mission-critical infrastructure in a highly technical and collaborative environment. How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1 This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Staff Infrastructure Engineer - Data Streaming based in the United States. As a Staff Infrastructure Engineer, you will shape the infrastructure powering a highly scalable, data-intensive cybersecurity platform.You will design and operate distributed data services such as Kafka and Redis across Kubernetes clusters and multi-cloud environments.Your work will support petabyte-scale data ingestion and trillions of events while maintaining exceptional reliability, performance, and efficiency.You will help create a highly automated, self-service infrastructure capable of operating across public cloud and air-gapped environments.The role combines deep technical ownership with platform engineering, automation, observability, disaster recovery, and cost optimization.You will partner with engineering, FinOps, and globally distributed teams to improve the platform experience for hundreds of product teams.This is an opportunity to solve complex infrastructure challenges at exceptional scale while helping build resilient systems for mission-critical security workloads.