Sr SRE & Automation Engineer (Customer Facing)

Job description

About Bitdeer Technologies Group Bitdeer is a world-leading technology company for AI and Bitcoin mining infrastructure. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers and building AI computational infrastructure to support the AI revolution. Bitdeer handles complex processes involved in computing such as equipment procurement, transport logistics, data center design and construction, equipment management, and daily operations. Bitdeer also offers advanced cloud capabilities to customers with high demand for artificial intelligence. Headquartered in Singapore, Bitdeer has deployed data centers across multiple countries, including the United States, Norway, Bhutan, and Ethiopia. To learn more, visit Job Description NeoCloud is building an AI-operated GPU cloud — and because it is a customer-facing cloud service, reliability is the product. Tenants run mission-critical training, fine-tuning, and inference workloads on our GPU infrastructure and trust us with their SLAs. In this role you own the reliability of the customer-facing GPU cloud service end-to-end: from tenant onboarding and service provisioning, through workload execution, incident response, and post-incident recovery. You are the SRE who stands between raw infrastructure and the customer's experience — designing the observability, automation, and operational practices that make a 10,000-GPU cloud feel simple and dependable to the tenants who depend on it. What You'll Own End-to-end reliability of the customer-facing GPU cloud service — availability, job completion, provisioning latency, and tenant experience. Production Kubernetes clusters optimized for GPU workloads at scale (100–10,000 GPUs) as the runtime substrate for customer workloads. Nvidia GPU operator, device plugin, MIG configuration, GPU time-slicing, and multi-tenant GPU allocation policies. Topology-aware scheduling: GPU locality, NVLink domain awareness, network rail affinity — placing customer jobs on the right hardware. Customer & tenant lifecycle: onboarding, quota management, isolation enforcement (namespaces, network policies, RBAC, resource quotas, pod security), and offboarding/reclamation. Bare-Metal-as-a-Service (BMaaS): automated provisioning, tenant handoff, lifecycle, and reclamation. SLIs/SLOs/SLAs for the customer cloud service: cluster availability, job completion rates, provisioning latency, API availability. Incident management with customer communication: runbook automation, escalation, customer-facing status updates, and post-incident reviews. Monitoring & observability stack: Prometheus, Grafana, Alertmanager, PagerDuty — tenant-aware dashboards and alerting. GPU node failure handling: automated detection, drain/cordon/taint, and workload rescheduling — minimizing customer-visible impact. Infrastructure-as-code: Terraform providers/modules, Helm, and GitOps (ArgoCD/Flux) across GPU clusters. Customer-facing operational readiness: service documentation, tenant runbooks, capacity planning, and support tiering. Customer-Facing Ownership You are accountable for the customer's reliability experience — when a tenant's job fails or a node drops, you own the detection, remediation, and communication loop. Define and publish customer-facing SLAs/SLOs and drive error-budget-based prioritization between feature work and reliability. Partner with customer success / support to close the feedback loop between customer-reported issues and systemic improvements. Build self-service observability that lets customers answer their own questions — status, quota, job health — reducing support load. Feed the AIOps Substrate The remediation-actuator and workflow engine land here — you make the control plane safe for automated action. Your CRDs and runbooks are the schema the platform's predictors and remediators write against. Every human intervention you do this quarter becomes an autonomous workflow next quarter — turning customer-impacting incidents into self-healing events. What Success Looks Like in Year 1 Customer-facing GPU cloud service SLAs published and met — availability, job completion, provisioning latency. Automated drain/reschedule around predicted GPU faults, at scale, without customer-visible impact. BMaaS live for external tenants with self-service onboarding. MTTD and MTTR for customer-impacting incidents reduced through automation. Tenant self-service observability live — customers can see their own job health, quota, and status. Requirements 5+ years in SRE / cloud operations, with at least 2 years operating GPU workloads at scale. Deep understanding of Kubernetes operations and GPU workload management (Nvidia GPU operator, device plugin, MIG, time-slicing, GPU scheduling). Experience with topology-aware scheduling and GPU-specific resource management. Hands-on experience building multi-tenant cloud platforms with strong isolation guarantees. Customer-facing cloud service experience — defining and operating against customer SLAs/SLOs, handling tenant incidents and communications. Experience with bare-metal server provisioning and lifecycle automation (Ironic, MAAS, or custom). Proficiency in Terraform, Helm, and GitOps workflows (ArgoCD/Flux). Strong SRE background: SLI/SLO/SLA frameworks, error budgets, incident management, capacity planning. Experience with Prometheus, Grafana, and alerting at scale. Strong programming skills in Go or Python for automation / operator development. AIOps aptitude — you view the control plane as an execution surface for automated remediation, not just a scheduler. Runbook-as-code mindset — every SRE playbook you write should be executable by the platform. -------------------------------------------------------------------- Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws. Bitdeer does not discriminate against employees or applicants based on conditions such as race, color, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.