Staff Software Engineer (Octopai - Core Platform, Runtime Services)

Job description

Business Area: Engineering Seniority Level: Mid-Senior level Job Description: At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises. About Octopai Octopai is a fast-growing Data Intelligence company (Acquired by Cloudera) that empowers enterprises to understand, trust, and optimize their data. Our platform provides automated data lineage, cataloging, and discovery to accelerate analytics, improve governance, and drive business impact. As we expand from SaaS into large-scale enterprise and on-premise deployments, we are investing in strengthening our core platform infrastructure and runtime services — the foundational layer that powers Octopai’s ingestion, analysis, metadata processing, and operational execution. We are looking for a Staff Software Engineer to help build and evolve these critical platform components. Role Overview As a Staff Software Engineer (Core Platform, Runtime Services) , you will focus on designing and developing the foundational runtime and shared services that enable Octopai’s platform to operate at scale. This role is deeply technical, centered around building robust infrastructure primitives, runtime orchestration, platform-wide reliability, and performance improvements. You will work on the systems that sit at the heart of Octopai’s execution engine — supporting metadata harvesting, lineage computation, distributed processing, and enterprise workloads across SaaS and customer-managed deployments. As a Staff Software Engineer you will: Design and implement core runtime services that power Octopai’s platform execution, analysis workflows, and operational stability. Build shared infrastructure components used across the backend organization, including service frameworks, job execution layers, scheduling, and platform abstractions. Improve scalability, reliability, and performance of platform services supporting metadata ingestion and lineage computation. Develop platform capabilities for workload orchestration, lifecycle management, retries, fault handling, and resource optimization. Partner with DevOps and Cloud teams to ensure runtime services are deployable and manageable in Kubernetes-based and enterprise environments. Build foundations for observability, including logging, metrics, tracing, and troubleshooting tools for core services. Ensure runtime infrastructure supports enterprise-grade requirements such as isolation, security, compliance, and predictable operations. Contribute to architecture discussions and evolve Octopai’s core platform foundations through strong design and implementation practices. Support engineering quality through clean architecture, code reviews, and scalable backend patterns. We’re excited about you if you have: 8–12+ years of backend or platform engineering experience, with demonstrated impact at Staff level. Strong expertise in building scalable backend infrastructure, runtime systems, or distributed service platforms. Excellent programming skills in C# (.NET), Java and modern backend engineering practices. Experience designing shared services that support orchestration, execution pipelines, or platform-level abstractions. Hands-on experience with SQL and NoSQL databases ( SQL Server, PostgreSQL, ArangoDB , or similar). Familiarity with messaging, caching, and coordination systems ( Kafka, Redis , etc.). Experience operating containerized services and supporting Kubernetes-based deployments. Strong understanding of production reliability, fault tolerance, observability, and operational best practices. Ability to work across teams and collaborate effectively with engineering, infrastructure, and product stakeholders. You may also have: Experience with workflow engines, distributed job execution systems, or large-scale runtime platforms. Background supporting hybrid SaaS + enterprise/on-prem deployments. Familiarity with performance optimization for metadata-heavy or graph-based workloads. Knowledge of search/index infrastructure (ElasticSearch/Solr) or ingestion pipelines. Exposure to enterprise compliance, security constraints, and deployment complexity. This role is not eligible for immigration sponsorship What you can expect from us: Generous PTO Policy Support work life balance with Unplugged Days Flexible WFH Policy Mental & Physical Wellness programs Phone and Internet Reimbursement program Access to Continued Career Development Comprehensive Benefits and Competitive Packages Paid Volunteer Time Employee Resource Groups EEO/VEVRAA #LI-AC1 #LI-HYBRID