Senior Staff Engineer, Data

Job description

As a Senior Staff Engineer, Data, you'll be a key technical leader driving the design, implementation, and evolution of FloQast's core data platform. You will define the standards and patterns that power data ingestion, governance, storage, processing, and access across all product and analytics systems — with Apache Spark as the primary compute engine at the heart of that stack. Your work will enable teams across engineering, product, and business operations to build on a reliable, scalable, and secure data foundation. You've spent years going deep on Spark in production. You reason through shuffle behavior, partition strategies, and memory pressure without reaching for documentation. You understand what the Catalyst optimizer does with your query plan and you write code that helps it. You've made the call between PySpark and Scala Spark on real workloads, run Structured Streaming pipelines over Kafka topics on MSK, and debugged slow stages in the Spark UI. You've built Spark jobs that read and write Apache Iceberg tables at scale — managing snapshot isolation, schema evolution, and compaction as operational concerns, not afterthoughts. At this level, you don't just tune pipelines. You set the architecture that determines whether the next order of magnitude is a rewrite or a config change. At FloQast, you'll apply that depth across our full lakehouse stack. FloLake runs on Iceberg over S3, orchestrated through MWAA, cataloged in AWS Glue, and queried via Trino and Athena. Kafka on MSK moves data through the hot path. You'll own the compute architecture decisions that 30,000+ tenants depend on, define how Spark jobs are structured and governed across the platform, and set the bar for how the team thinks about distributed systems. Engineers at all levels will look to you as the authority on what good looks like — and you'll build the systems that prove it. ### What You'll Do - Data Platform Architecture: Lead the design of scalable and reliable data pipelines, storage solutions, and access layers using modern cloud-native technologies. - Foundational Systems: Own and evolve the core data platform components (e.g., event streaming, lakehouse architecture, batch & streaming ETL, data cataloging). - Data Governance: Define best practices for data quality, lineage, privacy, and access control to ensure regulatory compliance and trust in the data. - Cross-functional Collaboration: Work closely with Product, Analytics, Infrastructure, and Security teams to align data platform capabilities with organizational goals. - Technical Leadership: Mentor engineers across the organization, establish coding and architectural standards, and influence the strategic direction of the platform. - Innovation & Modernization: Evaluate and integrate emerging technologies that improve performance, observability, developer experience, and cost efficiency. ### What You'll Bring - 10+ years of software engineering experience with deep expertise in data infrastructure, distributed systems, or backend platform engineering. - Proven track record of designing and delivering production-grade data platforms at scale (e.g., supporting hundreds of TBs of data, thousands of jobs/day). - Strong experience with tools like Snowflake, dbt, Kafka, Airflow, Spark, and cloud-native technologies (e.g., AWS, GCP, or Azure). - Hands-on experience building APIs and services for data access, metadata management, and platform observability. - Familiarity with data privacy regulations (GDPR, SOC2, HIPAA) and enterprise-grade security practices. - Effective communicator and collaborator who can influence engineering and non-engineering audiences alike. - Experience in a startup or high-growth SaaS environment. - Exposure to AI/ML data pipelines or real-time analytics platforms.