Software Engineer, Large-Scale Data Query Systems

Job description

ABOUT EVENTUAL Every breakthrough Physical AI system — humanoid robots, autonomous vehicles, video generation models — is trained on petabytes of video, lidar, radar, and sensor data. But today's data platforms (Databricks, Snowflake) were built for spreadsheet-like analytics, not the multimodal corpora that power AI. Robotics and video-AI teams now lose 20-40% of their training time to dataloading alone. GPU bandwidth has grown 2-3× per generation. Storage and pipelines haven't. The gap widens every year. Eventual was founded in 2022 to close it. Our open-source engine, Daft https://daft.ai/, is the distributed data engine purpose-built for multimodal AI — already running 2 PB/day at Amazon, 60-100 PB at another FAANG company, and in production at Mobileye, TogetherAI, and CloudKitchens. We are building a video-native index on top of our engine for Physical AI that streams curated datasets to GPUs at line rate. Saturates B200s today. Aimed at NVL72 and Vera Rubin tomorrow. We're building this in partnership with the top PhysicalAI labs and public AI infrastructure companies today. We have raised $30M from Felicis, CRV, Microsoft M12, Citi, Essence, Y Combinator, Caffeinated Capital, Array.vc http://Array.vc, and angels from the co-founders of Databricks and Perplexity. We've assembled a world-class team from AWS, Render, Pinecone and Tesla. We have spent our careers powering the last generation of PhysicalAI in self-driving, and are excited to now do this for the next. Join our small (but powerful!) team working together 4 days/week in our SF Mission district office. YOUR ROLE: As a Software Engineer on the Systems team, you will build key capabilities for the Daft distributed data engine. You will be working on core architectural design and implementation of various components in Daft. While we are an experienced team that can provide constant guidance and mentorship, we value engineers who can autonomously scope and solve difficult technical challenges. KEY RESPONSIBILITIES: - Planning/Query Optimizer: intelligently optimize users’ workloads with modern database techniques - Execution Engine: improve memory stability through the use of streaming computation and more efficient data structures - Distributed Scheduler: improve Daft’s resource utilization, task scheduling and fault tolerance - Storage: improve Daft integrations with modern data lake technologies such as Apache Parquet, Apache Iceberg and Delta Lake - Our goal is to build the world’s best open-source distributed query engine, becoming the leading framework for data engineering and analytics. - We are a young startup - so be prepared to wear many hats such as tinkering with infrastructure, talking to customers and participating heavily in the core design process of our product! WHAT WE LOOK FOR: - We are looking for a candidate with a strong foundation in systems programming and ideally experience with building distributed data systems or databases (e.g. Hadoop, Spark, Dask, Ray, BigQuery, PostgreSQL etc) - 3+ years of experience working with distributed data systems (query planning, optimizations, workload pipelining, scheduling, networking, fault tolerance etc) - Strong fundamentals in systems programming (e.g. C++, Rust, C) and Linux - Familiarity and experience with cloud technologies (e.g. AWS S3 etc) - Most importantly, we are looking for someone who works well in small, focused teams with fast iterations and lots of autonomy. If you are passionate, intellectually curious and excited to build the next generation of distributed data technologies, we want you on the team! PERKS & BENEFITS - In-person tight knit team with 4x a week in office - Competitive comp and startup equity - Catered lunches and dinners for SF employees - Commuter benefit - Team building events & poker nights - Health, vision, and dental coverage - Flexible PTO - Latest Apple equipment - 401k plan with match!