Senior Data Software Engineer

Job description

### What your day could consist of: - **Architect and Build Pipelines** by designing, developing, and maintaining high-throughput data pipelines that seamlessly transform data across our ecosystem - **Execute Data Modeling and Schema Design**, utilizing DDL and DML operations to optimize data structures for both performance and long-term scalability - **Champion Engineering Excellence** by implementing automated testing, CI/CD pipelines, and Infrastructure as Code (Terraform) to ensure system health and reliability. - **Collaborate Cross-Functionally** with Data Science and AI teams to engineer feature stores and automated MLOps pipelines, moving cutting-edge research into live production environments - **Drive System Monitoring** and health checks, ensuring our containerized and orchestrated environments (Kubernetes, Docker, Airflow) run at peak efficiency. - **Act as a Technical Advisor** to stakeholders, translating complex data architectures into clear, non-technical conversations that guide business decisions ### Technical environment: - Databases: MySQL, Amazon Aurora, Amazon RDS, PostgreSQL - ata Storage and Analytics: AWS S3, Apache Iceberg, Snowflake, Apache Druid, Looker - Data Processing and Transformation: Apache Spark, DBT - Languages: Python (primary) - Infrastructure as Code: Terraform - Containerization and Orchestration: Kubernetes, Docker, Airflow ### The Ideal Candidate Will Bring: - **5+ years of professional software engineering experience**, with a proven track record of building and maintaining production-grade data systems. - **Deep Data Architecture expertise**, with the ability to design modern data stack components and schemas that balance immediate performance with future growth. - **Proficiency in Infrastructure as Code**, specifically using **Terraform** to manage and deploy your own infrastructure. - **A high level of software rigor**, including a strong background in automated testing, CI/CD, and general software engineering best practices - **Expert-level SQL skills** (particularly MySQL), with a focus on optimizing complex queries for maximum performance. - **Experience in a modern technical environment**, ideally having worked with Python, AWS (S3, Aurora, RDS), Snowflake, Apache Spark, and DBT - **Exceptional communication skills**, with the talent to distill complex technical concepts into clear, actionable insights for both technical and non-technical peers