Data Engineer Lead - Databricks

Job description

🚀 **Join Our Data Products and Machine Learning Development Remote Startup!** 🚀 Muttdata is a dynamic startup committed to crafting innovative systems using cutting-edge Big Data and Machine Learning technologies. We’re looking for a **Data Engineer Lead** to help take our expertise to the next level. If you consider yourself a data nerd like us, we’d love to connect! 🐶🚀 ### 🚀 What We Do - Leveraging our expertise, we build modern Machine Learning systems for demand planning and budget forecasting. - Developing scalable data infrastructures, we enhance high-level decision-making, tailored to each client. - Offering comprehensive Data Engineering and custom AI solutions, we optimize cloud-based systems. - Using Generative AI, we help e-commerce platforms and retailers create higher-quality ads, faster. - Building deep learning models, we enhance visual recognition and automation for various industries, improving product categorization, quality control, and information retrieval. - Developing recommendation models, we personalize user experiences in e-commerce, streaming, and digital platforms, driving engagement and conversions. ### 🌟 Our Partnerships - Amazon Web Services - Astronomer - Databricks ### 🌟 Our Values - 📊 We are Data Nerds - 🤗 We are Open Team Players - 🚀 We Take Ownership - 🌟 We Have a Positive Mindset 🔍 Curious about what we’re up to? Check out [our](https://blog.muttdata.ai/tag/Case%20Studies/1) case studies and dive into our [blog post](https://blog.muttdata.ai/) to learn more about our culture and the exciting projects we’re working on! 🚀 ### Responsibilities 🤓 - Lead end-to-end data migration initiatives, ensuring a smooth, secure, and business-aligned transition. - Define and communicate data architecture decisions, evaluating impacts, risks, and performance considerations across cloud data platforms such as AWS Redshift, Databricks, and related technologies. - Coordinate the discovery and documentation of dependencies, schema inventories, data flows, and critical processes involved in migration efforts. - Own migration and cutover planning, ensuring all activities, validations, and contingency measures are properly defined and executed. - Serve as the primary liaison between engineering, analytics, product, and business stakeholders, facilitating clear communication and alignment throughout the project lifecycle. - Translate technical changes in data pipelines, data warehouses, and platform architecture into clear business impact assessments. - Define and document assumptions, requirements, and acceptance criteria to ensure successful migration outcomes. - Oversee data validation processes to guarantee the accuracy, consistency, and integrity of migrated datasets against source-of-truth systems. - Proactively identify, assess, and mitigate migration-related risks, escalating blockers and dependencies when necessary to maintain project timelines and quality standards. - Drive problem-solving and decision-making in fast-paced, ambiguous environments, helping cross-functional teams stay focused on priorities and deliverables. - Create, maintain, and improve project documentation, including migration runbooks, testing strategies, rollback plans, and operational procedures. - Provide regular project status updates to technical and executive stakeholders, ensuring transparency around progress, risks, and next steps. - Promote data governance, quality, and operational best practices throughout the migration lifecycle. - Align engineers, analysts, and product stakeholders around priorities, timelines, and delivery expectations to ensure successful project execution. ### Required Skills 💻 - Experience in Data Engineering, including building and optimizing data pipelines. - Experience working with Databricks, Python, and Spark, building batch and streaming. - Advanced English Level. - Familiarity with a BI visualization tool like Looker or Tableau. ### 🎁 Perks - **Remote-first culture** – work from anywhere! 🌍 - **AWS, DBT, Google Cloud, Azure & Databricks** certifications fully covered - In-Company **English Lessons.** - **Birthday off + an extra vacation week** (Mutt Week! 🏖️) - **Referral bonuses** – help us grow the team & get rewarded! - **[Maslow](https://maslow.hr/):** Monthly credits to spend in our benefits marketplace. - ✈️**🏝️ [Annual Mutters' Trip](https://www.youtube.com/watch?v=dq1x2krHGb8&t=2s)** – an unforgettable getaway with the team. - **👶 Monthly Childcare Reimbursement** – Because supporting families matters too