Senior Data Engineer

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer based in the United States.** This role is focused on designing, building, and optimizing scalable data solutions that transform complex datasets into reliable, actionable insights. You will develop production-grade pipelines and workflows supporting large-scale data ingestion, transformation, storage, and analysis. The position combines cloud-native engineering, big data technologies, software development, and data architecture. You will work closely with engineering, product, analytics, SRE, and other cross-functional teams to deliver dependable data capabilities. The role offers the opportunity to contribute to innovative work involving big data, artificial intelligence, and machine learning. Success requires strong engineering judgment, a focus on performance and reliability, and the ability to solve complex technical problems in an iterative environment. This is a remote, full-time opportunity with the potential to make a meaningful impact through modern, data-driven solutions. ### Accountabilities - Design, develop, maintain, and optimize scalable data pipelines and workflows using Azure Databricks and related cloud technologies. - Build ETL/ELT pipelines that integrate data from multiple internal and external sources while maintaining strong standards for data quality, reliability, and performance. - Develop and maintain SAS programs and SSIS packages supporting intermediate and downstream data processing. - Write, optimize, and maintain complex SQL queries for data transformation, processing, reporting, and analytical use cases. - Implement and support Big Data tools and frameworks for large-scale data processing and integration. - Monitor data workflows, troubleshoot complex processing issues, perform root-cause analysis, and implement preventative solutions. - Optimize pipelines and data workflows for scalability, reliability, performance, and cost efficiency. - Evaluate and recommend infrastructure and technology improvements as data volumes and processing requirements evolve. - Partner with product management and SRE teams to understand requirements and translate them into actionable engineering solutions. - Collaborate with software engineers, analysts, stakeholders, and cross-functional partners to align data strategy, architecture, and delivery outcomes. - Apply sound engineering practices and design patterns to support maintainable, scalable data and software solutions. - Contribute to solutions involving big data, artificial intelligence, and machine learning that help transform complex information into actionable insights. ## Requirements - Bachelor’s degree in science, engineering, computer science, or a related discipline, or equivalent practical experience. - 8+ years of overall experience in data engineering or a closely related field. - 8+ years of experience designing and building production-grade batch data pipelines for data ingestion and transformation. - Strong proficiency in SQL, Databricks, Python, Apache Spark, Git, and Azure services, including Azure Synapse and Azure Data Lake Storage Gen2. - Strong understanding of relational database principles, data modeling, database management, and data warehousing. - Strong knowledge of Big Data querying and processing tools. - Demonstrated experience optimizing large-scale data pipelines and ETL workflows. - Experience working with cloud-based data platforms and modern data engineering architectures. - Strong analytical and problem-solving capabilities, with the ability to investigate complex technical issues and identify sustainable solutions. - Excellent written and verbal communication skills and the ability to collaborate effectively with technical and non-technical stakeholders. - Ability to manage multiple projects and priorities in a fast-paced, iterative environment. - Experience with cloud migrations and legacy modernization is preferred. - Experience with Snowflake is a plus. - Healthcare industry experience and knowledge of designing systems that protect sensitive PHI and PII are preferred. - A proactive, resourceful, resilient, and continuous-improvement mindset. ## Benefits - Remote work opportunity available nationwide within the United States. - Full-time schedule of approximately 40 hours per week. - Base salary range of **$117,600–$161,700 per year**, with actual compensation varying based on geographic location, experience, education, certifications, and job-related skills. - Eligibility for an annual performance-based incentive plan. - Health benefits effective from day one. - Generous paid time off. - 401(k) retirement savings plan with company matching. - Employee well-being program and resources supporting overall health and productivity. - Paid volunteer time off. - Telephone equipment appropriate for the role's business requirements. - Internet expense support may be available to eligible remote employees in certain states. - Occasional travel to company offices for training or meetings may be required. - Remote work requires a dedicated workspace and reliable internet connection, with a recommended minimum of 25 Mbps download and 10 Mbps upload speeds.