Senior Data Engineer

Job description

Location: Pune, Maharashtra Summary The Senior Data Engineer, plays a critical role in designing, building, and operating enterprise grade data products that power analytics, reporting, and AI across Hershey’s business domains. This role ensures data pipelines, models, and foundational components are scalable, performant, governed, and reusable — enabling consistent access to trusted data at enterprise scale. Senior Data Engineers work closely with Data Product Managers, Architects, Domain SMEs, and Platform Engineering to translate business requirements into robust technical solutions using modern data engineering patterns and cloud-native tooling. The role supports the end-to-end lifecycle of data products, from ingestion and modeling to certification, documentation, and sustainable operations. This role is foundational to Hershey’s long-term data strategy. By contributing to reusable, governed, and durable data products, the Senior Data Engineer helps establish the enterprise system of record for analytics, enabling efficiency, consistency, and innovation across the company. What We Are Building for Hershey This role contributes directly to Hershey’s enterprise data strategy by engineering durable, reusable, and governed data products that become the system of record for analytics and future AI innovation. These data products are designed to scale across domains, reduce duplication, increase trust, and accelerate enterprise decision-making. By applying engineering rigor, governance-by-design practices, and modern cloud-native architecture, the Senior Data Engineer helps transition Hershey from one-off data solutions to long‑lived, production-grade assets that fuel efficiency, growth, and innovation across the enterprise. Major Duties & Responsibilities 1. Data Product Engineering & Delivery Partner with business stakeholders to translate business requirements into technical designs and acceptance criteria. Design, build, and maintain scalable and reusable data pipelines and transformations using Azure and Databricks. Implement robust ingestion frameworks, standardized patterns, and repeatable workflows aligned to enterprise engineering practices. Build and optimize physical and semantic data models supporting analytics, reporting, and AI/ML use cases. 2. Technical & Architectural Implementation Apply best practices related to performance tuning, cost optimization, security, and reliability. Ensure alignment with enterprise architecture, including shared infrastructure, canonical models, and platform standards. Implement efficient data processing patterns (e.g., Delta Lake, medallion architecture, orchestration frameworks). 3. Governance, Quality, & Operations Embed governance-by-design principles including lineage, metadata, documentation, and certification standards. Implement data quality rules, monitoring, and automated checks to ensure accuracy, completeness, and trust. 4. Collaboration Across Domains Work closely with business domain teams to validate that data products meet end-user needs and match business definitions. Partner with Platform Engineering on pipeline frameworks, infrastructure patterns, and optimization. Collaborate with Data Operations & Enablement to ensure strong documentation, certification, and governance coverage. Minimum Knowledge, Skills, and Abilities Data Engineering & Pipelines: Strong experience with ETL/ELT design, distributed processing, pipeline optimization, and enterprise scale data workflows. Cloud & Platform Expertise: Hands-on experience with Databricks and cloud data platforms such as Azure (preferred) or AWS, including data lakes, orchestration, and scalable compute. Programming & Development: Proficient in Python, SQL, modular coding, APIs, source control, and automation tools. Data Modeling & Architecture: Skilled in dimensional and semantic modeling, database design, and building performant, governed analytics-ready structures. Data Quality, Metadata & Curation: Capable of cleansing, transforming, and validating data; familiar with lineage, cataloging, and data quality frameworks. Collaboration & Communication: Able to translate business needs into technical solutions and clearly communicate technical concepts in cross-functional settings. Experience and Education Bachelor's/Master's in Data Science, Engineering, or a related field. 5 – 7 years of experience in data, analytics, or engineering roles Proficiency in one or more general-purpose programming languages (e.g., Java, C/C++, C#, Python, JavaScript). Working knowledge of SQL-based and NoSQL technologies (e.g., PostgreSQL, MySQL, MongoDB) #LI-SS1 #LI-HYBRID