Data Engineer – Specialist

Job description

Role: Data Engineer - Specialist Location: Hyderabad / Bangalore Full/ Part-time: Full time Build a career with confidence Carrier Global Corporation, global leader in intelligent climate and energy solutions is committed to creating solutions that matter for people and our planet for generations to come. From the beginning, we've led in inventing new technologies and entirely new industries. Today, we continue to lead because we have a world-class, diverse workforce that puts the customer at the center of everything we do. About the Role: As a Specialist Data Engineer, you will design and build scalable, reusable, and governed data pipelines and curated datasets on an AWS-based lakehouse architecture. You will help evolve a transactional lakehouse using open table formats (e.g., Apache Iceberg) and implement engineering standards that keep the platform interoperable across query engines and cloud providers. You’ll work across batch and streaming, contribute to a medallion (raw/silver/gold) pattern, and ensure datasets are discoverable, trustworthy, and platform-agnostic through metadata, lineage, and governance integration. Key Responsibilities: 1) Data Pipeline & Lakehouse Engineering Design and implement robust, reusable data pipelines for batch and streaming use cases using AWS-native services (e.g., S3, Glue, Kinesis) and orchestration tools (e.g., Airflow) where applicable. Build and standardize medallion-layered ingestion and transformation patterns (raw → silver → gold) as a repeatable engineering approach. Develop and optimize Iceberg (or similar open table format) datasets with strong practices for schema evolution, partitioning, and performance for multi-engine consumption. 2) Open Standards, Interoperability & Cloud-Agnostic Delivery Apply open standards to reduce lock-in by designing storage and metadata layers that work across engines (e.g., Athena/Trino/EMR/Redshift/Snowflake/Databricks depending on enterprise choices). Contribute to enterprise adoption of Apache Iceberg as the open table standard for interoperability and portability across environments. Implement standardized interfaces for pipelines and data products (e.g., config-driven patterns) to support portability and consistent operations. 3) Data Quality, Governance, Metadata & Lineage Embed automated quality checks, data validation, and pipeline test coverage, ensuring trusted datasets for analytics and AI/ML. Emit lineage/metadata signals by instrumenting pipelines to produce OpenLineage events (or equivalent enterprise lineage standards) and register assets in the enterprise catalog as required. Ensure consistent ownership, documentation, and discoverability for produced datasets/data products. 4) Operational Excellence (DataOps/DevOps) Champion CI/CD for data pipelines and infrastructure changes, including automated checks and safe promotion across environments. Implement observability (metrics, logs, alerts) and contribute to incident triage and reliability improvements for production pipelines. Partner with Security/Platform teams on IAM least privilege, access controls, and governed data access patterns. 5) Technical Leadership & Collaboration Work closely with platform engineers, data product owners, governance teams, and downstream consumers to deliver curated datasets and reusable platform capabilities. Mentor junior engineers and help define internal standards, frameworks, and best practices for lakehouse engineering. Required Qualifications 6 to 10 years years of experience in data engineering or related roles. Proficiency in Python and SQL. Strong understanding of batch and streaming data processing. Experience delivering production‑ready data products. Benefits We are committed to offering competitive benefits programs for all of our employees, and enhancing our programs when necessary. Enjoy your best years with our retirement savings plan Have peace of mind and body with our health insurance Make yourself a priority with flexible schedules, parental leave and our holiday purchase scheme Drive forward your career through professional development opportunities Achieve your personal goals with our Employee Assistance Programme. Our commitment to you Our greatest assets are the expertise, creativity and passion of our employees. We strive to provide a great place to work that attracts, develops and retains the best talent, promotes employee engagement, fosters teamwork and ultimately drives innovation for the benefit of our customers. We strive to create an environment where you feel that you belong, with diversity and inclusion as the engine to growth and innovation. We develop and deploy best-in-class programs and practices, providing enriching career opportunities, listening to employee feedback and always challenging ourselves to do better. This is The Carrier Way . Join us and make a difference. Apply Now! Carrier is An Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status, age or any other federally protected class. Job Applicant's Privacy Notice: Click on this link to read the Job Applicant's Privacy Notice