Senior Data Engineer (Pyspark, Databricks)

Job description

Senior Data Engineer (PySpark) About the Role We are looking for a Senior Data Engineer with 5+ years of experience to join our Data Engineering team. The ideal candidate should have strong hands-on experience with PySpark, Spark, SQL, and data pipelines, along with the ability to work with large-scale datasets. You will play a key role in building and optimizing data pipelines while improving their reliability, efficiency, scalability, and performance. This is also an opportunity to work on AI-ready datasets and modern data platforms, with exposure to technologies such as Databricks and Spark. Key Responsibilities Build and maintain end-to-end data pipelines. Develop and implement best practices for data modeling and pipeline development. Create AI-ready analytical datasets with appropriate structure, metadata, documentation, and business context. Solve complex data pipeline challenges using PySpark and SQL. Work with stakeholders to understand requirements and incorporate business logic into data pipelines. Work with large-scale datasets and optimize data processing for performance and reliability. Contribute to the development and improvement of data engineering standards and processes. Gain hands-on exposure to Databricks, Spark, AI technologies, and modern ETL tools. Collaborate closely with engineering and business stakeholders to deliver high-quality data solutions. What We're Looking For 5+ years of experience in Data Engineering or a related technical role. Strong hands-on experience with PySpark, Spark, and SQL. Proven experience building and maintaining data pipelines. Experience working with large-scale datasets. Strong understanding of data transformations, modeling, and pipeline architecture. Hands-on experience with Databricks, Delta Lake, or similar technologies. Ability to understand business requirements and translate them into effective data solutions. Strong problem-solving and analytical skills. Excellent verbal and written communication skills. Self-motivated, collaborative, and comfortable taking ownership. Willingness and ability to learn new technologies quickly. Nice to Have Apache Airflow dbt Snowflake Modern ETL/ELT tools AI/ML data pipelines Education Bachelor’s or Master’s degree in Computer Science Culture of Belonging: At SSI, we are committed to fostering a culture of belonging where everyone feels valued, respected, and empowered to contribute and grow.