Data Engineer II

Job description

About G2 - The Company G2 is the world's largest and most trusted software marketplace. When you join G2, you’re joining the industry’s leading team that helps businesses reach their peak potential by powering decisions and strategies with trusted insights from real software users. Now, we have joined forces with Capterra https://www.capterra.com/, SoftwareAdvice https://www.softwareadvice.com/, and GetApp https://www.getapp.com/ to create the largest source of online data and software insights to fuel intelligent buying in the age of AI. With 200M+ combined annual visitors and 6M verified reviews, we are now the centralized place to enable software buyers to make better and faster decisions with confidence. And we are just getting started! We are setting out to transform the global B2B software industry and become the most trusted data foundation for buyers and sellers of software for the age of AI. Does that sound exciting to you? Come join us as we try to reach our next PEAK! About G2 - Our People At G2, everything we are and what we do is grounded in our PEAK values— (Performance + Entrepreneurship + Authenticity + Kindness. Working at G2 means you are part of a value-driven, growing global community that climbs PEAKs together. We cheer for each other’s successes, learn from our mistakes, and support and lean on one another during challenging times. With ambition and entrepreneurial spirit we push each other to take on challenging work, which will help us all to grow and learn. You will be part of a global, diverse team of smart, dedicated, and kind individuals - each with unique talents, aspirations, and life experiences. At the heart of our community and culture are our people-led ERGs, which celebrate and highlight the diverse identities of our global team. As an organization, we are intentional about our DEI https://company.g2.com/dei and philanthropic work (like our G2 Gives https://company.g2.com/gives program) because it encourages us all to be better people. ABOUT THE ROLE G2 is looking for a Data Engineer - II, who will lead the design, development and maintenance of complex data engineering projects and pipelines for the G2 data platform. You'll lead and contribute significantly to complex data engineering projects and the implementation of complex ETL and ELT processes within an AWS and Snowflake ecosystem, demonstrating expertise in scalable and efficient data processing. IN THIS ROLE, YOU WILL: Data infrastructure and processing: - Lead the design and development of data pipelines for seamless integration of data from various sources into the G2 Data Platform. - Optimize data pipelines, ensuring cost effectiveness, scalability, and reliability. - Constantly innovate to make the data stack follow the latest trends. - Service data requests from various users of the G2 data platform. - Actively contribute to data modeling and design reviews, striving for improved adoption and efficiency. - Execute the project tasks aligned with project timelines and objectives under the guidance of senior team members. - Develop repeatable and scalable code that processes data to ensure data availability in the platform is as real-time as possible. - Actively contribute to the development and advancement of the data platform. - Promote architectural changes that increase the scalability of our data infrastructure while maintaining efficiency in all phases of the development lifecycle. Data quality assurance and governance : - Learn and adopt best practices in data engineering to contribute to robust solutions. - Own the implementation of data quality and data governance initiatives and drive them to completion. - Recommend and ensure the data platform follows privacy and security standards and requirements. - Document data architecture, data model, and workflows for how it should be consumed. Mentorship and Collaboration : - Guide junior engineers by providing technical support, expertise, best practices, and constructive feedback on data engineering techniques. - Collaborate with peers, actively participating in knowledge sharing sessions and contributing to a collaborative team environment. - Seek guidance and mentorship from senior team members to enhance technical and analytical skills. MINIMUM QUALIFICATIONS: We realize applying for jobs can feel daunting at times. Even if you don’t check all the boxes in the job description, we encourage you to apply anyway. - 4+ years of experience as a data engineer or ETL developer. - 2+ years of development experience with sound skills in data modeling, optimization, and database architecture. - Experience in the design and development of data pipelines using cloud and open-source tools. - Proficiency in writing and debugging SQL queries. - Good programming skills in Python or Java. - Must have good knowledge of performance tuning, optimization and debugging of data pipelines. - Working knowledge of the AWS data services (DynamoDB, RDS, Data Pipeline, EMR, Lambda, Glue, ECS, etc.) and cloud data warehouses like SnowFlake. - Proficiency in handling structured and unstructured data. - Proficiency in ELT/ELT tools like AWS Glue, Step Functions, Data Pipelines, Airflow, Airbyte, and DBT. - Familiarity with distributed computing and frameworks like Apache Spark, Hadoop, and Apache Kafka for handling large volumes of data. - Familiarity with software engineering principles and best practices. WHAT CAN HELP YOUR APPLICATION STAND OUT: - Experience with Docker and Kubernetes. - Proficiency in data modeling, schema design, and optimizing data structures for performance in Snowflake. - Working experience in startup environments. - Experience with Agile process methodology, CI/CD automation, Test Driven Development. - Knowledge of data governance, security, and compliance standards within cloud-based data solutions. - Understanding of any reporting tools such as Tableau, Qlikview ,Looker, PowerBI, etc. - Database administration background. Our Commitment to Inclusivity and Diversity At G2, we are committed to creating an inclusive and diverse environment where people of every background can thrive and feel welcome. We consider applicants without regard to race, color, creed, religion, national origin, genetic information, gender identity or expression, sexual orientation, pregnancy, age, or marital, veteran, or physical or mental disability status. Learn more about our commitments here https://culture.g2.com/commitments. -- For job applicants in California, the United Kingdom, and the European Union, please review this applicant privacy notice https://legal.g2.com/applicant-privacy-notice before applying to this job. How We Use AI Technology in Our Hiring Process G2 incorporates AI-powered technology to enhance our candidate evaluation process. These tools may assist with initial application screening, skills assessment analysis, and identifying candidates whose qualifications align with specific role requirements. While AI technology supports our recruitment workflow, all final hiring decisions remain under human oversight and judgment. Your Choice Matters: If you would prefer that your application be reviewed without AI assistance, you can opt out by entering your email address in the email entry field at the bottom of the Automated Processing Legal Notice. Choosing to opt out will not disadvantage your application in any way—we will ensure your materials receive a thorough manual review by our hiring team. For additional details about how we handle your information throughout the application process, please review G2's Applicant Privacy Notice https://legal.g2.com/applicant-privacy-notice.