Staff Data Engineer-GCP/Data Solutions

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Staff Data Engineer-GCP/Data Solutions based in United States.** This is a senior technical role focused on designing and delivering scalable data solutions that support analytics, applications, data science, and business decision-making. You will architect complex batch and real-time data pipelines across cloud environments, with a strong emphasis on GCP, AWS, BigQuery, and Databricks. The role combines hands-on engineering with technical leadership, requiring you to solve complex data challenges and influence architecture and engineering standards. You will collaborate closely with Product, Data Science, Application, Analytics, Business, and Engineering teams to translate evolving requirements into reliable data solutions. Your work will span data integration, streaming, warehousing, data quality, monitoring, security, scalability, and production support. As a Staff-level contributor, you will also mentor engineers, participate in design reviews, and help advance tools, patterns, and technical practices. The ideal candidate is a strategic and adaptable data engineer who can operate effectively in high-scale, mission-critical environments while delivering high-quality results. ### Accountabilities - Design and implement data patterns that support analytics datasets, including data cleansing, standardization, calculations, validation, and the mapping and linking of information across multiple sources. - Build, maintain, streamline, and orchestrate complex end-to-end data pipelines across large-scale cloud environments, including GCP and AWS. - Design and develop scalable data warehousing, data integration, and streaming solutions using technologies such as BigQuery and Databricks. - Develop large-scale batch and real-time data pipelines using big data Change Data Capture (CDC) processing frameworks and structured streaming technologies. - Design and implement real-time streaming solutions using Pub/Sub or comparable streaming tables and data lake technologies. - Build and support complex data pipelines and Application Programming Interfaces (APIs), identifying and integrating complex upstream data sources to enable new capabilities. - Develop and maintain data integrations supporting application engineering, system integration, analytics, and broader business requirements. - Perform data validation, quality assurance, automated testing, and end-to-end technical testing to ensure data integrity, system functionality, and solution reliability. - Design and implement data quality checks and monitoring capabilities aligned with business Service Level Agreements (SLAs) and data quality requirements. - Diagnose and resolve complex production support issues while maintaining the performance, scalability, reliability, security, and efficiency of data platforms. - Participate in technical and architectural design reviews with Staff Engineers and adapt engineering patterns based on emerging technologies, tools, and best practices. - Contribute to the full Software Development Life Cycle (SDLC), including planning, design, development, certification, implementation, testing, and ongoing support. - Collaborate with Product, Data Science, Application, Analytics, Business, and other technical teams to understand data and infrastructure requirements and resolve complex data-related challenges. - Work with external technical teams, shared services organizations, and vendors when necessary to deliver timely and high-quality solutions. - Develop documentation, training materials, technology standards, and engineering guidance for assigned teams. - Contribute to the refinement of engineering tools, standards, processes, and training across the organization. - Build complex data models that enable insightful analytics while maintaining high standards of data integrity and quality. - Apply machine learning concepts and contribute to predictive model implementations, including prompt engineering considerations for AI models. - Contribute to overall data architecture, modeling, security, scalability, reliability, and performance decisions. - Mentor and provide technical guidance to Senior Data Engineers while participating in peer reviews and fostering continuous learning across the team. ### **Requirements** - Bring at least 8 years of professional experience in data engineering or a closely related field. - Hold a Bachelor’s degree in a relevant discipline or possess equivalent professional experience. - Have experience working with large-scale cloud infrastructure, large datasets, and mission-critical Service Level Agreements (SLAs). - Demonstrate advanced proficiency in SQL and Python for data engineering and analytical workloads. - Hold a data engineering certification from a recognized cloud technology provider. - Demonstrate experience with cloud data platforms and technologies, particularly GCP and/or AWS, with hands-on experience supporting complex data solutions. - Have experience with BigQuery, Databricks, data warehousing, data lakes, data meshes, and modern data architecture patterns. - Demonstrate strong knowledge of data architecture and data modeling best practices and the ability to determine appropriate approaches for different data and analytics platforms. - Have experience designing and supporting batch and real-time streaming pipelines, including structured streaming, Pub/Sub or comparable technologies, and Change Data Capture (CDC) frameworks. - Demonstrate strong understanding of coding standards, software design principles, design patterns, testing practices, and secure engineering. - Have experience with Agile or Lean Startup development methodologies and the ability to operate effectively within iterative delivery environments. - Possess a strong understanding of business intelligence, analytics, reporting, and application integration. - Demonstrate strong prioritization and problem-solving skills, with the ability to deliver complex projects despite ambiguity or incomplete information. - Communicate effectively through strong verbal, written, and data presentation skills, with the ability to work with both technical and business stakeholders. - Demonstrate the ability to collaborate across multiple teams and locations while contributing positively to shared development environments. - Be adaptable and willing to learn and apply emerging technologies as data engineering practices evolve. - Remain calm and effective under pressure and be comfortable working a flexible schedule when business or operational needs require it. - Demonstrate a collaborative mindset, openness to peer feedback, and willingness to both mentor others and learn from colleagues and technical mentors. - Be comfortable working in a fast-paced environment with the flexibility to work from home and/or an office as required. ### **Benefits** - Opportunity to work in a Staff-level data engineering role with broad technical ownership and influence. - Exposure to large-scale cloud infrastructure, modern data architectures, real-time streaming, analytics, machine learning, and AI-related technologies. - Collaborative environment involving Product, Data Science, Application, Analytics, Engineering, and Business teams. - Opportunities to mentor other engineers and contribute to technical standards, architecture, engineering practices, and professional development. - Flexible work environment with the ability to work from home and/or an office as business needs require. - Opportunity to work on complex, high-impact data solutions supporting mission-critical business operations.