ETL Tester- Bigquery

Job description

 ## Overview of 66degrees 66degrees is an end-to-end AI transformation partner that guides enterprises from complex business challenges to clear, quantifiable outcomes. Our company is the culmination of several successful firms, each a leader in its own right in cloud, artificial intelligence, and data. This convergence of talent and expertise is how we help businesses reach their own "inflection point," where chaotic data becomes a strategic asset, complexity becomes clarity, and AI becomes an engine for growth. Our ultimate vision is to be the catalyst for a future where every business operates as an intelligent enterprise, with autonomous systems unlocking human potential. At 66degrees, we believe in thriving through challenges and winning together. These values not only guide us in achieving our goals as a company but also for our people. We are dedicated to creating a significant impact for our employees by fostering a culture that sparks innovation and supports professional and personal growth along the way. **Overview of Role** As a Senior QA Engineer – Data, you will lead the QA effort for complex data engineering pipelines on Google Cloud Platform. You’ll design and implement multi-layered testing strategies—integration, end-to-end, and data quality tests—across tools like dbt, Dataflow, Dataproc, BigQuery, AlloyDB, Cloud SQL, Cloud composer and Cloud Run, etc. Your work ensures the accuracy, reliability, and performance of data systems at scale. **Responsibilities** Testing Strategy & Test Design • Define and maintain testing methodologies for the full GCP data engineering stack: Dataflow (TestPipeline), Dataproc (spark-testing-base or pytest), Cloud Run container tests, and SQL-based data validation in BigQuery/dbt. • Develop and execute data quality frameworks using dbt tests (schema, singular, freshness) and external tools like Great Expectations, Soda Core, and Dataplex. Pipeline & Database Testing • Implement integration, and regression tests for ETL/ELT pipelines, including container- level and HTTP-triggered tests for Cloud Run. • Use emulators or dedicated test instances to test Spanner, Cloud SQL, and AlloyDB. Validate stored procedures and database functions with sample data. End-to-End (E2E) Pipeline Validation • Orchestrate comprehensive E2E tests via Cloud Composer/Apache Airflow or scripting. Simulate real-world data flows and validate intermediate and final outputs. CI/CD & Automation • Embed QA in CI/CD pipelines (e.g., GitLab CI, Jenkins, GitHub Actions), automating test execution at all levels including data quality validations and dbt runs. • Use IaC tools (Terraform, Deployment Manager) to provision reproducible test environments. Collaboration & Stakeholder Engagement • Partner with data engineers and stakeholders to review design for testability. • Mentor junior QA team members, champion QA best practices, and lead efforts to improve data quality KPIs and test effectiveness. Monitoring & Observability • Utilize Cloud Logging, Monitoring, and observability tools (e.g., Elementary Data) to track pipeline health, test results, and identify anomalies. **Required Qualifications** • 5+ years in QA or software testing, including focused on data pipelines/data warehouses • Proficient in complex SQL and writing data validation queries. • Strong experience with GCP data tools: dbt, BigQuery, Dataflow, Dataproc, Cloud Run, Spanner, AlloyDB, Cloud SQL. • Hands-on with data-quality frameworks: Great Expectations, Soda Core, dbt-utils, dbt-expectations. • Familiarity with database emulators and test environment provisioning. • CI/CD automation experience (Jenkins, GitLab CI, GitHub Actions). • Skilled in scripting languages (Python, pytest). • Excellent communication, mentoring ability, critical thinking, and attention to detail. • API testing with PI Testing Postman REST APIs **Preferred Qualifications** • GCP Professional Data Engineer certification • Experience enhancing dbt tests with dbt-utils and dbt-expectations. • Knowledge of BigQuery data-quality services like Dataplex. • Proficient with observability tooling (Cloud Logging, Elementary Data). • Familiarity with data governance, lineage, and compliance validation. • Exposure enterprise-based data migration project • Exposure to Datawarehouse modernization ***Please note, we cannot sponsor or transfer any visas, of any kind, at this time.*** *66degrees is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to actual or perceived race, color, religion, sex, gender, gender identity, national origin, age, weight, height, marital status, sexual orientation, veteran status, disability status or other legally protected class.* ### **AI Transparency & Disclosure** As an AI transformation partner, 66degrees leverages intelligent solutions to enhance our recruitment experience. We utilize AI tools—including LinkedIn Recruiter’s Hiring Assistant and interview transcription technologies—to assist with sourcing, role analysis, and capturing interview highlights. These tools augment our process, but we "Commit to Our Craft" by ensuring all final hiring decisions are made by our human Talent Team. By applying, you acknowledge the use of these technologies to help us "Win Together" in finding the best fit for our team.