Senior Databricks Engineer

Job description

Who We Are HIKE2 is an advisory and innovation partner helping organizations design and build what’s next. We work with complex and regulated industries, including law firms, financial services, insurance, professional services, and high-growth SaaS companies, to modernize operating models and embed data, automation, and AI into the core of how work gets done. Our clients aren’t just upgrading technology. They are rethinking workflows, activating governed AI, and designing environments where humans and intelligent systems work side by side. We believe meaningful transformation happens when strategy, data, design, and engineering move together. Siloed initiatives don’t scale. Real progress requires clarity of vision, strong data foundations, responsible governance, and disciplined execution. Our teams bring deep industry expertise, human-centered design, and advanced data and cloud capabilities to architect secure, scalable solutions built for real-world complexity. From modern data platforms to AI-enabled workflows and enterprise automation, we help organizations move from experimentation to measurable impact. We thrive in change and move from blank slate to working systems. We care deeply about outcomes, trust, and building long-term partnerships. We don’t just implement technology. We help design the future of work. HIKE2 is looking for a Senior Data Engineer with deep experience in Databricks and modern data platforms, particularly in large, complex enterprise environments. This is a hands-on role for someone who has built and delivered data solutions at scale, ideally in greenfield settings, and is comfortable working with Fortune 500 clients. You’ll be expected to lead technical direction, make sound architectural decisions, and support the growth of other engineers on the team. You’ll work closely with clients and internal teams to design and implement enterprise-grade data platforms and pipelines from the ground up. This role requires someone who can operate across the full lifecycle from early architecture and design through delivery and optimization, while keeping solutions practical, reliable, and aligned with business needs. What You’ll Do Design and build large-scale data platforms on Databricks (Delta Lake, Spark, Unity Catalog) in Azure Develop and maintain batch and streaming data pipelines for high-volume, complex data sources Implement medallion/lakehouse architectures from the ground up in greenfield environments Build and optimize data models to support analytics, reporting, and downstream applications Integrate Databricks with enterprise systems (APIs, event streams, warehouses, ML workflows) Tune Spark jobs and pipelines for performance, reliability, and cost at scale Support production deployments, including CI/CD pipelines, testing, and release management How You’ll Work Partner directly with enterprise clients to translate requirements into working technical solutions Collaborate with architects, engineers, and data scientists across multiple workstreams Balance speed and quality, knowing when to move fast and when to harden solutions Make pragmatic decisions in ambiguous, evolving environments (especially greenfield builds) Contribute hands-on while also guiding design and approach across the team Communicate tradeoffs clearly to both technical and non-technical stakeholders Work within modern engineering practices (version control, code reviews, automated testing) Demonstrated ability to mentor and guide data engineers and analysts What You’ll Own End-to-end delivery of Databricks-based data solutions—from design through production support Technical direction and key architecture decisions for large-scale implementations Data pipeline reliability, monitoring, and incident response in production environments Performance and cost efficiency of workloads running in Databricks and Azure Data quality, governance alignment, and adherence to enterprise security standards Reusable patterns, frameworks, and standards for scaling future implementations Mentorship and technical development of other engineers on the team Desired Skills and Experience: Deep Databricks-native expertise, including experience architecting and implementing end-to-end lakehouse solutions that run primarily or entirely on Databricks. Advanced experience with modern Databricks architecture patterns, including declarative pipelines / Delta Live Tables, Unity Catalog, Delta Lake, workflow orchestration, governance, performance tuning, and operational monitoring. Familiarity with infrastructure-as-code (Terraform, Bicep), environment provisioning, and CI/CD automation (Github, Azure DevOps) for Databricks-based platforms. Strong learning agility, technical curiosity, and comfort using AI-enabled development workflows or automation tools to accelerate delivery and improve quality. Familiarity with other modern cloud data architectures and tools, including cloud-native data warehouses (Snowflake, BigQuery, Redshift), data lakes, orchestration frameworks (Airflow/Astronomer), transformation tools (dbt), catalog/governance platforms, and scalable batch or streaming data processing services (Kafka, Kinesis). The salary range for this position is: $135,000-185,000K. Actual compensation within the range will be dependent upon the individual's skills, experience, qualifications and location, and applicable employment laws. This range is exclusive of any bonuses or commissions. HIKE2 offers a wide range of benefits which may include medical, dental, vision, 401k, holiday pay, vacation, personal and family sick leave, and more. Don’t meet every requirement? As an innovation consultancy, we recognize that talent is not always related to job history and skills come from valuable experiences that aren’t always shown on a resume. If you are excited to make a positive impact, but your experience doesn’t align perfectly, we encourage you to apply. You may be the right candidate for our organization, whether it be this role or another. Accommodation Statement We are committed to providing reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs throughout the hiring process. If you require an accommodation to complete the application or participate in the interview process, please let us know. BGC Notice Employment may be contingent upon successful completion of a background check, conducted in compliance with applicable federal, state, and local laws. For only those qualified applicants that are impacted by the Los Angeles County Fair Chance Ordinance for Employers, the Los Angeles' Fair Chance Initiative for Hiring Ordinance, the San Francisco Fair Chance Ordinance, San Diego County Fair Chance Ordinance, and the California Fair Chance Act, where applicable, arrest or conviction records will be considered for Employment in accordance with these laws. At HIKE2, we recognize that conviction records may have a direct, adverse, and negative relationship to responsibilities such as accessing sensitive company or customer information, handling proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted workplace for all. US Citizenship Due to the nature of our client work, all applicants must reside in the U.S. and hold US Citizenship to be considered for employment. Hike2 cannot sponsor work authorizations or visas now or in the future. EEO Statement We are an equal opportunity employer and are committed to creating an inclusive environment for all employees and applicants. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity or expression, national origin, age, disability, genetic information, protected veteran status, marital status, or any other status protected by applicable federal, state, or local law. Disclaimer This job description is not intended to be an exhaustive list of duties, responsibilities, or qualifications and may be modified at any time to meet business needs.