Data Engineer

Job description

Your Journey to Success: The Data Engineer will design, implement, and manage scalable data pipelines, databases, and processing systems to support engineering, product, and analytics teams. This includes developing and administering platforms like Databricks and Snowflake, optimizing data architecture, and ensuring performance, scalability, and reliability. Your Mission: Data Infrastructure Development: Design, build, and maintain scalable data pipelines to process large volumes of structured and unstructured data. Optimize data architecture for both real-time and batch processing workflows. Integrate and maintain PostgreSQL, MySQL, Databricks, and Snowflake within our data ecosystem. Databricks Development and Administration: Develop and manage workflows on Databricks to support ETL/ELT processes. Optimize Databricks performance for distributed computing and data processing. Monitor, troubleshoot, and ensure the reliability of Databricks clusters and jobs. Snowflake Development and Administration: Design and optimize Snowflake schemas and data warehouses for efficient querying. Develop and manage data ingestion pipelines for Snowflake using modern tools and frameworks. Administer Snowflake environments, including account management, performance tuning, and resource optimization. Database Management: Administer PostgreSQL and MySQL databases, ensuring high availability and performance. Implement and enforce data governance practices to ensure data security and compliance. Collaboration: Partner with analytics and engineering teams to define data needs and deliver solutions for business insights. Work with product teams to align data solutions with product goals and user requirements. Monitoring and Troubleshooting: Set up monitoring and alerting systems for data platforms and pipelines. Proactively debug and resolve data-related issues to ensure system reliability. Innovation and Best Practices: Stay current with emerging trends and advocate for tools and practices that enhance data engineering efficiency. Document workflows, pipelines, and platform configurations to foster team collaboration and knowledge sharing. What Sets You Apart Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience). 3+ years of professional experience in data engineering or related fields. Proficiency in Databricks development and cluster administration. Hands-on experience with Snowflake development and administration, including performance tuning and security configuration. Strong understanding of relational databases, particularly PostgreSQL and MySQL. Expertise in programming languages like Python, Scala, or SQL for data processing and automation. Familiarity with ETL/ELT tools and frameworks such as Apache Airflow or dbt. Experience with cloud platforms (AWS, Azure, or GCP). Preferred Skills: Knowledge of distributed systems and big data frameworks like Spark. Experience with CI/CD pipelines for deploying data workflows. Familiarity with real-time streaming technologies (Kafka, Kinesis, etc.). Why Join Us? Work with a passionate, innovative team driving the next wave of industry solutions. Opportunities for professional growth and continuous learning. Competitive compensation, comprehensive benefits, and a flexible work environment. In return for your dedication, we offer: Salary $125,000 Health benefits Top-tier health benefits include; medical, dental, vision, LTD, and life insurance Mental health benefits Additional benefits Unlimited PTO In the WeWork office In the heart of the financial district in downtown Kansas City Open space concept & team-focused atmosphere Company-wide celebrations Team bonding events and happy hours CHECK US OUT: OUR APP : Welcome to MFour! Our Story: Founded in 2011, MFour is a fast-growing data and analytics technology company “democratizing” consumer insights. Named a 2023 “top 25 L.A. tech companies to watch,” by Built In, we’ve created a SaaS ecosystem, giving businesses access to united shopper behavior + opinion data that uncovers the whos, whats, whys, whens, and wheres behind consumers in ways never before possible. We help 25% of the Fortune 100 plus organizations of all sizes make product, brand, and advertising decisions. Our Product: Using the nation’s most downloaded, highest-rated, and only Apple-approved app & web data collection and survey app (Surveys On The Go®), MFour is the only consumer market research company that guarantees accurate, validated responses for every survey, giving businesses the data and insights they need to make impactful decisions. From validated opinion and behavior trackers to advertising exposure measurement, MFour is trusted by hundreds of leading companies. Our Goal: To eliminate the unknown (data blind spots) within market research in a way that recognizes a consumer’s right to privacy, so marketers can make better decisions about new and existing products, advertising choices, and competitive positioning. MFour is trusted by hundreds of leading brands, including Google, Microsoft, Samsung, Walmart, Disney, Spotify, and Lowe’s. , Our Core Values: Simplicity, Humility, Quality, Consistency, and Innovation.