Senior Product Manager, AI Experimentation Platform

Job description

Adventure is our Culture. Join a team that celebrates a lifestyle as bold as the terrain we love. At Backcountry, we are rooted in adventure, recognition, and wellbeing on and off the mountain. We spotlight employee stories, celebrate milestones, and offer exclusive outdoor perks. Whether you are at HQ, in a retail store, or remote, you will be part of a team that thrives on energy, exploration, and connection. **Reports to:** Vice President of Product and UX **Location:** Cottonwood Heights, Utah (Hybrid - 3x/week in-office) ### About the Role Backcountry is moving from manual, low-volume A/B testing to an always-learning agentic platform that designs, launches, monitors, and optimizes experiments across our family of ecommerce sites. As **Senior Product Manager, AI Experimentation Platform**, you will own this platform from the ground up — setting the roadmap, defining where AI agents act autonomously and where humans stay in the loop, and driving measurable gains in conversion and revenue. At six to twelve months, success looks like a working platform that runs trusted experiments across multiple sites, a significant increase in testing velocity without sacrificing statistical rigor, and revenue and conversion improvements you can trace directly to platform-run experiments. This is a lean team. You will own a lot, move fast, and make decisions with full end-to-end responsibility. ### What You'll Do - Own the vision, roadmap, and prioritization for the agentic experimentation platform, deciding what gets built first, what waits, and what to buy versus build. - Define how AI agents participate across the experiment lifecycle — hypothesis generation, variant creation, test setup and QA, live monitoring, and ship-or-rollback decisions. - Set guardrails that keep autonomous testing safe: statistical standards, sample ratio mismatch and anomaly checks, revenue and margin thresholds, approval workflows, and automatic rollback. - Build adaptive optimization into the platform using multi-armed and contextual bandits so traffic shifts toward winning experiences while tests are still running. - Partner with engineering and data science on core infrastructure: assignment and bucketing, feature flags, event tracking, the metrics layer, and the stats engine built on GA4 and BigQuery. - Define and instrument platform success metrics — test velocity, win rate, revenue impact, and result trustworthiness — from day one. - Make experimentation self-serve for merchandising, marketing, UX, and engineering teams so they can launch and read out tests without a specialist. - Create standards and playbooks for metric definitions, test design, and readouts so results are consistent, comparable, and trusted across every site. - Partner across engineering, design, analytics, merchandising, and marketing to turn business goals into a shipped platform and a culture of testing. ### Required Qualifications - 5+ years of product management experience, including ownership of a platform, data, or internal tooling product. - Hands-on experience running or building A/B testing and experimentation programs at scale in ecommerce or other high-traffic digital products. - Strong statistical grounding in experiment design: power and sample size calculation, significance, peeking, multiple comparisons, and guardrail metrics. - Hands-on experience building with AI and LLMs, including agentic workflows, prompt design, tool integrations, and evaluating AI output quality. - Working knowledge of experimentation infrastructure — feature flagging, randomization, event tracking, and warehouse-native metrics. - Strong analytical skills with hands-on use of SQL, GA4, BigQuery, Looker, or similar tools. - Demonstrated impact on conversion, revenue, or testing speed with measurable results. - Excellent communication skills and the ability to influence engineering, data, and business stakeholders. ### Preferred Qualifications - Experience with experimentation platforms such as Optimizely, Statsig, Eppo, GrowthBook, LaunchDarkly, or an in-house equivalent. - Experience with adaptive experimentation — multi-armed or contextual bandits, Bayesian methods, or variance reduction techniques like CUPED. - Experience testing AI or ML-driven experiences such as search, recommendations, or personalization. - Experience running experimentation across multiple brands or storefronts on a shared platform. - Background in outdoor, specialty, or multi-brand ecommerce. ### Why Join The people who do best here are builders. They take ownership, move fast, and want to see the direct impact of their work. - **Cross-Functional Impact:** Build the system that determines how every one of our sites improves, with results that show up directly in revenue and conversion across the portfolio. - **AI-First Skill Building:** Work on a team that treats AI as a real upgrade to quality and speed, and build a product where AI agents do real work in production — from hypothesis generation to autonomous optimization. - **End-to-End Ownership:** Define what an agentic experimentation platform looks like here, from the first architecture decisions to the first autonomous test, with full ownership of the roadmap and outcomes. - **Competitive Benefits:** Paid time off policies, 401(k) match, medical/dental/vision and a variety of supplemental policies, and employee discounts across our portfolio of brands. ### 1. **Recruiter Screen** - A 30-minute conversation with our recruiting team to align on the role, your background, and what you are looking for. 2. **Take-Home Challenge** - A short AI or product building exercise so we can see how you approach problems and execution. 3. **Hiring Manager Interview** - A deep dive with the Vice President of Product and UX on your experience, product mindset, and approach to experimentation. 4. **Technical and Case Discussion** - A working session with engineering and analytics on experiment design, statistics, and platform architecture. 5. **In-Person Interview** - We'd like to meet you at our Cottonwood Heights office. Details and logistics will be arranged with your recruiter. 6. **Reference Checks** - Conducted in parallel with the final stages where possible. 7. **Offer** - We move quickly for the right candidate. *Interview process is subject to change. Any updates will be communicated promptly and clearly.* 1. **Recruiter Screen** - A 30-minute conversation with our recruiting team to align on the role, your background, and what you are looking for. 2. **Take-Home Challenge** - A short AI or product building exercise so we can see how you approach problems and execution. 3. **Hiring Manager Interview** - A deep dive with the Vice President of Product and UX on your experience, product mindset, and approach to experimentation. 4. **Technical and Case Discussion** - A working session with engineering and analytics on experiment design, statistics, and platform architecture. 5. **In-Person Interview** - We'd like to meet you at our Cottonwood Heights office. Details and logistics will be arranged with your recruiter. 6. **Reference Checks** - Conducted in parallel with the final stages where possible. 7. **Offer** - We move quickly for the right candidate. *Interview process is subject to change. Any updates will be communicated promptly and clearly.* ### Please Note - Part of our interview process is a **mandatory in-person interview** with someone on our team prior to an offer. Candidates that are unwilling or unable to meet for an in-person interview will be removed from consideration immediately. - Due to a high volume of fraudulent applications, **you must share a valid LinkedIn profile URL in** **the application questions below to be considered**. If you do not have a LinkedIn profile, you must provide a credible reason in that field and supply alternative evidence of your professional background to verify your identity.