AI/ML Product Manager

Job description

The Role Disco is a commerce media company run like an AI company. We brought our bidding and ad-serving stack in house. We built an agentic development pipeline where every pull request, every message, and every review flows through our own harness. Our CEO ships models into production through AI agents. That was the first act. The second act is the part nobody owns yet: the layer that ties all of it together. The model orchestration, the harnesses, the eval loops, the agent graphs, and the routing logic that decides which model answers which problem. Today that layer is a collection of sub-agents pointed at tasks. We want it rethought, productized, and owned. We're hiring an AI/ML Product Manager to own Disco's AI platform end to end. You're the person who makes sure we're model-agnostic and harness-agnostic, so when a vendor jacks up prices or a better model ships tomorrow, we swap in a day instead of re-platforming. You own the eval loops that tell us what's working, the harnesses that make AI safe to push to production, and the agentic pipeline that lets a team of 25 move like a team of 200. This is a product role with real technical depth. You won't just write specs and hand them to engineers. You'll understand the orchestration layer well enough to redesign it, evaluate models and harnesses against each other, and dive into the data and the code to see where tokens are being burned, where an eval is lying, or why a workflow broke. You're the closest thing Disco has to an owner of the AI operating system. What You'll Own Model Orchestration & Routing Own the layer that routes every task to the right model: reasoning to the frontier models, volume to the cheap ones, image and creative work to the models that are actually good at it. This is bespoke routing, not one model for everything. Keep Disco model-agnostic and harness-agnostic. When a vendor raises prices 10x or a better model drops, we swap without being beholden to any single LLM contract. Track the total cost of every model call. Own the LLM budget like it's your P&L, because it's about to become one of the largest OpEx lines in the company. Harnesses & Eval Loops Own the harnesses that wrap AI so it can ship to production safely: what gets reviewed, what gets verified, what a non-ML user can and cannot push to traffic. Build and own the eval loops that tell us what's actually working. A model, an agent, a workflow, a cron — if it runs on AI, there's an eval that says whether it's good, and you own that eval. Design eval graphs that catch regressions before they cost money, and the guardrails that keep an enthusiastic experiment from becoming a production incident. Agentic Pipeline & Automation Own the agentic development pipeline: the AWS Bedrock harness, the automated PR flow, the QA gates. Make it faster, cheaper, and more reliable. Productize the internal AI tooling. Cron generation, workflow automation, memory, context — these are scripts today. They should be products with owners, versions, and evals. Attack the silent cost centers. Crons and background jobs that burn LLM tokens are the first target: turn expensive agent loops into cheap, deterministic code where it belongs. AI Enablement Across the Team Make every team agentic. When a non-technical teammate wants to build, evaluate, or automate something with AI, your harnesses and loops are what make that safe. Own the knowledge of which models are best at what, and keep it current. The model landscape changes monthly; the team shouldn't have to track it. Set the standard for where AI output is trusted and where it gets verified before it ships. Who You Are You've shipped real AI products or platforms into production, and you can point to the eval loops, harnesses, or orchestration you built around them. You think in systems: routing, evaluation, feedback loops, cost. You don't just add an AI feature, you build the harness that makes it safe and the eval that proves it works. You're technically credible without being a full-time engineer. You can read code, query data, and reason about model tradeoffs, latency, and cost. You don't need to author the orchestration, but you need to be able to redesign it. You're model-agnostic by instinct. You care about outcomes, not loyalty to a vendor. When a model is bad at a task, you say so and swap it. You have strong product judgment and a bias toward shipping. You can take a fuzzy mandate like 'own the AI platform' and turn it into a roadmap, then execute it. You're comfortable with a level of ambiguity most PMs never see. There's no playbook for this role and no platform team behind you. You'll build the thing you want to use. You communicate clearly across a small, fast team. You'll explain a model routing decision to the CEO and a harness change to an engineer in the same week. You can dive into the data and the code to understand what's actually happening for any given workflow, eval, or cost anomaly. No 'that's someone else's problem.' Why Join Disco At Disco, you’ll have the opportunity to work at the intersection of data, technology, and commerce, and have a direct impact on how our business and the broader ecosystem evolves. Unique Market Position : We're not competing on volume, we're competing on intelligence and yield optimization Data Advantage : 160M+ proprietary profiles give us differentiated insights and capabilities Innovation Runway : Ground-floor opportunity to shape our next generation of ad products Growth Stage: Opportunity to scale a proven business model into new dimensions Strategic Impact : Your work directly impacts both advertiser success and publisher economics Perks & Benefits We believe in taking care of our team, both inside and outside of work. When you join Disco, you’ll get more than great perks and benefits. You’ll be surrounded by great people, do meaningful work, and have the opportunity to build something that truly matters. Flexible PTO + 12 paid holidays + 3 company-wide DisConnect Days $250/month lifestyle stipend for the things that support your wellbeing $100/month Disco credi t to shop from brands in our network $750/month childcare stipend for family care $132/month commuter benefit $500 home office setup reimbursement $20 daily Grubhub credit when working from our SF or NYC offices 401(k) to help you invest in your future Comprehensive insurance (medical, dental, vision & life) This is a hybrid role based out of our NYC office, with an expectation of 2–3 days in the office each week. Compensation for this role is $190K–$220K OTE, plus meaningful equity. About Disco Disco powers a next-generation commerce media network that connects over 1,000 eCommerce advertisers to consumers on some of the world's most recognizable retail platforms, including Mindbody, Gopuff, Fabletics, and Knitwell Group (Chico's, Talbots, Lane Bryant). When a consumer books a yoga class or buys a new pair of jeans, Disco's recommendation engine surfaces relevant, brand-safe offers from other consumer brands, turning everyday transactions into meaningful moments of discovery. Since launching in 2021, we've raised $26M from leading investors including Felicis Ventures, Bessemer Venture Partners, Shopify, Sugar Capital, RiverPark Ventures, and Indicator Ventures. Today, Disco's network spans 1,000+ advertisers and publishers, driving over $1B in GMV across 150M+ permissioned U.S. shopper profiles.