Product Manager, Model Gateway

Job description

ABOUT MERCOR Mercor's mission is to organize human intelligence to power the AI economy. We're a leading AI data company, building the layer between human expertise and frontier models. Millions of domain experts on the platform are paid over $4 million per day to train frontier AI models. Mercor's APEX benchmark family measures AI's real-world impact on professional work. Mercor Enterprise brings this same infrastructure to Fortune 500 companies: helping companies capture how their best people actually work, translating that expertise directly back into agents. Mercor is creating a new category of work where expertise powers AI advancement. Achieving this requires an ambitious, fast-paced and deeply committed team. You’ll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society. Mercor is a profitable Series C company valued at $10 billion. We work in-person five days a week in our San Francisco, NYC, or London offices. ABOUT THE ROLE In this Platform Product Manager, you'll own Mercor's LLM gateway, the shared layer every product surface depends on to reach frontier models reliably, affordably, and within policy. This is a high-leverage platform role: the decisions you make about priority queueing, cost attribution, rate limits, and model usage policy ripple across every team that builds on top of us. You'll own this product area with significant autonomy, working directly with an infrastructure team that owns the underlying shared services and with leadership to turn ambiguous, cross-cutting tradeoffs into durable systems and clear policy. Because the gateway sits between every product team and the models they rely on, your users are internal: the PMs and engineers building on the platform, plus the infra and finance partners who care about reliability and spend. Success looks like product teams moving fast without stepping on each other, model costs that are transparent and attributable down to the team and workload, and usage policies that scale as we add surfaces and models. WHAT YOU’LL DO - Own the LLM gateway end-to-end: Set the roadmap and policy for the shared layer that routes every product surface to the models it depends on, from priority queueing to cost attribution to model access, and own its lifecycle from conception through execution. - Design priority and fairness: Define the queueing and rate-limiting logic that decides whose workload runs when, balancing revenue blocking traffic against background jobs, and making the tradeoffs explicit and consistent. - Make cost visible and attributable: Work backwards from business outcomes to make product decisions that impact business health and growth. - Negotiate rate limits and usage policy: Turn contention over shared capacity into clear, defensible rules people can plan around. - Partner with infrastructure: Work hand-in-hand with the central infra team translating product needs into platform requirements and platform constraints back into product tradeoffs. - Work backwards from the business: Connect gateway decisions to business health, cost, reliability, and velocity, and use that to prioritize. WHAT WE’RE LOOKING FOR - Experience: 5+ years as a product manager, ideally on platform, infrastructure, or developer-facing products; prior background in SWE, ML, or DS strongly preferred. - Systems thinking: You reason fluently about queues, rate limits, quotas, and the tradeoffs between latency, throughput, and cost, and you can turn those tradeoffs into policy. - High ownership: You take full accountability for outcomes not just outputs. - Independence: Able to self-direct in ambiguous, cross-cutting contexts. - Technical fluency: Comfortable reading usage data, writing queries, and building scrappy dashboards or making small PRs to validate direction and unblock engineering. - Stakeholder alignment: You can bring many product teams around shared constraints and turn contention over capacity and spend into rules people accept and plan around. - Cost & metrics orientation: Comfortable with cost models, unit economics, and attribution, and equally able to push forward a longer-term platform vision. Benefits - Bi-annual performance bonus structure - Generous equity grant vested over 4 years - Up to $15k Relocation bonus - $10K housing bonus (if you live within 0.5 miles of our office) - $1.5K monthly stipend for meals - Free Equinox membership - $200 monthly laundry reimbursement - $200 monthly personal wellness reimbursement - Health, Dental, Vision insurance