Product Manager II, AI Evaluations

Job description

**WHAT IS BOX?** Box (NYSE:BOX) is the leader in Intelligent Content Management. Our platform enables organizations to fuel collaboration, manage the entire content lifecycle, secure critical content, and transform business workflows with enterprise AI. We help companies thrive in the new AI-first era of business. Founded in 2005, Box simplifies work for leading global organizations, including JLL, Morgan Stanley, and Nationwide. Box is headquartered in Redwood City, CA, with offices across the United States, Europe, and Asia. By joining Box, you will have the unique opportunity to continue driving our platform forward. Content powers how we work. It’s the billions of files and information flowing across teams, departments, and key business processes every single day: contracts, invoices, employee records, financials, product specs, marketing assets, and more. Our mission is to bring intelligence to the world of content management and empower our customers to completely transform workflows across their organizations. With the combination of AI and enterprise content, the opportunity has never been greater to transform how the world works together and at Box you will be on the front lines of this massive shift. **WHY BOX NEEDS YOU** We're looking for a high-agency, deeply AI-fluent Product Manager to own AI Evaluations — a highly cross-functional, highly visible role at the front lines of how Box assesses, launches, and manages the AI models that power our product. You'll run rigorous evaluations of frontier models, own the decision of which models power Box Agents, shape how the world's leading model providers understand enterprise performance, and own the full lifecycle of every model we support. This is a hands-on role for someone who deeply knows the AI landscape and can do their own work: statistically literate, comfortable in code, and able to build the evals, run the analyses, and prototype the pipelines themselves rather than spec them out for someone else. It sits at the intersection of technical depth, go-to-market execution, and external partnerships — connective tissue across Product, Marketing, Partnerships, and Engineering — and it's a fit for someone who thrives in ambiguity, moves fast on tight timelines, and brings order to messy, fast-evolving problem spaces. **WHAT YOU'LL DO** - Run rigorous evaluations across frontier models (OpenAI, Anthropic, Google/Gemini) and continuously mature the stack, methodology, and process behind them. - Define what makes a high-quality evaluation dataset and robust, cost-appropriate graders, and translate what end users care about into measurable criteria. - Drive synthetic data generation and data-generation pipelines to ensure they are high-quality, reliable, and scalable. - Own online evaluations, turning real production usage into rich insight on how our product is actually used. - Lead each model launch end to end, managing the process, the stakeholders, and the timeline. - Own the external narrative for each model launch — partnering with Product Marketing on blog, social, and PR content, and creating video demos yourself. - Build and manage deep relationships with model providers and AI companies — securing early access to frontier models, delivering technical deep-dives back to providers, and identifying collaboration opportunities on evaluations and benchmarking. - Identify, pitch, and structure opportunities to collaborate with providers and other AI companies specifically on evaluations and benchmarking. - Run evaluations to decide whether a model is production-ready, and own the decision of which models power agentic products at Box. - Assess the compliance posture of production models, partner on model pricing decisions, and define how and when we deprecate models. **WHO YOU ARE** We are an AI-first company. This means you approach your work with a growth mindset and find ways to leverage AI to help make faster, smarter decisions that will 10X your impact at Box. - You have a minimum of 2 years of product management experience. - You know the AI landscape deeply, follow frontier models closely, and can hold your own in a technical conversation with any model provider. - You're technical and hands-on: statistically literate and comfortable enough in code (Python, SQL, notebooks) to build your own evals, run your own analyses, and prototype your own data pipelines. - You've driven cross-functional launches across Product Marketing, Communications, Developer Relations, and Engineering. - You're an excellent written and verbal communicator who can represent Box externally, presenting complex, technical topics simply to senior stakeholders. - You operate with extremely high agency, taking ambiguous, undefined problems and driving them to resolution on tight timelines. - You're fluent across many stakeholders—from Product Marketing and Sales to Security, Legal, and Compliance—adapting your communication to each audience. - You're a natural disambiguator who makes messy, fast-moving AI problems simple and clear. Preferred skills - Direct experience with AI/ML products, model evaluation, or benchmarking. - Experience managing external partnerships with vendors or providers. Box lives its values, with community and in-person collaboration being a core part of our culture. Boxers are expected to work from their assigned office a minimum of 3 days per week.Your Recruiter will share more about how we work and company culture during the hiring process. At Box, we believe unique and diverse experiences benefit our culture, our products, our customers, our company, and our world. We aim to recruit a passionate, high-performing workforce that reflects the world we live in.If you are head-over-heels about this role but unsure if you meet all the requirements, we encourage you to apply! **EQUAL OPPORTUNITY** We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation. Box strives to respect the dignity and ‎‎independence of people with disabilities and is committed to giving them the same ‎‎opportunity to succeed as all other employees. Inclusiveness is core to our culture at Box, and we strive to ensure you get the most from your interview experience. Box makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please complete [this form](https://forms.gle/5W2F6Pe3PsZNsKYp9). Reasonable accommodations may include scheduling adjustments, document dictation and beyond. Notice to applicants in Los Angeles: Box, Inc and its related branches will consider for employment, qualified applicants with criminal histories in a manner consistent with the Los Angeles Fair Chair Ordinance. The Fair Chance Ordinance is provided [here](https://bca.lacity.org/fair-chance). Notice to applicants in San Francisco: Box, Inc and its related branches will consider for employment, qualified applicants with criminal histories in a manner consistent with the San Francisco Fair Chair Ordinance. The Fair Chance Ordinance is provided [here](https://sfgov.org/olse/sites/default/files/FCO%20poster2020.pdf). For details on how we protect your information when you apply, please see our [Personnel Privacy Notice.](https://cloud.app.box.com/v/BoxPersonnelPrivacyNotice) If you are a California-resident, please read our California Applicant & Candidate Privacy Notice [here](https://cloud.box.com/s/uf7c4w970ds5gpt1dufdk5bw49oym849). *Box is committed to fair and equitable compensation practices. Actual base salary (or OTE if commissionable role) is dependent upon factors such as: knowledge, skill level, experience, and work location. This role is also eligible for equity and benefits. *For more information**, check out our [benefits and perks](https://cloud.app.box.com/file/680236394456?s=3u38zhdv4gac9h0j7lmumm7arbe14yqa).** **In accordance with OFCCP compliance, here is the [Pay Transparency Provision](https://cloud.box.com/s/sv4dcjqb4tfyfc456rltlucuedth4aln).** Redwood City Pay Range $160,500—$200,500 USD