Job description

OKX will be prioritising applicants who have a current right to work in Singapore, and do not require OKX's sponsorship of a visa.

Who We Are

At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom.

OKX is a leading crypto exchange, and the developer of OKX Wallet, giving millions access to crypto trading and decentralized crypto applications (dApps). OKX is also a trusted brand by hundreds of large institutions seeking access to crypto markets. We are safe and reliable, backed by our Proof of Reserves.

Across our multiple offices globally, we are united by our core principles: We Before Me, Do the Right Thing, and Get Things Done. These shared values drive our culture, shape our processes, and foster a friendly, rewarding, and diverse environment for every OK-er.

OKX is part of OKG, a group that brings the value of Blockchain to users around the world, through our leading products OKX, OKX Wallet, OKLink and more.

About The Opportunity

We are looking for an execution-focused Product Manager who has built and improved conversational AI products in production — and has business results to prove it. A strong plus is hands-on experience with agent evaluation harnesses or internal agent platform product design: you've defined the systems that test, score, and operate agents at scale, not just shipped the agents themselves.

You work in logs and specs, not just decks. You know what a bad retrieval chunk looks like, you've personally written labeling guidelines, and you can point to a quarter where your work moved resolution rate by double digits.

What We Are Looking For

You have hands-on experience building and operating conversational AI products in production — not just shipping agents, but owning the quality systems, data pipelines, and operational platforms that keep them reliable at scale. Ideal candidates will have background in one or more of the following areas:

Knowledge Base & Data Quality — knowledge base architecture, retrieval quality tuning, content governance, labeling pipelines, annotation guidelines, training data impact tracking, and dataset freshness management
Agent Evaluation & Quality Assurance — evaluation harness design, test case schemas, automated scoring rubrics (correctness, groundedness, tool-use accuracy), LLM-as-judge evaluation, regression testing for non-deterministic systems, and feedback-driven improvement loops
Chatbot Operations & Dialogue Design — SOP-to-agent-flow translation, edge case handling, escalation path design, log-based failure triage, and metrics ownership (resolution rate, fallback rate, per-intent accuracy, CSAT)
Agent Runtime & Observability Platforms — agent runtime product requirements, tool permission models, task configuration interfaces, developer-facing observability dashboards, failure alerting logic, and debugging workflows
Human-in-the-Loop Workflows — low-confidence case routing, reviewer task interface design, correction data capture, and feedback loop integration back into training or knowledge pipelines

Chatbot & Knowledge Base (Core)

Built or rebuilt a knowledge base — defined structure, wrote/reviewed content, fixed retrieval quality, saw metrics improve
Designed SOPs that became agent flows — mapped real business processes, handled edge cases, shipped as working dialogue flows
Owned a labeling pipeline — wrote annotation guidelines, QA'd batches, tracked whether labeled data moved production metrics
Moved a metric that mattered — resolution rate, fallback rate, CSAT — and can explain exactly what changed

Agent Harness & Platform Product (Strong Plus)

Designed an agent evaluation harness: defined test case schemas, scoring rubrics, and spec'd automated evaluation pipelines with engineering
Product-designed an internal agent platform: defined requirements for agent runtime — tool permission models, task configuration interfaces, developer-facing observability dashboards, and failure debugging workflows; owned the roadmap and shipped iteratively
Closed the eval-to-improvement loop: used harness output to prioritize knowledge fixes, prompt revisions, or flow changes — not just reported scores but drove action from them
Designed human-in-the-loop review workflows: low-confidence case routing, reviewer task interfaces, correction data capture and feedback loop back into training or knowledge pipelines

What You’ll Be Doing

Chatbot Operations

Knowledge base ownership: structure, content quality, retrieval coverage, freshness governance
SOP & dialogue flow design: business process → agent flow → edge case handling → escalation paths
Labeling pipeline: annotation specs, annotator QA, training batch impact tracking
Daily quality work: log review, failure triage, weekly knowledge/flow update cadence
Metrics ownership: resolution rate, fallback rate, per-intent accuracy, CSAT

Agent Harness & Platform Product

Define and maintain agent evaluation frameworks: test case design, automated scoring criteria, regression test coverage
Own the quality feedback loop: harness results → prioritized fixes → re-evaluation → production deployment
Partner with engineering to define product requirements for agent runtime: spec observability features, tool call monitoring interfaces, failure alerting logic, and developer-facing debugging tools — own the backlog, not the ops
Design human-in-the-loop workflows: case routing logic, reviewer interfaces, correction data capture
Track agent version performance over time; maintain eval dashboards that teams actually use

What We Look For In You

3–6 years PM experience; minimum 2 years as primary owner of a production chatbot or AI agent product
Quantified business results: can describe baseline metrics, what you did, and outcome in numbers
Hands-on knowledge base, labeling, and conversation analysis experience (not just oversight)
Familiar with at least one chatbot/agent platform (Coze, Dify, Dialogflow, or similar)
Mandarin Chinese fluency required; English proficiency required

Nice-To-Haves

Designed an agent eval harness: written test case specs, defined scoring rubrics (correctness, groundedness, tool-use accuracy), and spec'd the automated evaluation pipeline with engineering
Product-designed an internal agent platform: defined product requirements for agent runtime — tool permission models, task configuration interfaces, developer-facing observability and debugging workflows; owned roadmap and shipped iteratively
Experience with LLM-as-judge evaluation: has used model-based scoring in a harness and understands its blind spots
Familiar with agent observability tooling (LangSmith, Langfuse, or internal equivalents) — to define what the product needs to surface, not to operate them
Experience spec'ing regression testing for non-deterministic systems: knows how to define quality regression detection when LLM outputs vary
Has written product specs for human-in-the-loop workflows: low-confidence case routing, reviewer task interfaces, correction data capture and feedback loop design
Background in customer service, operations, or financial services domain

Perks & Benefits

Competitive total compensation package
L&D programs and education subsidy for employees' growth and development
Various team building programs and company events
Wellness and meal allowances
Comprehensive healthcare schemes for employees and dependants
More that we love to tell you along the process!

#LI-WWW
#LI-ONSITE

Notice:

All official OKX vacancies are published on this website. While roles may appear on selected third-party platforms from time to time, information on other sites may be inaccurate or outdated. If in doubt, please apply directly through our official careers website.

Information collected and processed as part of the recruitment process of any job application you choose to submit is subject to OKX's Candidate Privacy Notice.

Senior Product Manager, Conversational AI Chatbot & Agent Quality at okx | AI Job Board