Senior AI Engineer, Tools & Agents

Job description

WHO WE ARE Video is 90% of the world's data. Most of it is invisible to machines. TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound, and motion — and power production-scale AI workloads across media, entertainment, sports, security, and government. We have raised more than $210 million from NEA, Radical Ventures, Amazon, NVIDIA, Snowflake, Databricks, Index Ventures, NAVER Ventures, Korea Investment Partners, Quadrille Capital, Red Bull Ventures, and AI pioneers including Fei-Fei Li, Silvio Savarese, and Alexandr Wang. We are a global company, headquartered in San Francisco with offices in Seoul, New York, and London, and employees around the world. We believe the differences in our cultural, educational, and life experiences make our products stronger. Building technology that understands the world in all its complexity requires people who see it from every angle. We are looking for individuals who are driven by hard problems and want their work to matter. Come build it with us! ABOUT JOCKEY Jockey is TwelveLabs' unified agentic system that reasons across your videos and images. It combines a reasoning model with a memory layer that builds a knowledge store from your corpus. No context window holds a video archive. We work at a million hours of video. A single model forward pass can tell you about one file; it can't reason across a corpus, and no context window closes that gap. Jockey decomposes a query, retrieves, segments, and reasons across thousands of videos and images. Point it at an archive, ask for a highlight reel or the best viral moments, and it returns timestamped cuts you can use. Corpus-level understanding you can act on is the whole product. Built for agents, not just people. As AI agents increasingly become the primary consumers of video, we're building production-grade infrastructure that scales to millions of hours while delivering reliable, high-quality results for both human users and autonomous agents. We build on models we own. Marengo, our embedding model, resolves a query like "the moment we almost missed the flight" into real retrieval. Pegasus, our video-language model, returns structured, timestamped moments on a schema you define. We ship and improve both continuously, so Jockey's quality compounds with every release — no re-integration for customers. Few teams get to build an agent on a stack they control end to end. Deep expertise, one system, open culture. Foundation models, knowledge construction, search, and the agent harness all live in one org. Each team owns its domain and is expected to have deep expertise in it — but like a Formula 1 team, we optimize for the global system, not local parts. A model gain that doesn't expand what the agent can do isn't a gain. We trace a single algorithm change through to end-system behavior, and share work in progress weekly, not just finished results. Anyone can pull the context they need from any team. ABOUT THE ROLE We're hiring a senior AI engineer to own the integration layer that makes TwelveLabs' video AI accessible to the world. Jockey is TwelveLabs' multimodal agent for video and image understanding. It's built on a knowledge store of ingested content, retrieval primitives like search, and an agent layer that orchestrates those primitives: planning, calling them, and reasoning over results to return grounded, cited answers. The engineer in this role owns the surfaces that make this system accessible to external developers and AI agents. Learn more about Jockey here https://www.twelvelabs.io/jockey. The engineer in this role decides how AI agents and developers connect to these capabilities, and you'll build the agent harness and the supporting infrastructure that makes that connection trustworthy at enterprise scale. The center of gravity is our MCP server. You'll own it end-to-end: how agents discover and invoke our capabilities, what the tool interfaces look like, how failure modes are handled, and how the surface evolves as the agentic ecosystem does. Everything else, auth, metering, rate limiting, is the substrate that makes that surface something an enterprise customer will trust. Location: San Francisco. Onsite or hybrid. No fully remote option. IN THIS ROLE, YOU WILL OWN - The agent-facing integration surface. TwelveLabs' MCP server and agent-to-agent interfaces are yours to design and operate. You'll decide how AI agents discover, connect to, and build on Jockey's retrieval and reasoning capabilities, and you'll be the person who knows what breaks and why. - The auth and access substrate. OAuth, RBAC, and multi-tenant isolation for enterprise customers. This is the work that turns a powerful capability into something a large organization will put in production. You'll design it from scratch rather than configure someone else's framework. - The enterprise-readiness layer. Per-API-key usage tracking, rate limiting grounded in real system constraints, and the reliability work that makes the platform trustworthy under real load. You'll define how the platform behaves under pressure before customers find out the hard way. - The research partnership. You'll work directly with our research team to turn frontier video and image understanding capabilities into durable product surfaces. The loop from research to what customers can build is short, and you're a key part of closing it. YOU MAY BE A GOOD FIT IF YOU HAVE - You've shipped real agentic or LLM-powered systems. Not demos, but production systems with real users and real failure modes you had to debug and fix. You can talk about what broke, why, and what you changed. - You understand how retrieval, reasoning, and agent orchestration fit together as a system. Jockey's architecture sits at that intersection, and the integration surfaces you build need to reflect how that system actually behaves under real conditions. - You've owned a developer-facing or external API end-to-end, with real decisions about contracts, versioning, and the experience of building on top of your surface. - You've built authentication and authorization in production, OAuth, OIDC, RBAC, or multi-tenant isolation, as a primary owner and not a consumer of someone else's framework. - You've thought seriously about enterprise-readiness: what it takes to go from a capability that works to a platform that a large customer will trust with their data and workflows. - You bring versatility across the full platform surface. We're not looking for a specialist in one of these areas. We're looking for someone who can own all of them and make good tradeoffs across them. - You've operated in a fast-moving environment before, a startup, a hypergrowth company, or an AI-first team where you shipped under ambiguity without heavy process support. PREFERRED QUALIFICATIONS - Experience with MCP, A2A protocols, or the broader agent ecosystem. You've contributed to or built on these surfaces and understand where they're headed. - Hypergrowth pedigree from a technically excellent startup. BENEFITS AND PERKS 🤝 An open and inclusive culture and work environment. 🚀 Work closely with a collaborative, mission-driven team on cutting-edge AI technology. 🏥 Full health, dental, and vision benefits ✈️ Extremely flexible PTO and parental leave policy. Office closed the week of Christmas and New Years. 🛂 VISA support where applicable