Senior Engineering Manager, Search - Cognition System

Job description

WHO WE ARE Video is 90% of the world's data. Most of it is invisible to machines. TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound, and motion — and power production-scale AI workloads across media, entertainment, sports, security, and government. We have raised more than $210 million from NEA, Radical Ventures, Amazon, NVIDIA, Snowflake, Databricks, Index Ventures, NAVER Ventures, Korea Investment Partners, Quadrille Capital, Red Bull Ventures, and AI pioneers including Fei-Fei Li, Silvio Savarese, and Alexandr Wang. We are a global company, headquartered in San Francisco with offices in Seoul, New York, and London, and employees around the world. We believe the differences in our cultural, educational, and life experiences make our products stronger. Building technology that understands the world in all its complexity requires people who see it from every angle. We are looking for individuals who are driven by hard problems and want their work to matter. Come build it with us! ABOUT JOCKEY Jockey is TwelveLabs' unified agentic system that reasons across your videos and images. It combines a reasoning model with a memory layer that builds a knowledge store from your corpus. No context window holds a video archive. We work at a million hours of video. A single model forward pass can tell you about one file; it can't reason across a corpus, and no context window closes that gap. Jockey decomposes a query, retrieves, segments, and reasons across thousands of videos and images. Point it at an archive, ask for a highlight reel or the best viral moments, and it returns timestamped cuts you can use. Corpus-level understanding you can act on is the whole product. Built for agents, not just people. As AI agents increasingly become the primary consumers of video, we're building production-grade infrastructure that scales to millions of hours while delivering reliable, high-quality results for both human users and autonomous agents. We build on models we own. Marengo, our embedding model, resolves a query like "the moment we almost missed the flight" into real retrieval. Pegasus, our video-language model, returns structured, timestamped moments on a schema you define. We ship and improve both continuously, so Jockey's quality compounds with every release — no re-integration for customers. Few teams get to build an agent on a stack they control end to end. Deep expertise, one system, open culture. Foundation models, knowledge construction, search, and the agent harness all live in one org. Each team owns its domain and is expected to have deep expertise in it — but like a Formula 1 team, we optimize for the global system, not local parts. A model gain that doesn't expand what the agent can do isn't a gain. We trace a single algorithm change through to end-system behavior, and share work in progress weekly, not just finished results. Anyone can pull the context they need from any team. ABOUT THE TEAM This team owns Search at TwelveLabs end-to-end. From the core retrieval platform to the agentic search layer, we build the foundation for how both humans and AI agents discover and explore video. Full-stack search systems: TwelveLabs is defining the future of video search, powered by the world’s leading video embedding model (Marengo) and video-language model (Pegasus). We build the full search stack—from keyword and semantic retrieval to multimodal and agentic search—enabling fast, accurate, and scalable search experiences. Search for humans and agents: As AI agents increasingly become the primary consumers of search, we’re building production-grade search infrastructure and agentic search systems that scale to millions of hours of video while delivering reliable, high-quality results for both human users and autonomous agents. Global impact: Our team operates at the intersection of cutting-edge research and real-world products, turning breakthroughs into customer value on rapid iteration cycles and powering video understanding for customers around the globe. ABOUT THE ROLE As the Senior Engineering Manager, Search for our Cognition System, you will own and lead the team that builds search at TwelveLabs end to end, This includes both the core search & retrieval platform and the agentic search harness on top of it. Your goal is not to build a cool demo, but a production system that ingests millions of hours of video and serves queries at very high RPS. We're looking for a technical leader who understands: - The complexity and engineering rigor required to stand up a production-grade search system that scales to millions of hours across both ingestion and index creation. - The agentic search harness that reasons about user intent (human and agent) across millions of videos to surface what the user is truly asking for. You are also a people manager who leads both senior and junior engineers and scientists. You are not a day-to-day IC, but you read traces, review code, and debate architecture as a peer and you make sure the system works in production, not just in a demo. IN THIS ROLE, YOU WILL - Own the search system end-to-end (e.g., vector/ANN indexing, lexical retrieval, hybrid fusion, reranking, and temporal (segment-level) search), built on Marengo embeddings and Pegasus and scaled to millions of hours across both ingestion and index creation. - Design and build the agentic search harness that infers human and agent intent and orchestrates multi-turn, session-based (conversational), and parallel search sessions, including subagents that can be invoked by a primary agent. - Set and own the search-quality bar for both human- and agent-initiated search, and drive continuous improvement against them. - Own reliability and systems design for the search stack so the system stays dependable as both human and agent traffic grows. - Drive cross-functional alignment with Research/Training, Platform/Infra, and Product on retrieval strategy, API contracts, and system boundaries. - Build and lead the team: recruit strong engineers and scientists, set engineering patterns and standards across the search stack, and raise the bar through architecture review, code review, and technical mentorship. YOU MAY BE A GOOD FIT IF YOU HAVE - 7+ years building production ML or search systems, with deep experience in search, retrieval, or recommendation at scale. - Engineering leadership where you've built and led teams of engineers (and scientists), and you can attract, hire, and grow senior ICs. - Deep experience with information retrieval systems (e.g. embedding-based search, lexical search (BM25 / OpenSearch / Elasticsearch), hybrid retrieval, reranking, and score normalization) at production scale. - Strong system-design instincts across ML infrastructure, distributed systems, and developer-facing platforms. PREFERRED QUALIFICATIONS - Experience building agentic search systems - Experience with multimodal or video search/retrieval - Familiarity with knowledge graph construction, entity resolution, or graph-backed retrieval. - Good English communication skills, verbal and written. BENEFITS AND PERKS - 글로벌 B2B 고객과 함께 성장하는 Global Team - 자율성과 협업을 모두 갖춘 하이브리드 근무 - 전 직원에게 맥북 및 70만 원 상당 재택근무 장비 지원, 3년 주기로 최신 장비 교체 - 식사·교통비 등 자유롭게 사용할 수 있는 월 60만 원 한도 법인카드 제공 - 사무실 내 스낵바(간식, 커피, 신선식품 제공) - 연말 2주간 겨울방학 운영 - 연 1회 건강검진 지원 - 영어교육 프로그램 지원