Prompt Engineer, Multi-Agent Systems

Job description

About You: You want to work on a system of agents that work together. Some of them talk to customers, and many of them never say a word to anyone. They interpret, decide, and pass work to each other, and you understand that the quality of the whole system depends on every link in that chain holding up in production. Making that behave reliably, at scale, with real customers on the other end, is the problem you want to own. You treat prompts as engineering artifacts, not text you tweak until a demo works. If you've ever shipped a prompt change that fixed one case and quietly broke nine others, you know why the evaluation layer matters as much as the prompt itself. And if you get a small thrill from a regression suite going green after a nasty bug, you'll feel at home here. You'll also recognize how we work. We measure what we ship: changes come with evidence behind them, a dataset, a score, a clear before and after. We ship weekly, so ideas get tested against real production behavior quickly rather than studied at length. Craft matters, and so does rigor — getting an agent to sound right is satisfying, but proving it still sounds right across thousands of conversations is the actual work. The surface area grows every quarter with new agents, new channels, and new products. You'll own a meaningful slice of it from day one. You Will: Design prompts for agents that work together. Define what each agent knows, what it's allowed to decide, what it hands off, and what it must never do, including the contracts between agents that keep the system coherent end to end. Build the evaluation layer. Every prompt you ship comes with a dataset and a regression suite. You'll curate test cases from real production behavior, write evaluator prompts, define scoring rubrics that actually track business outcomes, and catch regressions before customers do. Make the system observable. Tracing, prompt versioning, datasets, experiments, scoring. You'll instrument what isn't yet instrumented and use the data to find where agents are actually struggling, which is often somewhere other than where we'd guess. Debug production behavior. When something goes wrong in a live interaction, you'll pull the trace, isolate the root cause, ship the fix, and add the case to the eval set so it never returns. Work across different models. We use more than one model, because different jobs call for different tradeoffs in reasoning, speed, and cost. Each one follows instructions a little differently, so a prompt that works well on one may need real rework on another. You'll help choose which model fits which job, and keep agent behavior stable when we switch. Work with dynamic context. Our prompts are assembled at runtime from live data. You'll design the logic that decides what an agent sees and when, and you'll find that many "prompt problems" turn out to be context problems. Partner across the company. Product, engineering, sales, and support. You'll turn a vague "this interaction felt off" into a reproducible test case, a fix, and a metric that proves it's fixed. You Have: Hands-on experience building with LLMs in production, on systems that real users depend on day to day. Real evaluation experience: building datasets, writing evaluator prompts, defining rubrics, running experiments, interpreting results, and acting on them. Fluency with LLM observability and evaluation tooling: tracing, prompt versioning, datasets, experiment runs. The specific tool matters less to us than the habits. Experience with multi-step or multi-agent architectures: orchestration, tool and function calling, structured outputs, handoffs between agents. Experience designing or debugging voice agents, including latency budgets, turn-taking, and the constraints of real-time speech. Voice is a significant part of what we build. Comfort with prompt templating and conditional logic, and with treating prompts as versioned, reviewable artifacts. Systems thinking. You can hold a chain of agents in your head and reason about where a failure originated. Strong writing. Every word an agent says to a customer is a word you wrote. Nice to have: Experience working with open-weight models and their behavioral differences from frontier models. Experience in a regulated or compliance-sensitive domain. Background in real estate, sales tech, or high-volume outbound. Our Benefits: Competitive Compensation: Attractive salary and benefits package. Flexible Work Environment: Fully remote work with flexible hours to promote work-life balance. Professional Growth: Opportunities for career advancement and professional development. Health & Wellness: Comprehensive health and vision insurance plans. Paid Time Off: Generous PTO and paid holidays to recharge and relax. Collaborative Culture: A supportive team environment that values innovation and collaboration. Equity Options: Opportunity to own a part of Fello and share in our success. Cutting-Edge Projects: Work on innovative products that leverage AI and advanced technologies. About Fello: Fello is a profitable, hyper-growth, VC-backed B2B SaaS company building the agentic real estate platform of the future. Our platform combines data intelligence, marketing automation, and conversational AI to help real estate teams engage smarter and scale faster. At the center of it is Felix, our AI teammate, which takes real work off a team's plate — running follow-up across calls, texts, and emails today, with more of the team's day-to-day work coming next. We're not another tool in the stack. We're the system that powers it — learning, adapting, and executing in real time so teams can focus on growth, not guesswork. If you're excited about building an AI product that's changing how an entire industry works, you'll fit right in.