Prediction Researcher

Job description

## About Aaru Aaru builds simulations of human behavior. Each simulation contains a population of AI agents, each representing a person who could plausibly exist in the real world and capable of making decisions within a modeled environment. Companies and institutions use these simulations to test consequential choices before committing—from product launches and pricing decisions to strategic communications and policy changes. Building a useful simulation requires more than generating plausible text. Populations must represent real people and groups; predictions must be calibrated; simulations must remain coherent as conditions change; and the product must make the resulting evidence legible enough to support real decisions. We are a small, in-person team in New York. We work with urgency, high ownership, and intellectual honesty. We expect people to surface inconvenient evidence, change their minds quickly, and carry important work all the way to a result. ## About Prediction Research Prediction Research builds systems that estimate future or otherwise unknown outcomes from data. The team's primary object is the population-level outcome: given a population, a question, and the relevant context, what aggregate result should we expect, how uncertain should we be, and how should that estimate change when the conditions change? Some problems are best solved with structured statistical or machine-learning methods. Others may benefit from language models, retrieval, tools, explicit decomposition, simulated agents, or a combination of these approaches. The team's job is not to assume that the most complex method is best. It is to determine which information and method produce genuine predictive signal beyond strong, simpler baselines. Prediction Research is not prompt engineering and it is not a speculative forecasting exercise. It is empirical predictive science. A prediction of 60 percent should resolve near 60 percent under the conditions where it is made. Improvements must survive temporal holdouts, new populations, changing environments, and prospective outcomes. ## The role As a Prediction Researcher, you will own difficult, open questions about aggregate human behavior and future outcomes. You will formulate hypotheses, construct or curate datasets, build predictive methods, design evaluations, inspect failures, and communicate what the evidence supports—including when a result is null, unstable, or less useful than a simple baseline. Your work may combine structured data, statistical learning, probabilistic modeling, language models, retrieval, tool use, and explicit agent simulation. You will be expected to choose methods based on the problem and evidence rather than on novelty. A strong result is not merely a lower benchmark score; it is a predictive improvement that remains calibrated, survives honest holdouts, and matters for a real decision. You will work closely with Population Research, Evaluation Research, Simulation Engineering, Product Engineering, Data, and Deployment. Validated methods should become reproducible systems with clear limits, not remain isolated notebooks or research demos. ## What you will do - Own a high-value research question in forecasting, aggregate behavioral prediction, calibration, conditioning, subgroup decomposition, drift, or agentic prediction. - Turn ambiguous questions into falsifiable hypotheses, strong baselines, appropriate datasets, decisive experiments, and explicit criteria for success or stopping. - Build prediction methods from real-world records such as transactions, product usage, event histories, operational data, market data, surveys, customer data, and longitudinal outcomes. - Combine language models with structured data, retrieval, tools, quantitative models, and inference-time reasoning when the combination produces measurable value. - Develop estimates of population behavior and determine how those estimates vary with attributes, prior behavior, information exposure, environment, time, and intervention. - Design temporal holdouts and prospective tests that use only information available at the time a prediction would actually have been made. - Measure calibration, proper scoring, ranking quality, selective prediction, subgroup performance, robustness, and the decision cost of different errors. - Compare complex methods with strong alternatives, including historical rates, conventional statistical models, direct aggregate prediction, segment-level methods, and explicit population simulation. - Diagnose error by population, subgroup, time period, domain, data source, confidence level, and model component rather than relying only on an average metric. - Study when agent-based reasoning adds predictive signal and when it merely adds cost, variance, or persuasive explanations. - Build learning loops from resolved events and customer outcomes while protecting final evaluation sets from contamination. - Work with Population Research to determine whether a failed prediction originates in the representation of the population or in the prediction method. - Work with Evaluation Research to create measurements that are both diagnostic during development and credible enough for final claims. - Hand validated methods to Simulation Engineering with reproducible code, documented assumptions, known failure modes, and clear production requirements. - Write research notes and technical reports that make the question, method, data, uncertainty, result, and limitations understandable to others. - Communicate negative and inconclusive results plainly and redirect quickly when the evidence does not justify further investment. ## Representative research directions You might investigate questions such as: - Forecast a future business, market, policy, or behavioral outcome using only information that was available when the forecast would have been issued. - Predict demand, adoption, conversion, retention, purchasing behavior, message response, or another aggregate outcome from transaction and usage data. - Develop decomposition-based methods that predict a population marginal through meaningful subgroups without amplifying sparse-data error. - Estimate how an outcome should update after a price change, product launch, information event, policy change, or shift in the economic environment. - Model marginal drift over time as new information arrives and determine when a prior prediction should be revised, widened, or withdrawn. - Build an agentic forecasting system that retrieves evidence, decomposes a question, tests assumptions, compares hypotheses, and returns a calibrated estimate. - Compare direct prediction with population-based simulation to identify when representing individual agents creates a real gain. - Develop methods for rare, novel, or rapidly changing settings where labels are sparse and conventional supervised learning is unreliable. - Study how prediction quality changes with model capability, retrieval quality, inference-time computation, data coverage, historical context, and population representation. - Improve selective prediction so the system can identify cases where the available evidence does not support a confident claim. ## How we work We measure progress against future or otherwise held-out outcomes. Calibration, temporal validity, subgroup behavior, selection effects, leakage, and performance under shift are first-class research concerns. A simple baseline that wins is more valuable than a sophisticated method that cannot demonstrate signal. Exploration is encouraged, but experiments should be designed to fail clearly. We prefer a well-supported null that changes our beliefs over an ambiguous improvement that survives only through favorable choices. Research should eventually change what Aaru can build or what the company believes. Researchers own their work from question selection through implementation, evaluation, explanation, and production handoff. Collaboration is intensive, but responsibility for reaching a truthful conclusion is not diffuse. ## You might thrive in this role if - You have a record of rigorous work in machine learning, forecasting, statistics, econometrics, quantitative research, decision science, or a related field. - You have built predictive systems from messy, heterogeneous data and evaluated them against observed outcomes. - You are comfortable moving between statistical reasoning, model design, data construction, implementation, experiment design, and detailed error analysis. - You understand calibration and can explain why accuracy alone is often insufficient for consequential prediction. - You know how temporal leakage, selection effects, label construction, dataset shift, and subgroup imbalance can create convincing but invalid results. - You understand the strengths and failure modes of language models and are willing to combine them with—or replace them by—simpler quantitative methods. - You generate your own questions, prioritize the uncertainty that matters most, and design experiments that can resolve it. - You write strong research code and are willing to improve it enough that another team can reproduce and productionize the result. - You communicate clearly, including when the evidence is negative, fragile, or inconsistent with your preferred hypothesis. - You want to work in person in New York with a team that moves quickly and treats empirical truth as the standard. ## Strong candidates may also have - Work in time-series modeling, econometrics, recommender systems, demand forecasting, risk modeling, causal inference, quantitative social science, or probabilistic programming. - Experience with LLM agents, retrieval, tool use, post-training, synthetic environments, model-based reasoning, or inference-time scaling. - Experience with prospective forecasting, prediction markets, decision-making under uncertainty, or operational prediction systems. - Experience building proprietary datasets, joining noisy external sources, or constructing outcomes and features with careful temporal provenance. - A publication record or open research demonstrating original empirical work, though shipped or operational results are equally valuable. - Experience with behavioral, transaction, product, market, polling, or longitudinal data. - Experience taking a research method through production deployment and learning from how it behaved in the field. ## Candidates need not have - Prior experience in population simulation or a background exclusively in language-model research. - A PhD, provided you can demonstrate equivalent research depth and empirical rigor. - Every method listed above. We care most about your ability to choose the right method, test it honestly, and learn quickly. ## What success looks like - You own an important prediction problem and create a clear body of evidence about what works, what does not, and under which conditions. - Your methods outperform strong baselines on clean historical and prospective outcomes rather than only on development benchmarks. - Improvements are calibrated, robust across relevant groups and time periods, and honest about cases where the evidence is weak. - Your work clarifies whether errors originate in population representation, conditioning information, data quality, model choice, or evaluation design. - Research artifacts are reproducible, well documented, and straightforward for collaborators to inspect and challenge. - Validated methods move into production and improve the quality of Aaru's simulations or customer-facing decisions. - Negative results and failed approaches reduce uncertainty and prevent the company from investing in attractive but unsupported ideas. - Other researchers trust your judgment because you combine technical ambition with unusually strong empirical discipline. ## Location and benefits This role is based in New York City. Aaru is an in-person company, working five days a week in the office. Candidates should be located in the New York metropolitan area or open to relocation. Aaru offers a competitive base salary, equity participation, comprehensive medical, vision, and dental coverage, visa sponsorship and relocation support, and other benefits and perks. Final compensation depends on level and experience and is set within Aaru's internal bands.