Incident management / reliability / SRE Evaluator

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Incident Management / Reliability / SRE Evaluator based in the United States.** This remote, hourly opportunity is designed for experienced professionals in incident management, reliability engineering, and site reliability engineering. You’ll use your practical expertise to evaluate AI-generated work products for technical accuracy, rigor, and overall quality. The role provides an opportunity to apply real-world reliability and incident-management judgment to emerging AI systems. You’ll assess whether generated outputs reflect sound operational practices, appropriate reasoning, and domain-specific standards. Your feedback will directly contribute to improving how AI handles reliability, incident response, and SRE-related tasks. The engagement is flexible and fully remote, allowing you to complete project work on your own schedule. You’ll collaborate asynchronously with a research team while contributing specialized expertise to AI development. ### Accountabilities - Evaluate AI-generated documents, spreadsheets, slide decks, and other work products against established incident management, reliability, and SRE quality standards. - Assess generated outputs for technical accuracy, operational rigor, sound judgment, and alignment with real-world reliability practices. - Identify factual, technical, presentation, and reasoning errors that could affect the quality or usefulness of an AI-generated deliverable. - Apply subject-matter expertise to determine whether incident-management and reliability recommendations are practical, complete, and appropriately structured. - Provide clear, concise, and structured written feedback explaining errors, gaps, and opportunities for improvement. - Help establish and apply domain-specific quality criteria and rubrics when evaluating AI outputs. - Collaborate asynchronously with the research team to refine evaluation approaches and improve AI performance. - Complete assigned evaluation projects accurately and within agreed timelines while maintaining professional standards. - Perform work using examples and knowledge that do not require access to confidential or proprietary information from any employer, client, or institution. ## Requirements - **5+ years of relevant professional experience** in incident management, reliability engineering, site reliability engineering (SRE), or closely related disciplines. - Strong practical understanding of incident response, service reliability, operational processes, troubleshooting, and reliability best practices. - Ability to critically evaluate technical work products and distinguish accurate, actionable recommendations from flawed or incomplete reasoning. - Native or professional fluency in English, with strong written communication skills. - Highly proficient with Microsoft Office and Google Workspace, particularly Google Slides and/or PowerPoint. - Excellent attention to detail and the ability to identify factual, technical, aesthetic, and presentation-related issues. - Strong analytical and critical-thinking skills, with the ability to explain technical assessments clearly and objectively. - Ability to work independently and manage deliverables effectively in a fully remote, asynchronous environment. - Preferred: Advanced degree (Master’s or higher) from a reputable institution or equivalent advanced professional expertise. - Comfortable adapting established evaluation criteria and applying consistent judgment across a variety of AI-generated outputs. ## Benefits - **Compensation:** $80–$120 per hour. - Fully remote opportunity for professionals based in the United States. - Flexible work that can generally be completed on your own schedule. - Independent contractor engagement. - Weekly payments through Stripe or Wise based on services rendered. - Opportunity to apply your professional expertise to the development and improvement of AI systems. - Asynchronous collaboration with a research team. - Projects may be extended, shortened, or concluded early depending on project needs and performance. - Work does not require access to confidential or proprietary information from any employer, client, or institution. - Please note: H-1B and STEM OPT candidates cannot be supported for this opportunity.