Azure OpenAI & RAG Engineer

Job description

SkyePoint Decisions is a leading Cybersecurity Architecture and Engineering, Critical Infrastructure and Operations, and Applications Development and Maintenance IT service provider headquartered in Dulles, Virginia with operations across the U.S. We provide innovative enterprise-wide solutions as well as targeted services addressing the complex challenges faced by our federal government clients. Our focus is on enabling our clients to deliver their mission most efficiently and effectively – anytime, anywhere, securely. We combine technical expertise, mission awareness, and an empowered workforce to produce meaningful results. SkyePoint Decisions is seeking an experiencedAI Engineer with Azure OpenAI experience to design and implement a AI inference and Retrieval-Augmented Generation (RAG) pipeline for Department of State. This role is responsible for provisioning and configuring Azure OpenAI Service on Azure Government, building the RAG pipeline that grounds AI responses in agency playbooks and live vulnerability feeds, and implementing the confidence scoring mechanism that drives the human validation queue. The engineer works in close coordination with a Power Platform Developer on prompt context assembly and with the agency SME on response quality. **This position is based on-site at the Beltsville, MD office, with occasional travel to the Washington, DC office.** **Responsibilities:** - Provision and configure Azure OpenAI Service on Azure Government in compliance with DOS GovCloud FedRAMP High requirements. - Design and implement the RAG pipeline using Azure AI Search: vector index configuration, document chunking strategy for agency playbooks and runbooks, embedding model selection, and semantic search tuning. - Build and maintain automated data ingestion pipelines for CISA KEV (daily), NVD CVE database (daily), Microsoft Security Update Guide (weekly), and agency SharePoint document library. - Implement ServiceNow ticket context injection: when a matching SNOW ticket is retrieved, structure the vulnerability description, CVE identifier, asset details, and current ticket status into the AI prompt context for response generation. - Design and implement the AI confidence scoring mechanism — per-response confidence scores surfaced to agency analysts in the validation queue. - Implement CUI-safe prompt construction: ensure sanitized email content and structured SNOW ticket data are assembled into prompts without including raw CUI elements that have been vaulted. - Configure Azure AI Content Safety filters appropriate to the advisory use case. - Monitor model performance, token consumption, and response latency in the Azure Government environment. - Support the monthly review of AI confidence score distribution, override rate, and coverage metrics. - Produce AI system documentation including prompt templates, RAG configuration, and model monitoring procedures. **Required Qualifications:** - 3+ years of experience with Azure OpenAI Service including API integration, prompt engineering, and deployment in enterprise environments. - Demonstrated experience building RAG pipelines using Azure AI Search or equivalent vector search technology. - Experience with Python or JavaScript/TypeScript for AI pipeline development. - Familiarity with LLM prompt engineering techniques including few-shot prompting, chain-of-thought, and system prompt design. - Experience with FedRAMP-authorized cloud environments, specifically Azure Government. - Experience with REST API integrations for data feed ingestion (JSON APIs, rate limiting, error handling). - Active Top Secret security clearance OR ability to obtain TS within program timeline (interim clearance accepted to start). - US Citizenship is required. **Preferred Qualifications:** - Active Top Secret clearance. - Experience with LangChain, Semantic Kernel, or Azure AI Foundry for agentic AI pipeline orchestration. - Experience integrating AI systems with ServiceNow or similar ITSM platforms for context enrichment. - Familiarity with CISA KEV, NVD CVE data structures, and cybersecurity vulnerability terminology. - Experience with Microsoft Copilot Studio or Power Platform AI Builder integrations. - Azure AI certifications (AI-102, DP-100, or equivalent). - Experience with GCC High M365 environments and their AI service constraints. - Familiarity with NIST AI RMF, OMB M-25-21 AI governance requirements, or federal AI policy. **Compensation:** Salary Range: $140,000-$160,000 The SkyePoint Decisions salary range for this position is a general guideline only. It represents an estimated range for this position and is just one piece of our total compensation package. Salary at SkyePoint is determined by various factors, including but not limited to location, work schedule, the candidate’s combination of education, knowledge, skills, competencies, and experience, as well as contract-specific affordability, market data and business considerations. In addition to a competitive salary, SkyePoint offers benefits including a certification incentive program, PTO, floating federal holiday options, several insurance options including HMO and High Deductible plans with Health Savings Accounts [HSAs], Flex Spending Accounts [FSAs], Full Dental Plans, Vision, ST/LT Disability, Life Insurance, and 401k matched. **What We Can Offer You:** - At SkyePoint, we go B.I.G. (beginning in GRATITUDE) by recognizing all we have and giving back to our employees, families, and communities. It instills a positive mindset that permeates all we do. By beginning in gratitude, SkyePoint can continue to spread living in gratitude each day. - Great Benefits: Several insurance options including HMO and High Deductible plans with Health Savings Accounts [HSAs], Flex Spending Accounts [FSAs], Full Dental Plans, ST/LT Disability, Life Insurance, floating federal holiday options, and 401k matched - Certificate Incentive Program: To promote professional development, we recognize and reward employees who obtain new certifications aligned with business needs. - Flexible Work Environment SkyePoint Decisions is an established ISO 9001:2015 and ISO/IEC 27001:2013 certified small business and appraised at CMMI Level 3 for Services and Development. We possess a common vision of excellence and foster a collaborative team culture built upon individual performance and accountability. We invest in our people and systems to create value for our clients. It is the SkyePoint Way. We are grateful for the opportunity to work with exceptional people and give back to the communities we serve. Our employees value the flexibility at SkyePoint that allows them to balance quality work and their personal lives. SkyePoint Decisions is a participating E-Verify Employer. U.S. Citizenship is required for most positions. Equal Opportunity Employer/Veterans/Disabled. CCPA Disclosure Notice Here