Senior AI Application Engineer

Job description

### Senior AI Application Engineer DISHER is currently partnering with a global supply chain solutions organization on their Senior AI Application Engineer opportunity. In this role, you will own and evolve their Retrieval-Augmented Generation (RAG) layer and agentic AI capabilities. You will design, build, and optimize systems that connect large language models to high-quality, grounded data - and enable them to take intelligent actions in real-world workflows. You will function as a key contributor to build their commercialization of products that use AI as well as their internal AI capability. Partner with leadership to identify, scope, and deliver agentic AI use cases across the business and collaborate in shaping the AI roadmap. **What it's like to work here:** This organization empowers high-performance teams to solve complex global challenges through creativity, partnership, and technology. Employees enjoy a supportive, collaborative environment where initiative and growth are celebrated. With a focus on work-life balance and professional development, this company provides meaningful opportunities for technical and personal advancement in a distributed workplace. ### What you will get to do: RAG Pipeline & Retrieval - Own the end-to-end RAG pipeline, from data ingestion to response generation. - Design and implement Azure AI Search indexing strategies, including schema design, scoring profiles, and performance optimization. - Develop and refine chunking and metadata strategies to maximize retrieval relevance and context quality. - Tune retrieval pipelines (hybrid search, vector search, filters, reranking) for accuracy, latency, and cost. - Build and maintain prompt orchestration frameworks, including dynamic prompt assembly and context injection. - Ensure strong grounding and citation mechanisms so model outputs are traceable and verifiable. Agentic Systems & Integration - Design and implement AI agents and agentic workflows, including tool use, planning, and multi-step task execution. - Integrate agents with internal and external systems (APIs, databases, SaaS tools) to enable real-world actions. - Define and implement evaluation frameworks (offline & online) to measure retrieval quality, agentic performance, and answer correctness. - Design and enforce guardrails (safety, hallucination reduction, policy compliance) across generation and agentic behavior. - Collaborate with product, data, and platform teams to integrate AI capabilities into applications. - Monitor system performance and continuously iterate based on user feedback and metrics. Internal AI Practice - Use Case Development - Partner with business and technology leadership to identify, evaluate, and prioritize agentic AI opportunities across Flash's operations. - Translate business problems into AI system designs - scoping what's feasible, defining the data requirements, and owning delivery. - Build repeatable patterns and internal tooling that accelerate future AI development across the organization. - Contribute to their evolving AI governance and responsible use practices. ### What will make you successful: - Experience building or maintaining RAG systems in production environments. - Experience with hybrid search (keyword + vector) and reranking models. - Knowledge of evaluation frameworks (e.g. RAGAS, LLM-as-a-judge approaches) - Experience integrating agents with enterprise tools (CRM, ERP, ticketing systems, etc.). - Experience optimizing systems for scale, latency, and cost. - Strong knowledge of search technologies (Azure AI Search, Elasticsearch, or similar). - Hands-on experience with LLMs, embeddings, and vector databases. - Experience designing and deploying AI agents or agentic systems. - Demonstrated ability to integrate agents with enterprise systems - ERP, CRM, ticketing, or similar operational platforms. - Understanding of information retrieval concepts (ranking, relevance, recall/precision tradeoffs). - Experience with prompt engineering and LLM orchestration tools. - Familiarity with evaluation techniques for generative AI and agent systems. - Ability to translate ambiguous business problems into concrete AI system designs. - Ability to balance experimentation with production reliability. - Hands-on experience building AI agents and agentic workflows, including tool use, multi-step reasoning, and orchestration across APIs and services. - Understand how to design systems that can plan, retrieve context, take actions, and operate within defined guardrails in production environments. - Proven ability to build effective, reliable AI agents that complete real-world tasks safely and accurately. - Ability to provide measurable improvements in retrieval relevance and system performance. - Previous experience creating robust guardrails and evaluation systems that catch issues early. - Experience building scalable, maintainable architecture integrated into core products. - Self-starter that thrives on ownership. - Must have the current and continuing right to work in the United States of America without restrictions or expirations.