AI Infrastructure Engineer – Agents & ML Systems

Job description

ABOUT US: Havoc is a leader in all-domain collaborative autonomy. Its software-defined hardware approach powers military and commercial-grade autonomous systems across sea, air, and land to sense, decide, and act together in complex and contested environments. Havoc connects assets, enabling them to share information, adapt in real time, and continue operating even when communications are disrupted or denied. Havoc optimizes mission performance and minimizes human risk. Havoc was founded in 2024 and headquartered in Providence, Rhode Island. Learn more at Havoc: All-Domain Collaborative Autonomy http://havocai.com/ . ABOUT THE ROLE As an AI Infrastructure Engineer – Agents & ML Systems, you will help build the internal AI infrastructure that allows HavocAI teams to use modern AI systems safely, reliably, and effectively. You will develop tools, services, pipelines, and integrations that connect large language models, agentic workflows, internal data sources, engineering systems, and ML workflows. This role is ideal for a strong software or infrastructure engineer who is excited about the practical application of AI. You need not have worked on every part of the AI stack, but you should be curious, hands-on, and comfortable building production systems that connect models, tools, data, and users. You will work on systems that help internal teams search and reason over company data, automate engineering workflows, support simulation and autonomy development, curate data for future model training, and evaluate AI systems before they are trusted in critical workflows. This is a high-impact role at the intersection of software engineering, AI infrastructure, developer tooling, data systems, and applied ML. JOB RESPONSIBILITIES - Build internal AI infrastructure that connects LLMs and AI agents with internal tools, APIs, data sources, data lakes, telemetry stores, simulation tools, code repositories, documentation systems, logs, and engineering workflows. - Develop and maintain agentic AI systems for task automation, data analysis, engineering support, simulation workflows, and internal productivity. - Build tool integration and connector infrastructure for AI agents, including MCP and other emerging tool-use standards, spanning servers, tools, resources, prompts, connectors, and secure tool-use patterns. - Create pipelines for retrieval, RAG, context management, document processing, embeddings, and internal knowledge search. - Support ML infrastructure workflows such as data preparation, dataset curation, experiment tracking, model evaluation, fine-tuning support, and model deployment. - Build evaluation frameworks for agent performance, tool-use reliability, task success, model quality, regression testing, and failure analysis. - Develop observability, logging, tracing, auditability, monitoring, and debugging tools for AI agents, model calls, MCP tools, and ML pipelines. - Partner with Autonomy, Software, Data, Simulation, Product, and Operations teams to identify high-value AI use cases and turn them into reliable internal tools. - Secure agentic AI systems end-to-end with least-privilege tool access, sandboxed tool execution, prompt-injection and misuse mitigation, secrets management, human-in-the-loop approvals, and safe handling of sensitive and defense data. - Maintain documentation, reusable examples, templates, and best practices that help internal teams adopt AI tools safely and effectively. QUALIFICATIONS - Bachelor's degree in Computer Science, Engineering, Machine Learning, Data Science, Applied Mathematics, or a related technical field. - 3+ years of experience in software engineering, infrastructure engineering, ML infrastructure, backend systems, data engineering, developer tools, or related technical roles. - Strong programming experience in Python, TypeScript, Go, C++, or similar languages. - Experience building production software systems, APIs, services, data pipelines, or internal platforms. - Experience with, or strong interest in, LLM applications, AI agents, tool-using systems, RAG pipelines, or AI developer tools. - Familiarity with modern AI infrastructure concepts such as embeddings, vector search, prompt management, evaluation, model serving, fine-tuning, or MLOps. - Ability to integrate systems across APIs, databases, object stores, documents, logs, internal tools, and structured or unstructured data sources. - Strong understanding of production engineering fundamentals, including reliability, observability, testing, and maintainability. - Strong grounding in securing AI and agentic systems, including least-privilege tool access, prompt-injection and misuse mitigation, secrets management, and safe handling of sensitive data. - Ability to work across Software, Data, ML, Infrastructure, and Product teams. - Strong debugging skills and comfort working with complex distributed systems. - U.S. citizenship and the ability to obtain and maintain a security clearance. PREFERRED SKILLS - Experience with MCP, including MCP servers, clients, tools, resources, prompts, or connector patterns. - Experience with LangGraph, LangChain, LlamaIndex, Semantic Kernel, OpenAI APIs, Anthropic APIs, local/open-weight models, or similar AI tooling. - Experience with vector databases, embeddings, retrieval systems, knowledge graphs, document processing, search, or RAG systems. - Experience with LLM fine-tuning, supervised fine-tuning, preference tuning, synthetic data generation, evaluation datasets, or model benchmarking. - Experience with Kubernetes, Docker, Ray, Airflow, Dagster, MLflow, Weights & Biases, Kafka, Postgres, S3-compatible storage, or similar infrastructure. - Experience building internal platforms, developer tools, workflow automation systems, or data/ML infrastructure. - Experience with human-in-the-loop workflows, approval systems, audit logs, policy enforcement, or safe tool invocation patterns. - Experience integrating AI tools with engineering workflows such as GitHub, CI/CD, issue trackers, documentation systems, simulation platforms, or data lakes. - Familiarity with autonomy, robotics, simulation, telemetry, perception, or defense technology workflows. - Experience building secure AI systems for sensitive, regulated, government, defense, or enterprise environments. - Active or prior security clearance. BENEFITS: - 100% Employer paid Health, Dental and Vision Insurance for you and your families - Life Insurance (Employer Paid) - Ability to participate in the companies 401k program (Matching) - Unlimited PTO policy with an enforced 2 week minimum - Equity Package - Work / Home Office Stipend - Global Entry - 16 Week Paid Parental Leave - Monthly Health and Wellness Stipend OUR VALUES: - Innovation: We are driven to break new ground. Every day presents an opportunity to challenge the status quo, think boldly, and deliver advanced solutions that transform the future of defense technology. - Integrity: We hold ourselves to the highest ethical standards, ensuring transparency, accountability, and trust in all our actions and partnerships. - Mission-Driven: We are focused on achieving impactful outcomes that align with our core mission—protecting lives through innovation. - Forward-Leaning: We continuously seek out new opportunities and remain at the forefront of technological advancements. We embrace change and anticipate the challenges of tomorrow with confidence and creativity. - Ownership of All Tasks: At HavocAI, no problem is too complex or too trivial. We believe that greatness comes from tackling the hardest challenges, but also in handling the smallest, sometimes thankless, tasks with the same level of commitment and care. - Servant Leadership: We lead by serving others, whether it’s supporting our employees, partners, or the broader community. Empowering those around us is key to achieving long-term success and making a lasting impact. HavocAI is an Equal Opportunity Employer and is committed to creating an inclusive and diverse workplace. We welcome applicants from all backgrounds and do not discriminate based on race, color, religion, gender, sexual orientation, age, national origin, disability, veteran status, or any other legally protected status.