Research Engineer - Model Evaluation & MLOps

Job description

Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct sponsorship from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the full stack powering frontier AI models and real-time applications. ## **About the role** As a Research Engineer focused on Model Evaluation & MLOps, you will build the tools and infrastructure needed to evaluate, deploy, and operate multimodal foundation models reliably. You will rapidly enable Sciforium’s models and the latest open-weight models on GPUs, automate quality and performance benchmarking, and improve the MLOps workflows that connect research experiments to reliable releases. **Key Responsibilities** ### **Model Enablement & Automated Evaluation** - Rapidly integrate new internal and open-weight language and multimodal models into our GPU evaluation and inference environments. - Build automated benchmarks for model quality and systems performance, including latency, throughput, and memory usage. - Create standardized, reproducible comparisons across Sciforium models, external baselines, and runtime configurations. ### **MLOps & Model Lifecycle** - Build and maintain experiment tracking, model registry, and versioning for models, datasets, and evaluation configurations. - Automate the path from research checkpoints to validated deployments through CI/CD and reproducible workflows. - Monitor model quality and systems performance, and diagnose failures or regressions across model and deployment pipelines. ### **Research & Systems Collaboration** - Build reusable tools that help researchers launch evaluations, compare experiments, and reproduce results. - Profile end-to-end model workloads and collaborate with distributed systems, inference, and GPU kernel engineers on deeper performance issues. ## **Must-Haves** Candidates may be stronger in some areas than others. We are looking for strong software engineering foundations, hands-on ML systems experience, and depth in at least one of model evaluation, MLOps, or model deployment. - **Experience:** 2+ years of professional ML or software engineering experience, including work on production ML systems, ML platforms, or MLOps infrastructure. - **Software Engineering:** Strong Python and software engineering skills, with experience building reliable production systems. - **Machine Learning Expertise:** Hands-on experience with PyTorch, TensorFlow, or JAX and a good understanding of modern language or multimodal model architectures. - **Evaluation & MLOps:** Experience with model evaluation or benchmarking and core model lifecycle workflows such as experiment tracking, versioning, deployment, or monitoring. - **GPU Systems:** Experience running, benchmarking, and debugging models with one or more GPU inference runtimes, such as vLLM, SGLang, TensorRT-LLM, or equivalent, in containerized cloud or on-premises environments. - **Communication:** Ability to document systems clearly and collaborate across research, infrastructure, and product engineering teams. - **Education:** MS or PhD in Computer Science, Computer Engineering, Machine Learning, or a related technical field, or equivalent practical experience. ## **Nice-to-Have** - Familiarity with Hugging Face Transformers or similar model libraries. - Experience enabling models on AMD GPUs and ROCm. - Contributions to open-source evaluation, model, or ML infrastructure projects. ## **Benefits include** - Medical, dental, and vision insurance - 401k plan - Daily lunch, snacks, and beverages - Flexible time off - Competitive salary and equity ## **Equal opportunity** Sciforium is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.