AI Architect - Databricks

Job DescriptionWe are looking for an experienced AI Architect with strong expertise in Databricks, Data Engineering, Machine Learning, and Cloud-based AI solutions to lead the design and implementation of scalable AI/ML platforms and intelligent data solutions. The ideal candidate will define enterprise AI architecture, drive modern data and AI transformation initiatives, and work closely with business, engineering, and analytics teams to deliver high-impact solutions.This role requires a strong understanding of end-to-end AI/ML lifecycle, Lakehouse architecture, Databricks ecosystem, and the ability to translate business use cases into production-ready AI solutions.Key ResponsibilitiesDesign and implement enterprise AI/ML architecture using Databricks Lakehouse Platform.Define scalable solutions for data ingestion, feature engineering, model training, deployment, monitoring, and governance.Architect and optimize AI/ML pipelines using Databricks, Spark, Python, SQL, and cloud-native services.Lead the setup of MLOps / LLMOps frameworks for model lifecycle management, CI/CD, model registry, and automated deployment.Work with business stakeholders to identify and prioritize AI/ML use cases, including predictive analytics, NLP, recommendation engines, and generative AI.Build and guide architecture for LLM / Generative AI solutions, including RAG, vector databases, prompt orchestration, and model integration where applicable.Establish best practices for data quality, security, compliance, observability, scalability, and responsible AI.Collaborate with Data Engineers, Data Scientists, Product Owners, and Cloud teams to ensure solution alignment with enterprise architecture standards.Provide technical leadership in selecting AI/ML tools, frameworks, and cloud services aligned to business and platform strategy.Support architecture reviews, technical design workshops, PoCs, and enterprise AI roadmap planning.Required Skills & Experience8- 15 years of experience in Data / AI / Analytics architecture, with strong exposure to enterprise-scale implementations.Hands-on experience with Databricks including:Databricks LakehouseDelta LakeUnity CatalogMLflowDatabricks WorkflowsModel Serving / Feature Store (preferred)Strong programming experience in Python, PySpark, SQL, and AI/ML solution development.Solid experience in designing and deploying Machine Learning pipelines in production.Good understanding of MLOps / LLMOps, including model versioning, CI/CD, deployment, monitoring, and governance.Experience with cloud platforms such as Azure, AWS, or GCP, preferably with Databricks integration.Familiarity with Generative AI / LLM ecosystems, such as OpenAI, Hugging Face, LangChain, vector stores, embeddings, and RAG architectures.Strong understanding of data engineering, ETL/ELT, distributed computing, and data platform modernization.Experience in solution architecture, technical governance, and stakeholder engagement.

Job Description

We are looking for an experienced AI Architect with strong expertise in Databricks, Data Engineering, Machine Learning, and Cloud-based AI solutions to lead the design and implementation of scalable AI/ML platforms and intelligent data solutions. The ideal candidate will define enterprise AI architecture, drive modern data and AI transformation initiatives, and work closely with business, engineering, and analytics teams to deliver high-impact solutions.

This role requires a strong understanding of end-to-end AI/ML lifecycle, Lakehouse architecture, Databricks ecosystem, and the ability to translate business use cases into production-ready AI solutions.

Key Responsibilities

• Design and implement enterprise AI/ML architecture using Databricks Lakehouse Platform.
• Define scalable solutions for data ingestion, feature engineering, model training, deployment, monitoring, and governance.
• Architect and optimize AI/ML pipelines using Databricks, Spark, Python, SQL, and cloud-native services.
• Lead the setup of MLOps / LLMOps frameworks for model lifecycle management, CI/CD, model registry, and automated deployment.
• Work with business stakeholders to identify and prioritize AI/ML use cases, including predictive analytics, NLP, recommendation engines, and generative AI.
• Build and guide architecture for LLM / Generative AI solutions, including RAG, vector databases, prompt orchestration, and model integration where applicable.
• Establish best practices for data quality, security, compliance, observability, scalability, and responsible AI.
• Collaborate with Data Engineers, Data Scientists, Product Owners, and Cloud teams to ensure solution alignment with enterprise architecture standards.
• Provide technical leadership in selecting AI/ML tools, frameworks, and cloud services aligned to business and platform strategy.
• Support architecture reviews, technical design workshops, PoCs, and enterprise AI roadmap planning.

Required Skills & Experience

• 8- 15 years of experience in Data / AI / Analytics architecture, with strong exposure to enterprise-scale implementations.
• Hands-on experience with Databricks including:
• Databricks Lakehouse
• Delta Lake
• Unity Catalog
• MLflow
• Databricks Workflows
• Model Serving / Feature Store (preferred)
• Strong programming experience in Python, PySpark, SQL, and AI/ML solution development.
• Solid experience in designing and deploying Machine Learning pipelines in production.
• Good understanding of MLOps / LLMOps, including model versioning, CI/CD, deployment, monitoring, and governance.
• Experience with cloud platforms such as Azure, AWS, or GCP, preferably with Databricks integration.
• Familiarity with Generative AI / LLM ecosystems, such as OpenAI, Hugging Face, LangChain, vector stores, embeddings, and RAG architectures.
• Strong understanding of data engineering, ETL/ELT, distributed computing, and data platform modernization.
• Experience in solution architecture, technical governance, and stakeholder engagement.

Unison Consulting was launched in Singapore on September 2012, the hub of the financial industry, with innovative visions in the technocratic arena. We are a boutique next-generation Technology Company with strong business-interests in Liquidity risk, Market Risk, Credit Risk and Regulatory Compliance. Unison provides technology consulting and services to implement Risk Management and Risk Analytics System for Financial Institutions. Our services suite comprises of Techno-Functional consulting, systems integration, Business Intelligence, information management, and custom development of IT solutions, plus project management expertise for financial institutions. We have expertise in latest cutting edge technology to achieve better total cost of ownership. Through our qualified professionals, we assist you drive your unique risk management strategies, whether that means efficient monitoring, improving risk appetite of the financial institutions, complying with regulations, or capturing growth opportunities through innovation, this is what maximizes your decision taking potential. At Unison Consulting, we view clients as partners, and our success is only measured by the success of our partners. So we put it all on the table in order to exceed expectations. Our staff consists of young, energetic and innovative consultants who are never afraid to challenge the conventions and push the boundaries in an effort to help our clients. For every project, no matter how large or how small, we strive to not only meet your needs, but deliver a showcase in your field.

Unison Consulting was launched in Singapore on September 2012, the hub of the financial industry, with innovative visions in the technocratic arena. We are a boutique next-generation Technology Company with strong business-interests in Liquidity risk, Market Risk, Credit Risk and Regulatory Compliance. Unison provides technology consulting and services to implement Risk Management and Risk Analytics System for Financial Institutions.

Our services suite comprises of Techno-Functional consulting, systems integration, Business Intelligence, information management, and custom development of IT solutions, plus project management expertise for financial institutions. We have expertise in latest cutting edge technology to achieve better total cost of ownership. Through our qualified professionals, we assist you drive your unique risk management strategies, whether that means efficient monitoring, improving risk appetite of the financial institutions, complying with regulations, or capturing growth opportunities through innovation, this is what maximizes your decision taking potential.

At Unison Consulting, we view clients as partners, and our success is only measured by the success of our partners. So we put it all on the table in order to exceed expectations. Our staff consists of young, energetic and innovative consultants who are never afraid to challenge the conventions and push the boundaries in an effort to help our clients. For every project, no matter how large or how small, we strive to not only meet your needs, but deliver a showcase in your field.