Senior Machine Learning Solutions Architect

Role Overview

We are seeking a highly skilled Senior Databricks AI/ML Engineer to lead the design, development, and deployment of advanced analytics and machine learning solutions. The ideal candidate will bring deep expertise in Databricks, strong hands-on experience in both AI and traditional ML models, and advanced proficiency in SQL and Python. The engineer will work closely with cross-functional teams to build scalable data and ML pipelines, optimize models, and deliver impactful insights for business growth.

Key Responsibilities Design, build, and deploy end-to-end AI and traditional ML models using Databricks. Develop scalable ETL/ELT pipelines, feature engineering workflows, and model training pipelines. Leverage Databricks Lakehouse capabilities for data preparation, model development, and production deployment. Manage the entire machine learning lifecycle using MLflow, including experiment tracking, model registry, and reproducible workflows. Write high-quality, optimized Python and SQL code for data processing and model operationalization. Collaborate with data engineers, data architects, and business stakeholders to understand requirements and deliver analytical solutions. Implement best practices for versioning, CI/CD for ML, and performance optimization. Monitor model performance and retrain models as needed in production environments. Troubleshoot performance bottlenecks, optimize cluster usage, and ensure cost-efficient Databricks operations. Provide technical guidance and mentorship to junior team members.

Required Skills & Experience Minimum 5 years of hands-on experience in Databricks, including Spark, Delta Lake, and Lakehouse architecture. Minimum 4 years of experience in AI/ML, covering both classical ML algorithms and advanced deep learning techniques. Proven expertise in Python, including libraries such as Pandas, NumPy, Scikit-Learn, and PySpark. Strong proficiency in SQL, including writing complex queries and performance tuning. Hands-on experience with MLflow for tracking, versioning, and managing ML models. Practical experience with data preprocessing, feature engineering, model evaluation, and deployment. Solid understanding of distributed computing concepts and performance optimization in Spark. Experience working in Agile environments and version control tools such as Git.

Good to Have Knowledge or hands-on experience with SAP systems, especially in data extraction or integration with SAP Datasphere, SAC, BW, or SAP S/4HANA. Exposure to cloud platforms (Azure/AWS/GCP) and containerization technologies such as Docker or Kubernetes. Experience in integrating ML models with business applications or APIs.