Lead Data Engineer

Lead Data Engineer - - Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse). - Create and maintain documentation for data pipelines, processes, and best practices. - Knowledge of Performance Optimization, Monitoring and Automation would be a plus. - Understanding of data governance, compliance, and security best practices. - Knowledge in anyone of the cloud platforms AWS is a must. - Strong Proficiency in Pyspark, Databricks Notebooks, SQL, Python, Scala, or Java. - Certifications in Databricks would be beneficial. - Strong understanding of data engineering best practices, data modeling, Data warehousing and ETL processes. - Staying up-to-date with the latest Databricks features and best practices. - Proficiency in SQL for data querying and manipulation. - Strong problem-solving and troubleshooting skills. - Excellent communication and collaboration skills