Databricks Architect

Architect - Databricks (Cloud Data) Job Title:

Architect - Databricks (Cloud Data) Primary skills:

Databricks, PySpark, and architecture solutions. Secondary skills:

Unity Catalog, StarBurst, Azure DevOps, CICD, and orchestration layers. Mode of work:

Work from office Experience:

13+ Years of Experience

Responsibilities: · Lead migration from Native Spark to Databricks. · Architect migration to Databricks. · Build Data Governance solutions using tools like Unity Catalog and StarBurst. · Develop orchestration workflows using Databricks and ADF. · Implement CICD pipelines for Databricks in Azure DevOps. · Process near-real-time data through Auto Loader and DLT pipelines. · Design secure, scalable infrastructure for Delta Lake and Spark SQL. · Optimize Databricks environments for cost-effectiveness. · Extract complex business logic from on-prem solutions like SSIS, Informatica, Vertica, etc., into PySpark. · Build analytical dashboards on top of the Databricks Lakehouse. Requirements - Must Have: · 13+ years of overall experience, with at least 4+ years as a solution architect. · Experience with Databricks, PySpark, and modern data platforms. · Expertise in cloud-native data platform architecture and solution design. · Familiarity with Agile/Scrum methodologies. Experience in pre-sales and in the P&C insurance domain is an advantage.