Data Engineer

Hiring: Data Engineer (6–7 Year’s Experience) Location: Remote Experience: 6-7 years Working Hours: 11:00 AM – 9:00 PM IST

About the Role We are looking for experienced Data Engineers (6–7 years) with strong expertise in Azure Databricks, ADF, Python, Apache Spark, and SQL. This role involves building scalable data pipelines, optimizing workflows, and collaborating with business teams to deliver high-performing data solutions. ⭐ Key Responsibilities Design & Develop Data Solutions • Build scalable and optimized data pipelines using Azure Databricks, Apache Spark, and other Azure services. • Develop workflows using Azure Data Factory (ADF) and other low-code/no-code automation platforms. Data Integration • Integrate structured, semi-structured, and unstructured data into Databricks. • Design and implement ETL/ELT pipelines with Delta Lake and Azure Data Lake. Performance Optimization • Optimize Spark jobs for performance, reliability, and cost efficiency. • Troubleshoot bottlenecks in distributed data pipelines. Collaboration & Enablement • Work closely with data analysts, business users, and citizen developers to support self-service analytics. • Enable low-code/no-code solutions for rapid prototyping and automation. Automation & DevOps • Automate data ingestion, transformations, deployments, and workflow orchestration. • Implement CI/CD pipelines using GitHub Actions or similar tools. Governance & Security • Ensure compliance with data governance, security, and privacy policies. • Work with metadata and governance tools for data cataloging and lineage.

️ Required Technical Skills Primary Skills (Must-Have) • Azure Databricks • Azure Data Factory (ADF) • Python • Apache Spark • SQL Secondary Skills (Good-to-Have) • Power Apps • Power Automate • Azure Logic Apps • ETL process expertise • GitHub Actions / CI-CD pipelines • MLflow • Machine learning workflows

Required Experience • 6–7 years of hands-on experience in data engineering. • Strong understanding of ETL/ELT methodologies and data modeling. • Experience working with the Azure ecosystem (Databricks, Data Lake, Synapse, ADF).

Preferred Qualifications • Experience building CI/CD pipelines for Databricks. • Knowledge of MLflow, Delta Lake, and machine learning lifecycle workflows. • Familiarity with Azure Purview or other data governance tools. • Experience working in Agile environments.

Soft Skills • Strong communication and stakeholder management. • Ability to handle multiple workstreams in a dynamic environment. • Problem-solving mindset with attention to detail.