Data Engineer

Senior Data Engineer – Python & Azure

Experience:

5+ Years Type:

Full-time Location:

Hyderabad About the Role We are looking for a highly skilled

Senior Data Engineer

with deep expertise in the

Microsoft Azure

ecosystem to architect, build, and scale our next-generation data platform. You will be a technical leader responsible for designing robust data pipelines that power advanced analytics, BI, and AI workloads across the organization. As a Senior Engineer, you won’t just build pipelines; you will partner with data scientists, analysts, and business stakeholders to ensure our data is high-quality, secure, and performant. Key Responsibilities Pipeline Architecture:

Design and maintain end-to-end ETL/ELT workflows using

Azure Data Factory (ADF) ,

Databricks , and

Synapse Analytics . Data Modeling:

Implement scalable data models (Star Schema, Snowflake, Data Vault) for both analytical and operational stores. Real-time & Batch Ingestion:

Manage ingestion frameworks for diverse sources (APIs, SaaS, On-prem) and streaming use cases using

Azure Event Hubs

or

Kafka . Governance & Security:

Enforce data quality and governance standards (lineage tracking, metadata management) and manage security via

Azure Active Directory (AAD) . Cloud Optimization:

Manage

ADLS Gen2

storage and compute costs while ensuring system resilience and scalability. Leadership & Mentorship:

Partner with cross-functional teams and mentor junior engineers on coding standards and Azure best practices. Required Technical Skills Core Azure Services:

Expert-level knowledge of Azure Data Factory, Azure Databricks (PySpark/Delta Lake), Synapse Analytics, and Azure Functions. Programming:

Strong proficiency in

Python

or

Scala

and advanced

SQL

(T-SQL, Spark SQL, or Synapse SQL). Big Data Processing:

Extensive experience with

PySpark

for distributed data processing. DevOps:

Hands-on experience with

Git , CI/CD (Azure DevOps/GitHub Actions), and Infrastructure as Code (Terraform/Bicep). Data Architecture:

Deep understanding of Data Lakehouse architecture, partitioning, indexing, and query optimization. Soft Skills & Qualifications Bachelor’s degree in Computer Science, Engineering, or a related technical field. Proven ability to work in an

Agile , fast-paced environment. Excellent communication skills with the ability to explain technical trade-offs to non-technical stakeholders. A self-driven mindset with a passion for cloud technologies and problem-solving.