Data Engineer

KPI Partners is seeking a highly skilled Data Engineer with expertise in Databricks, AWS, Airflow, and Pyspark to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining data pipelines and architectures that support our data-driven decision-making processes. Job Title:

Data Engineer (Databricks, Airflow, AWS, Pyspark) Location:

Hybrid – Hyderabad/Bangalore/Pune Experience:

5+ years Employment Type:

Full-time

Key Responsibilities: Develop and maintain scalable data pipelines using Databricks and Pyspark to support various analytical and reporting needs. Collaborate with data scientists and analysts to understand data requirements and translate them into technical specifications. Implement ETL processes to extract, transform, and load data from various sources into the AWS environment. Optimize data workflows and improve the performance of existing data pipelines in Airflow. Ensure data quality and integrity by implementing monitoring and validation processes. Work with cross-functional teams to gather requirements and provide technical support for data-related projects. Stay updated with the latest industry trends and technologies related to data engineering and big data solutions. Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer with a strong background in Databricks, AWS, Airflow, and Pyspark. Proficient in SQL and experience with data modeling and database design. Strong understanding of data warehousing concepts and methodologies. Familiarity with DevOps practices and CI/CD processes. Excellent problem-solving skills and attention to detail. Strong communication skills and ability to work collaboratively in a team environment. If you feel you have gained this experience in your past/current role and would like to contribute to this opportunity, please feel free to email me a copy of your updated resume and the below-mentioned details at

saswati.paul@kpipartners.com