Data Pipeline Architect

We are seeking an experienced

Data Engineer

with strong expertise in

Azure Data Services, Big Data technologies, and ETL/ELT processes

. The ideal candidate will have domain knowledge in

CPG

and

Manufacturing sectors

, along with hands-on experience in

data governance, data modeling, and cloud-based data engineering

. This role involves building scalable data pipelines, integrating industrial data platforms, and enabling advanced analytics for smarter decision-making. Roles & Responsibilities: Design and develop data pipelines using A

zure Data Factory, Azure Databricks, and Azure Data Lake Gen2. Implement

ETL/ELT

processes, data integration techniques, and data governance standards. Work with

Big Data technologies

such as

Apache Spark and PySpark

for large-scale data processing. Gather, engineer, and prepare data for analytics and business insights. Integrate systems and data across diverse environments, ensuring seamless connectivity. Build solutions to extract and consolidate data from multiple sources (sales, inventory, product, customer databases). Collaborate with ML Engineers, Data Scientists, and business stakeholders to enable automation and advanced analytics. Contribute to coding standards and best practices for data engineering workloads. Support modernization initiatives aligned with Industry 4.0 and ISA-95 standards.

Technical Requirement: 5–7 years of experience in

data engineering and data governance

within industrial domains (CPG & Manufacturing). Strong knowledge of

Azure Data Services: Azure Data Factory, Azure Databricks, Azure Data Lake Gen2 Expertise in

ETL processes

, data integration tools, and pipeline development. Proficiency in

Python, SQL, and Apache Spark. Understanding of

Industry 4.0, ISA-95 standards

, and maintenance strategies. Familiarity with

AI/ML concepts

and ability to work with advanced analytics teams. Experience with

DevOps (CI/CD)

for data pipeline automation.

Good to have: Exposure to data modeling, data warehousing, and cloud architecture. Knowledge of business intelligence tools and statistical programming. Ability to research and implement new tools for automation and optimization.

Please note: These positions are not fully remote. Selected candidates will be required to work from an Infosys office near their current location.