Data Engineer Consultant

Company Description NextMantra AI is an IT services and consulting company that leverages AI-powered hiring technology and domain expertise to help organizations accelerate digital transformation and build high-performing technology teams. With a proprietary AI Adaptive Cross-Questioning Video Assessment Platform, the company ensures thorough technical evaluations to connect businesses with highly qualified, project-ready talent. NextMantra AI specializes in delivering consulting and talent solutions for industries such as financial services, technology, FinTech, HealthTech, E-commerce, AI/ML, and more. With expertise spanning SEA, MENA, APAC, and the USA, NextMantra AI provides scalable tech capabilities, strategic hiring solutions, and compensation benchmarking to empower businesses in dynamic, high-growth sectors.

Role Description We are seeking a lead professional with 7-8 years of experience in Data Engineering. The ideal candidate will possess a comprehensive skill set, including proficiency in SQL, data warehousing, Azure Data Factory, Kubernetes, Python, shell scripting, Azure pipelines, and Snowflake or similar technologies. Responsibilities include collaborating with Development, QA, and other stakeholders to manage multiple projects simultaneously. Key Responsibilities: Data Integration and Management:

Lead initiatives focused on integrating, transforming, and managing large datasets from diverse sources, ensuring optimal availability and performance. Advanced SQL Queries:

Analyze complex queries to resolve data issues; write ETL code to meet project requirements. Azure Data Factory:

Leverage Azure Data Factory to orchestrate data workflows and pipelines, implementing best practices for cloud-based integration and processing. Azure Pipelines Deployment:

Manage CI/CD pipelines using Azure Pipelines, automating and streamlining deployment processes. Kubernetes Deployment:

Oversee deployment and management of data solutions within Kubernetes environments, ensuring application scalability, reliability, and security. Python and Shell Scripting:

Develop custom scripts in Python and shell for tasks related to data processing, automation, and system integration. Project Leadership:

Lead multiple projects simultaneously, coordinating among development, QA, and business teams; provide technical leadership and mentorship to junior team members. Requirement Analysis:

Work with stakeholders to understand business requirements and design and execute builds to fulfill those needs. Configuration and Execution:

Configure systems according to project specifications and execute data builds efficiently. API Configuration:

Understand and configure APIs using Python code. Qualifications: Bachelor’s degree in Computer Science, Information Technology, or a related field. 7-8 years of experience in data engineering with a strong background in technical leadership. Proven expertise in SQL, data warehousing, and cloud environments (preferably Microsoft Azure). Hands-on experience with Azure Data Factory, Azure pipelines, Kubernetes, Python, shell scripting, and Snowflake or similar technologies. Experience handling extensive datasets and developing ETL code to import data into a datamart. Demonstrated ability to collaborate with cross-functional teams in a dynamic setting. Strong problem-solving and analytical capabilities. Retail domain experience is a plus. Excellent communication and interpersonal skills. Immediate availability is preferred.