Python Technical Lead

Role Summary

We are seeking a Hands-On Python Lead Engineer to lead the design and implementation of enterprise-grade big data pipelines. This is a technical leadership role requiring significant daily coding. The successful candidate will architect scalable ETL frameworks using Python, PySpark, and Apache Airflow, serving as both a lead developer and a system designer. Experience integrating with Salesforce ecosystems is strongly preferred.

Core Responsibilities

Architectural Design & Framework Development

- ETL Framework Design: Architect modular, metadata-driven ETL frameworks to standardize data ingestion and processing across the organization. - Airflow Orchestration: Design complex workflow orchestration strategies. Customize Airflow environments, including developing custom operators, sensors, and plugins to support business logic. - Scalability & Performance: Engineer high-performance pipelines capable of handling large-scale datasets; optimize DAG execution, memory management, and concurrency settings.

Hands-On Implementation

- Code Contribution: Actively write, test, and deploy production-quality code in Python and PySpark (60%+ of role focus on hands-on development). - Code Quality Assurance: Enforce strict coding standards, design patterns, and unit testing coverage through rigorous code reviews.

Integration Strategy

- Data Integration: Architect robust patterns for consuming data from Databases, Data Lakes, and APIs. - CRM Data Integration: Oversee the strategy for ingesting and synchronizing data from Salesforce, ensuring adherence to API limits and data integrity.

Required Qualifications

- Python & PySpark Expertise: 8+ years of software development experience. Mastery of Python and deep proficiency in PySpark for distributed data processing. - Airflow Proficiency: Extensive experience setting up, managing, and customizing Apache Airflow in a production environment. - ETL Architecture: Proven track record of building custom ETL frameworks from scratch, moving beyond basic scripting to software engineering-based data solutions. - Database Skills: Advanced SQL knowledge and experience with cloud data warehouses.

Preferred Qualifications

- Salesforce Knowledge: Experience with Salesforce data models, SOQL, and bulk data extraction APIs. - CI/CD: Experience implementing CI/CD pipelines for data engineering (GitHub Actions, Jenkins).