Senior Data Engineer (Team Lead)

Location:

In-office Duration:

Full-time (Permanent) Experience:

4–6 years

Mandatory Skills: Data pipeline architecture & development (ETL/ELT) at scale Advanced data modeling, data warehousing High proficiency in Python; building & debugging RESTful APIs End-to-end ownership of identifying, evaluating, and onboarding new data sources Data research, mining, merging & enrichment from diverse sources Evaluate data quality & reliability before closing partnerships Build & maintain web scraping/crawling infrastructure Design data merging, deduplication & enrichment strategies System architecture — designing scalable, reliable data platforms Manage & guide a team of 2–4 engineers (code reviews, mentoring, setting standards) Architecture decision-making and technical direction for the data team Strong communicator — cross-functional collaboration with product/business stakeholders

Mandatory Tech Stack: Python, Django, MongoDB AWS (S3, Glue, Lambda, Redshift) Apache Airflow, Spark/PySpark Elasticsearch (search-based data mining) Web scraping tools (Scrapy, BeautifulSoup, Selenium) RESTful APIs, Git/GitHub

Preferred Skills: Experience with data marketplace platforms (AWS Data Exchange, Snowflake Marketplace) Real-time/streaming data processing Data quality & validation frameworks CI/CD pipelines, infrastructure-as-code Agile/Scrum

Preferred Tech Stack: Go (Golang) Kafka, Docker, CloudWatch Third-party data APIs & marketplace integrations