Data Engineer

How you'll role As a Senior Data Engineer, you will play a crucial role in developing and optimizing data pipelines and managing data solutions within our cloud-based services. You will collaborate with cross-functional teams to ensure the success of data-driven projects while leveraging your expertise in cloud platforms and agile methodologies. Your efforts will significantly improve our data handling strategies, contributing to our business outcomes and technological advancements. What you'll do Develop and maintain ETL/ELT pipelines using Apache Airflow and PySpark Design database schemas and dimensional models to support data normalization Develop and manage data warehouses, data lakes, and data lakehouses Work with relational and NoSQL databases including Azure SQL, PostgreSQL, MongoDB, and Google Firestore Handle various data file formats such as Avro, CSV, Parquet, and Delta Create and manage APIs for data integration with an understanding of REST and microservices architecture Ensure codebase maintenance and optimization using Git for version control Implement integration testing and quality assurance procedures Collaborate with team members for technical design support and project delivery Integrate new data sources to meet evolving business requirements What you'll need Bachelors degree in Computer Science, Data Science, or related field Minimum of 5 years of experience in data engineering and cloud computing Proficiency in Python and SQL, particularly PostgreSQL or SQL Server Experience with Apache Airflow, PySpark, and Databricks Knowledge of cloud platforms, with a focus on Azure Understanding of data warehousing, dimensional modelling, and big data file formats Familiarity with APIs, REST, and microservices Demonstrated experience in managing projects effectively in a cloud ecosystem Ability to lead technical teams and mentor in an Agile environment