Senior Data Engineer

Project Description:

We are seeking a skilled and hands-on Data Engineer with proven experience in Databricks, Apache Kafka, and real-time data streaming solutions. Responsibilities: - Design and implement scalable data pipelines using Databricks and Kafka - Build and maintain real-time streaming solutions for high-volume data - Collaborate with cross-functional teams to integrate data flows into broader systems - Optimize performance and reliability of data processing workflows - Ensure data quality, lineage, and compliance across streaming and batch pipelines - Participate in agile development processes and contribute to technical documentation Mandatory Skills Description: 5+ years of experience in data engineering roles - Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience). - 4+ years of experience with Databricks data pipelines - Strong hands-on experience with PySpark and/or Apache Spark. - Experience working with CI/CD pipelines and DevOps best practices. - Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards and experience with healthcare data integration projects. - Proficiency with CI/CD, GitHub (pull requests, branching, code reviews). - Experience writing unit tests and performing data validation - Experience supporting production systems and deployments - Strong analytical and troubleshooting skills. Languages: English: C1 Advanced