Head of Data Engineering

iSelect and Compare the Market are trusted, well‑known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer‑focused experience that makes it easier for Australians to find the right products to suit their needs.

What You’ll be doing As Head of Data Engineering, you will lead the development, scalability, and operational excellence of our core data platforms. You’ll drive best‑in‑class engineering practices, ensure data integrity and availability, and oversee automation, monitoring, and optimisation of data workflows. You will also shape modern data architectures, implement cloud‑based big data solutions, and champion DataOps methodologies to deliver efficient, reliable, and innovative data capabilities.

Define and deliver the data engineering roadmap, ensuring governance, security, and data quality.

Architect, build and optimise scalable data solutions utilising Databricks as our Enterprise Date Platform.

Implement DataOps practices for automation, CI/CD, and testing.

Build and lead high‑performing teams while fostering collaboration and innovation.

Stay ahead of emerging trends and adopt tools to enhance automation and cost efficiency.

What’s in it for you At Compare the Market and iSelect we offer a dynamic and collaborative work environment where your talents will be recognised and rewarded. You’ll have the opportunity to make a meaningful impact and contribute to our continued success, we also provide:

Vibrant and social community who have annual celebrations, family fun days and regular events.

Enjoy flexible work arrangements, including the option of one day working from home each week or a 9‑day fortnight.

Enjoy additional “ME” leave days.

Employee discounts on Car, Home, Travel insurance.

Income protection insurance provided to support you in the event of non‑work‑related illness or injury.

What Sets You Apart

Demonstrated leadership and team‑building skills, with experience managing high‑performing engineering teams.

Ability to develop strategy, influence senior stakeholders, and align data solutions with business objectives.

Extensive experience with cloud‑based data platforms (AWS, Azure, GCP) and data lake/lakehouse architectures.

Expertise in big data frameworks (Apache Spark, Hadoop, Snowflake, Redshift, BigQuery, Delta Lake) and strong skills in Python, PySpark, SQL.

Proven ability to design and optimise ETL/ELT pipelines for structured and unstructured data.

Hands‑on experience with DataOps practices, including CI/CD (Terraform, dbt, GitHub Actions, Azure DevOps) and orchestration tools (Airflow, Dagster, Prefect).

Knowledge of real‑time streaming technologies (Kafka, Kinesis, Delta Live Tables).

Strong understanding of data governance, security, and compliance (Unity Catalog, GDPR, CCPA, HIPAA), including RBAC, encryption, and data masking.

Sound like a good fit? Let’s talk.

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

Compare the Market and iSelect are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside the Auto and General Holdings Group without your consent.

#J-18808-Ljbffr