Senior Data Engineer

QAT Global, a 30+ year IT consulting firm based in the US, is seeking a highly skilled Senior Data Engineer (Kafka / CDC / AWX) with strong English communication skills. This position will work directly with the customer located in the US and with development teams in our US, Costa Rica, and Brazil locations.

- This opportunity can be 100% remote

Project Context

You will be working on a Unified Reporting Platform, responsible for capturing real-time data changes from multiple systems and publishing them to Kafka (Amazon MSK).

Source Systems:

- SQL Server (Direct) - PostgreSQL (Gateway) - PostgreSQL (Payments / Tokenization)

Core Stack:

- Debezium (CDC) - Kafka (Amazon MSK – Kafka 3.x) - Schema Registry - AWS ECS Fargate - Dead Letter Queue (DLQ)

Responsibilities:

- Design, configure, and maintain Debezium connectors (SQL Server and PostgreSQL) - Ensure reliable CDC pipelines with low latency and high data integrity - Manage Schema Registry and schema evolution strategies - Implement and monitor Dead Letter Queues (DLQ) - Perform testing, validation, and troubleshooting of data pipelines - Monitor pipeline performance, lag, and data consistency - Collaborate with engineering teams to ensure data quality and reliability

Required Qualifications:

- Proven experience with Debezium in production environments - Strong understanding of Change Data Capture (CDC) concepts - Hands-on experience with Kafka (topics, partitions, offsets, consumers) - Experience with SQL Server CDC and/or PostgreSQL logical replication - Familiarity with Schema Registry and schema evolution - Experience troubleshooting data pipelines in production - Ability to work in distributed systems and cloud environments

Preferred Qualifications:

- Experience with Amazon MSK - Experience running workloads on AWS (ECS, Fargate) - Experience with DLQ patterns and data validation strategies - Background in event-driven architectures

Key Technical Skills:

- Debezium - CDC (Change Data Capture) - Kafka / MSK - PostgreSQL / SQL Server - Schema Registry - AWS

Nice to Have:

- Experience with high-volume, real-time data pipelines - Experience with observability and monitoring (lag, throughput, failures) - Experience in financial or payment systems