Freelance Data Engineer ,Spark, Kafka, BigQuery, GCP/Azure

Company Description ThreatXIntel

is a growing

Cybersecurity, IT Staffing, and Consulting company

delivering end-to-end technology and security solutions. Our services include

cloud security, web and mobile application security testing, DevSecOps, vulnerability assessments, IT consulting, and professional staffing services . We support global corporate clients by

hiring and deploying skilled professionals

across IT and cybersecurity domains while helping organizations strengthen security, optimize operations, and scale efficiently. ThreatXIntel is committed to enabling business growth through secure, reliable, and high-quality technology solutions.

Role Overview We are looking for a

Freelance Data Engineer

with strong expertise in building and optimizing scalable

ETL/ELT pipelines

for batch and real-time data processing.

Key Responsibilities: Design, develop, and maintain ETL/ELT data pipelines Build batch and real-time ingestion, transformation, and loading workflows Work with

Spark (PySpark/Scala)

and streaming technologies like

Kafka/Flink Develop and optimize scalable data architectures including

data lakes ,

data warehouses , and streaming platforms Improve performance of Spark jobs, SQL queries, and data workflows for speed, efficiency, and cost optimization Implement

data quality checks, monitoring, and alerting systems

Required Skills: Strong proficiency in

Python

and

SQL Experience with

Apache Spark

(Spark SQL, DataFrames, Streaming) Hands-on experience with

Kafka

or

Pub/Sub Familiarity with

GCP

and/or

Azure data services Knowledge of

BigQuery ,

Snowflake ,

Redshift , and NoSQL databases Experience with

Airflow, Databricks, Docker, and Kubernetes

is a plus