Data Engineer

Responsibilities Building robust large-scale social media crawler for various social media platforms Maintain day-to-day operation of said crawlers alongside our data quality team to make sure the completeness and timeliness of incoming data Work in team along with several stakeholders to better understand and communicate about data requirements and issues Qualifications Intermediate or advanced understanding of SQL Have proficiency in several programming languages, mainly in Javascript (Node) and Python. Having a good experience in using Scala for production (not related to Spark) is a big plus Have intermediate or advanced understanding of several data-related technologies such as MariaDB, Postgres, Redis, Kafka, and Elasticsearch Have ample experience using Linux-based system Have understanding on several data acquisition techniques such as webscraping, API-based data consumption, and so on

#J-18808-Ljbffr