Data Engineer (Hadoop/Spark)

The Data & Feature Engineer's main objective is to drive data availability for business analytic & reporting purposes. Key areas of responsibility:

• Analyze data from data producers and transform them into unified form, organize them within the data platforms and/or analytical/production feature store, produce base & derived features in line with Risk Modelling team requirements
• Develop, maintain and monitor data pipelines preparing suitable data for analytic & reporting activities
• Set up effective data quality monitoring to detect and remedy discrepancies (collaborate with HQ teams)
• Drive utilization of Big Data Hadoop platform to reduce dependence on DWH Analytical Playgrounds
• Support implementation of Feature Store solution aiming to significantly transform data preparation, model development, deployment and monitoring
  
  • Build local expertise for end-to-end development, deployment and monitoring of ML models using Feature Store solution
  • Deliver data processing mechanisms for real-time data processing and analytical use to utilize the value of new technologies

Experience and expertise

Need-to-have areas

• Experience as a Data Engineer
• Experience with Big Data ecosystem, data integration, transformation, processing and visualization tools (Hadoop, Hive, Spark, Kafka, NiFi, Airflow, Jupyter) is a big plus
• Programming Language: SQL and Python on intermediate level
• University degree in Computer Science, Engineering or related field is preferred
• Good written and oral communication skills
• Able to communicate in English

Nice-to-have areas

• Experience with PowerBI/Tableau BI tools
• Experience with ML Ops is advantage
• Experience from working in the data-related area in online, fin-tech, e-commerce, digital, IT dev
• Able to communicate efficiently as well as to interpret and promote a big data culture to the organization

Personal characteristics 

• Real passion for data, new data technologies, and discovering new and interesting solutions to the company’s data needs
• Ability to work on own initiative and to tight deadlines, resilient to pressure
• Ability to formulate, explain, discuss and negotiate topics with IT and Business teams
• Good analytical and problem-solving skills
• Ability to critically review and identify potential data discrepancies