Mô tả công việc
Giới thiệu về công ty
International Centre for Trade Transparency, Ltd. (ICTTM), trading as The International Centre for Trade Transparency and Monitoring, is seeking a skilled and experienced Senior Data Engineer to join our innovative development team.
At present, we are creating a product known as ADAMftd (Advanced Data Analytics and Modeling for Foreign Trade Data).
Trách nhiệm công việc
- Design, implement, and maintain scalable and reliable data pipelines and workflows using Python, Airflow, and PySpark
- Develop and optimize data lakes, leveraging Delta Lake and the medallion architecture for efficient and reliable storage
- Integrate and utilize various cloud storage solutions for seamless data management
- Collaborate with software engineering teams to ensure smooth integration of data pipelines with application components
- Mentor and guide junior data engineers, fostering best practices and a data-driven mindset
- Work with cross-functional teams to identify and implement data engineering solutions that support business objectives
- Drive the adoption of new technologies and methodologies to improve data pipeline performance and reliability
- Monitor and troubleshoot data pipeline issues, ensuring high data quality and availability
- Contribute to the development of data governance policies and procedures
- Stay up-to-date with the latest trends and best practices in data engineering
Kỹ năng & Chuyên môn
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- 3+ years of professional experience in data engineering, with demonstrated capability to lead a team
- Strong proficiency in Python and experience with data engineering frameworks such as Airflow and PySpark
- Experience designing and implementing data lakes, preferably using Delta Lake and the medallion architecture
- Familiarity with various cloud storage solutions and their integration with data pipelines
- Knowledge of data modeling, data warehousing, and ETL/ELT processes
- Understanding of data quality, data governance, and data security best practices
- Excellent problem-solving and communication skills
- Fluent English
Technology:
- Python
- Airflow, Data Lakes, PySpark
- Medallion Architecture, Cloud Storage, Data Pipelines, Data Workflows, Scalable Storage
Soft Skills:
- Leadership
- Communication
- Problem-Solving
- Mentorship
- Data-Driven Mindset
- Fluent English
Phúc lợi dành cho bạn
- Opportunities for professional growth and development in a fast-growing startup
- Collaborative and inclusive work environment
- Flexible work arrangements
- Exciting projects and cutting-edge technologies
- Opportunity to make a significant impact in a startup ready to invest in top talent
Công ty
International Centre for Trade Transparency Limited
Ngành nghề
B2B Services
Quy mô công ty
100-499
Quốc tịch công ty
United-Kingdom