Senior Data Engineer

• Design, implement, and maintain scalable and reliable data pipelines and workflows using Python, Airflow, and PySpark
• Develop and optimize data lakes, leveraging Delta Lake and the medallion architecture for efficient and reliable storage
• Integrate and utilize various cloud storage solutions for seamless data management
• Collaborate with software engineering teams to ensure smooth integration of data pipelines with application components
• Mentor and guide junior data engineers, fostering best practices and a data-driven mindset
• Work with cross-functional teams to identify and implement data engineering solutions that support business objectives
• Drive the adoption of new technologies and methodologies to improve data pipeline performance and reliability
• Monitor and troubleshoot data pipeline issues, ensuring high data quality and availability
• Contribute to the development of data governance policies and procedures
• Stay up-to-date with the latest trends and best practices in data engineering

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 3+ years of professional experience in data engineering, with demonstrated capability to lead a team
• Strong proficiency in Python and experience with data engineering frameworks such as Airflow and PySpark
• Experience designing and implementing data lakes, preferably using Delta Lake and the medallion architecture
• Familiarity with various cloud storage solutions and their integration with data pipelines
• Knowledge of data modeling, data warehousing, and ETL/ELT processes
• Understanding of data quality, data governance, and data security best practices
• Excellent problem-solving and communication skills
• Fluent English

Technology:

• Python
• Airflow, Data Lakes, PySpark
• Medallion Architecture, Cloud Storage, Data Pipelines, Data Workflows, Scalable Storage

Soft Skills:

• Leadership
• Communication
• Problem-Solving
• Mentorship
• Data-Driven Mindset
• Fluent English