Data Engineering Manager

• All Levels (scope and complexity will increase from single product and domain to multiple products and/or domains).
• Structure end-to-end processes of data extraction, transformation, and storage to deploy models and analytical solutions in real-time and offline analytic processing.
• Handle activities such as quality assurance, data migration and integration, and solution deployment to ensure the business gets the best value.
• Implement, maintain, and further develop ETL processes, data lineage, and observability across large-scale data ingestion from a variety of sources.
• Design and implement data flows to ensure efficient and reliable data movement (e.g., unit and integration tests).
• Provide technical support in understanding business problems and designing smart data products.
• Collaborate within an agile culture, contributing to the team's success.
• Maintain strong relationships with stakeholders to ensure collaboration and delivery of objectives (Relationship creation and management of broader team’s stakeholders to ensure strong connection and collaboration for team).
• Continued self-development and research of best-in-class analytical approached (management and/or development and capability building of Junior/Senior Data Engineers).
• Potential responsibility for selection of AA Vendors and AA Tooling workbench (Procurement & Enablement of tools sits with GIS).

Qualifications:

• BA/MSc in Statistics, Mathematics, or a related field; equivalent work experience accepted. (MSc/PhD)
• 3+ years in a pure analytical/science role (5+ years)

Experience / Skills (all levels): 

• Excellent written and verbal communication skills.
• Significant commercial experience in a similar position.
• Proficiency with Python (inc. strong data analytics skills using Python).
• Excellent software engineering skills (including unit testing, integration testing, OOP).
• Experience working with large datasets through Spark and RDBMS.
• Solid knowledge of PySpark with the ability to apply it to write Spark applications as well as analyze data in a distributed environment.
• Experience working with a large development team (CI/CD).
• Very good SQL skills and ability to extract information from databases.
• Ability to build and maintain strong collaborative relationships with stakeholders across the business.
• Demonstrated ability to effectively manage multiple simultaneous projects and deliverables.
• Communicate effectively with various stakeholders, tailoring your message to your audience's level of technical understanding

Nice to have:

• Experience with Azure development environment
• Experience in building/releasing Infrastructure as Code (e.g., Terraform).
• Experience with containerization technologies and orchestrators (e.g., Docker and Kubernetes).
• Understanding of machine learning models, deployment, and monitoring.
• Interest in machine learning and/or desire to develop skills.
• Experience working in agile organizations.
• Excellent written and verbal English.