Expert, Software Engineering (Data Operations)

Job Purpose:

The Data Ops Engineer is responsible for day-to-day technical development and delivery of data pipelines into data platform.

You will ensure delivery of solutions based on the backbone of good architecture, best data engineering practices around operational efficiencies, security, reliability, performance and cost optimization. The key focus of the role is:

• Design, build and optimize data engineering pipelines to extract data from different sources and applications and feed into cloud data platform
• Work closely with Data & BI team in building, testing and producing data extraction, transformation and reporting solutions within cloud platform
• Support Data & BI team with BAU activities when required
• Provide accurate and timely information that can be used in day to day operational and strategic decision making

The Data Ops Engineer is responsible for day-to-day technical development and delivery of data pipelines into data platform.

You will ensure delivery of solutions based on the backbone of good architecture, best data engineering practices around operational efficiencies, security, reliability, performance and cost optimization. The key focus of the role is:

• Design, build and optimize data engineering pipelines to extract data from different sources and applications and feed into cloud data platform
• Work closely with Data & BI team in building, testing and producing data extraction, transformation and reporting solutions within cloud platform
• Support Data & BI team with BAU activities when required
• Provide accurate and timely information that can be used in day to day operational and strategic decision making

Success Profile - Qualification and Experiences Qualifications:

• Graduated from university majoring in IT or electronics and telecommunications.

Work Experience: Essential:

• Relevant work experience at Data Ops positions
• Hands on experience in building ETL/ELT solutions for large scale data pipelines within AWS cloud
• Hands on experience on SQL Server and Snowflake data warehouses
• Hands on experience in data processing (using Spark, Python) for cloud data platforms, scheduling and monitoring of ETL/ELT jobs in AWS Glue and Lambda.
• Hands on experience in solution architecture, data ingestion, query optimisation, data segregation, ETL, ELT, CI/CD framework.
• Experience using Data Ops to develop data flows and the continuous use of data
• Experience with cloud-based technologies within AWS specifically
• Experience in developing technical and support documentation, translate business requirements and needs into reporting and models