- Work with stakeholders, product owner, cross-functional team to understand their data requirements and develop solutions that meet their needs and optimal solutions.
- Design, develop, and maintain scalable and efficient data pipelines and infrastructure
- Implement best practices for data modeling, data warehousing, and ETL processes
- Process, clean, and verify the integrity of data used for analysis
- Monitor data pipelines to ensure data quality and performance
- Implement best practices for data security and compliance
- Communicate and collaborate with software engineers, AI engineer to ensure seamless integration of data infrastructure with other systems
For AWS experience, candidates:
- Design, implement and maintain scalable data pipelines that can handle largeamounts of data using AWS services such as S3, EMR, Glue, Lambda, Athena,and Redshift
- Develop and maintain data warehouse and data lake solutions using AWS services