Data Engineer

• Build, deploy, manage and have end-to-end ownership of the metrics platform infrastructure
• Building, scaling, and monitoring data services as well as performing root cause analysis investigations post incidents
• Work with the engineering team (DevOps) to explore and create new design/architectures geared towards scale and performance, including RESTful APIs
• Develop an in-depth understanding of the entire metrics lifecycle, including user journeys, and the interplay of the different tools and services in the metrics lifecycle
• Be a champion of consistent, high-quality metrics at Care and help drive the technical solutions and the organizations involved towards these goals.
• Work towards the democratization of metrics ownership enabling Carehealther from different tech families the define, create and publish and own relevant metrics for their respective products.
• Develop automation to automate the metrics lifecycle workflows such as ingestion, aggregation, ETL processing, certification, publication etc.
• Maintain and optimize the performance of our metrics platform infrastructure to ensure accurate, reliable and timely delivery of key insights for decision making.
• Deploy, scale and operate Modern high-performance real-time OLAP data stores such as Apache Pinot with a Solid understanding of distributed computing.
• Build a scalable and reliable data pipeline between our metrics store and data sources from streaming or batch data sources.
• Work on processing stream data on stream processing frameworks such as Apache Flink to generate real-time metrics and extract real-time insight to power Carehealth’s business.
• Design an architecture that bridges the real-time and offline data domains to provide a consistent view of metrics across all time spans from seconds to yearly time windows.
• Work with modern large-scale data systems such as pinot, Flink, Spark, Trino, Kafka and more.

The must-haves:

• 1-2 years of experience in leading small to medium-sized teams will be required
• 3+ years of experience in data engineering/data platform roles
• A deep passion for data and building high-quality and high-scale data platforms
• Experience designing and /or building high-performance, scalable data infrastructure.
• Have a user-centric mindset and truly care about building solutions that enable your peers and stakeholders to achieve greater heights.
• Write unit, functional and end-to-end tests consistently and thoroughly.
• Excited about working with new data technologies and discovering new and interesting solutions to the company’s data needs
• Excellent communication skills to communicate with the product development engineers to coordinate the development of data pipelines and or any new products features that can be built on top of the results of data analysis
• Knowledgeable of cloud systems like AWS, Azure, or Google Cloud Platform
• Good experience with programming languages like Python, Go, Scala, Java, or scripting languages like Bash.

Good-to-haves:

• A degree or higher in Computer Science, Electronics or Electrical Engineering, Software Engineering, Information Technology or other related technical disciplines.
• Good experience working on streaming data processing systems such as Kafka, Flink, Spark Streaming and others.
• Experience in handling large data sets (multiple PBs) and working with structured, unstructured and geographical datasets
• Good experience in handling big data within a distributed system and knowledge of data processing in distributed OLAP environments.
• Familiar with tools within the modern data ecosystem, such as Trino, Spark, Flink, Kafka, and others.
• Design and implement RESTful APIs and build and deploy performant modern web applications in React, NodeJS and TypeScript.
• Deep understanding of databases and best engineering practices - include handling and logging errors, monitoring the system, building human-fault-tolerant pipelines, understanding how to scale up, addressing continuous integration, knowledge of database administration, maintaining data cleaning and ensuring a deterministic pipeline.