Data Infrastructure Engineer

Adeva is a global talent network that enables work without boundaries by connecting tech professionals with top companies worldwide. 

We are looking for a Data Infrastructure Engineer to work closely with our data team, infrastructure team, and development group to maintain and optimize our data infrastructure, with a focus on machine learning, data pipelines, orchestration, and real-time streaming using Kafka/MSK or DMS. You will be responsible for building, configuring, maintaining, and monitoring our data infrastructure, while utilizing various technologies.

Responsibilities

• Design, build, and optimize data pipelines for collecting, transforming, and loading data from various sources into our data warehouse and data lake.
• Implement and maintain a real-time data streaming infrastructure using Kafka/MSK or DMS.
• Develop and maintain orchestration and scheduling tools to automate the execution of data pipelines and machine learning workflows.
• Collaborate with the data team to design and implement scalable and efficient machine learning architectures and workflows.
• Configure and maintain infrastructure elements using configuration tools such as Ansible and Chef.
• Implement monitoring for all aspects of the data infrastructure and improve the current monitoring of the core product infrastructure.
• Ensure continuous documentation of current processes (configuration standards, setups, diagrams, etc.).
• Participate in on-call rotations.

Requirements

• 5+ years of experience in building and maintaining large-scale data infrastructure, with a focus on machine learning, data pipelines, orchestration, and real-time streaming.
• Strong experience with AWS cloud infrastructure and related services, such as S3, EC2, EMR, and Lambda.
• Experience in designing and implementing scalable machine learning architectures and workflows.
• Experience with real-time data streaming technologies, such as Kafka and MSK or DMS.
• Strong understanding of data orchestration and scheduling tools, such as Prefect and Airflow.
• Experience with containerization technologies, such as Docker and Kubernetes.
• Strong knowledge of SQL and NoSQL databases, such as MySQL, Postgres, and MongoDB.
• Experience in implementing infrastructure as code using tools like Terraform and CloudFormation.
• Experience with monitoring and logging tools, such as Datadog and ELK.
• Experience in configuring and maintaining infrastructure elements using configuration tools such as Ansible and Chef.
• Fluent English and excellent communication skills.
• Ability to work as part of an international, distributed team and resolve potential issues and challenges that come with remote work.

About Adeva

Adeva is an exclusive tech professionals' network that connects some of the best tech professionals in the world with leading enterprise organizations and startups. Our network is distributed all over the world, with engineers in more than 35 countries. Our company culture builds connections, careers, and employee growth. We are creating a workplace from the future that values flexibility, autonomy, and transparency. If that sounds like something you’d like to be part of, we’d love to hear from you.