Data Engineer

Adeva is a global talent network that enables work without boundaries by connecting tech professionals with top companies worldwide. 

We are looking for an experienced Data Engineer to join a long-term project focused on modernizing our client’s data infrastructure. You’ll play a pivotal role in a comprehensive migration to Databricks Unity Catalog and a full transition to Databricks-as-a-Service (DBXaaS) for a leading organization in the crypto industry. This project involves working on cutting-edge data platforms, building scalable pipelines, and implementing a unified governance model for data security and efficiency.

Responsibilities

• Lead the migration of data infrastructure to Databricks Unity Catalog and DBX-as-a-Service.
• Rebuild and optimize scalable data pipelines using Spark, SQL, and Python.
• Transform complex JSON structures into structured Databricks tables.
• Design and implement data models to support analytics and reporting needs.
• Develop Source to Target Mappings (STMs) and write efficient ETL code.
• Integrate data from diverse sources, ensuring quality, reliability, and scalability.
• Implement automated testing and monitoring to ensure data accuracy and pipeline health.
• Collaborate with cross-functional teams to gather requirements and align on data integration goals.
• Troubleshoot and optimize pipeline performance in a fast-paced, agile environment.

Requirements

• Expertise in Spark and SQL for data modeling, transformation, and performance tuning.
• Hands-on experience with Databricks and data governance within Unity Catalog.
• Proficiency in Python, especially for handling complex JSON and building robust pipelines.
• Strong experience with cloud platforms (Azure preferred; AWS or GCP acceptable).
• Knowledge of data orchestration tools such as Airflow.
• Familiarity with RDBMS and NoSQL technologies and their appropriate use cases.
• Solid foundation in software engineering practices, version control (e.g., Git), CI/CD, and testing frameworks.
• Experience with Data as Code methodologies and containerization using Docker (Kubernetes is a plus).

About Adeva

Adeva is an exclusive network of engineers, product and data professionals that connects consultants with leading enterprise organizations and startups. Our network is distributed all over the world, with engineers in more than 35 countries. Our company culture builds connections, careers, and employee growth. We are creating a workplace from the future that values flexibility, autonomy, and transparency. If that sounds like something you’d like to be part of, we’d love to hear from you.