Senior Databricks Engineer

Adeva is a global talent network that enables work without boundaries by connecting tech professionals with top companies worldwide. 

We are seeking highly skilled and motivated Senior Data Engineers with expertise in Databricks and Azure. As a Senior Data Engineer, you will be responsible for implementing domain-specific use cases within the Medallion architecture lakehouse environment. Working with data that is already available in the Bronze layer, you will focus on transforming and refining data through the Silver layer and building high-quality data products and data marts in the Gold layer. You will work closely with business intelligence development teams, domain stakeholders, and analytics teams to ensure efficient data flow and enable data-driven decision-making across critical business domains.

Responsibilities

• Collaborate with the Product Owner, Business Analyst and other team members to understand domain-specific data requirements and design scalable data pipelines and architectures
• Transform and enrich data from the Bronze layer (raw data) into the Silver layer by performing data quality checks, cleansing, validation, standardization, and modeling activities
• Build and maintain data products and data marts in the Gold layer that are curated, business-ready, and optimized for analytics and reporting purposes
• Develop efficient ETL/ELT workflows to extract, transform, and load data from various sources through the medallion architecture layers
• Optimize and fine-tune data pipelines for performance, reliability, and scalability using PySpark and Databricks
• Implement comprehensive data quality checks and monitoring to ensure data accuracy, consistency, and integrity across all layers
• Develop Power BI reports and dashboards that leverage the Gold layer data products to deliver business insights
• Work with BI developers and data analysts to provide them with the necessary data infrastructure and tools for analysis and reporting
• Troubleshoot and resolve data-related issues, including performance bottlenecks and data inconsistencies
• Stay up to date with the latest trends and technologies in data engineering and recommend improvements to existing systems and processes
• Document data engineering processes, data flows, and system configurations 

Domain-Specific Responsibilities:

• Sales Domain:
• Finance Domain:
• Project Business Domain:

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field
• 5+ years of experience as a Data Engineer on Databricks and Azure, with a focus on designing and building data pipelines within a medallion architecture (Bronze, Silver, Gold layers)
• Strong problem-solving and analytical skills
• Effective communication in English and collaboration in agile environments
• Experience working in global, multicultural teams

Technology Must-Haves:

• PySpark and SparkSQL: Advanced proficiency in PySpark and SparkSQL for large-scale data transformations and processing
• Databricks: Hands-on experience with Databricks workspace, Delta Lake, workflows, and notebooks
• Power BI: Proven experience developing interactive reports and dashboards in Power BI
• Strong programming skills in SQL and Python (Scala knowledge is a plus)
• Hands-on experience with Azure services such as Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure SQL, Synapse Analytics is an advantage
• Proficiency in data modeling, database design, and SQL optimization
• Solid understanding of data integration patterns, ETL/ELT best practices, cloud computing, and security principles 

Domain-specific Requirements:

You should have demonstrable experience in at least one of the following domains:

• Sales: Experience working on Sales projects such as Sales Controlling, Sales Reporting, revenue analysis, or similar.
• Finance: Understanding of financial concepts, terminology, and experience implementing Financial Reporting solutions
• Project Business: Knowledge of project management metrics, including margins, invoiced/billable hours, profitability, contribution margin (Deckungsbeitrag), billability, and project controlling 

Nice to Haves:

• Experience with Unity Catalog for data governance, permissions management, and secure data sharing
• Familiarity with big data frameworks like Hadoop, Spark, and Hive
• Certifications in Databricks or Azure services
• Experience with data streaming technologies such as Apache Kafka or Azure Event Hubs 

About Adeva

Adeva is an exclusive network of engineers, product and data professionals that connects consultants with leading enterprise organizations and startups. Our network is distributed all over the world, with engineers in more than 35 countries. Our company culture builds connections, careers, and employee growth. We are creating a workplace from the future that values flexibility, autonomy, and transparency. If that sounds like something you’d like to be part of, we’d love to hear from you.