Data Platforms Senior Engineer – Databricks

Overview

The Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teamsThe Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teamsThe Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teams

The Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teams

The Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teams

Responsibilities

Design, develop, and maintain scalable data pipelines and ETL/ELT processes.Develop and optimize data processing solutions using Databricks and Apache Spark.Build data pipelines using PySpark, Python, and SQL.Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.Implement incremental data processing and performance optimization techniques.Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.Design, develop, and maintain scalable data pipelines and ETL/ELT processes.Develop and optimize data processing solutions using Databricks and Apache Spark.Build data pipelines using PySpark, Python, and SQL.Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.Implement incremental data processing and performance optimization techniques.Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.Design, develop, and maintain scalable data pipelines and ETL/ELT processes.Develop and optimize data processing solutions using Databricks and Apache Spark.Build data pipelines using PySpark, Python, and SQL.Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.Implement incremental data processing and performance optimization techniques.Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Develop and optimize data processing solutions using Databricks and Apache Spark.
• Build data pipelines using PySpark, Python, and SQL.
• Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.
• Implement incremental data processing and performance optimization techniques.
• Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.
• Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.

Qualifications

Bachelor’s degree in computer science, Software Engineering, Information Systems, or a related field.Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.Strong handson experience with Databricks platform.Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing largeExperience with version control systems (e.g., Git).Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them into operational requirementsBachelor’s degree in computer science, Software Engineering, Information Systems, or a related field.Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.Strong handson experience with Databricks platform.Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing largeExperience with version control systems (e.g., Git).Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them into operational requirementsBachelor’s degree in computer science, Software Engineering, Information Systems, or a related field.Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.Strong handson experience with Databricks platform.Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing largeExperience with version control systems (e.g., Git).Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them into operational requirements

• Bachelor’s degree in computer science, Software Engineering, Information Systems, or a related field.
• Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.
• Strong handson experience with Databricks platform.
• Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).

Expert proficiency in SQL and Python

• Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).

SSIS, Azure Data Factory, Azure DataBricks

• Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).

data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies)

• Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing large

Big Data technologies

• Experience with version control systems (e.g., Git).
• Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them into operational requirements