Specialist - Data Engineering

Role description

Role Summary

Location-

Seeking an experienced Azure Databricks Engineer to design develop optimize and support scalable cloudbased data platforms using Azure Data Services The candidate will be responsible for building highperformance data pipelines implementing data engineering best practices and driving data modernization initiatives

Key Responsibilities-

Design and implement scalable ETLELT pipelines using Azure Databricks and Azure Data Factory

Develop data processing solutions using PySpark Spark SQL and Python

Build and maintain Delta Lake architectures and Databricks workflows

Optimize performance of Spark jobs clusters and notebooks

Implement CICD pipelines using Azure DevOps for Databricks artifacts

Work with ADLS Gen2 Azure Synapse and Azure SQL for enterprise data solutions

Ensure data quality governance security and compliance standards

Monitor troubleshoot and support production data pipelines

Collaborate with architects business stakeholders and crossfunctional teams

Mentor junior engineers and participate in code reviews