Sr. Data Engineer

Sr. Client Partner in San FranciscoPune City, Indiafull timeSenior
Active

Job description

We are seeking a highly skilled Senior Data Engineer with 7–8 years of experience to design, develop, and optimize data pipelines and solutions. The ideal candidate will have expertise in PySpark, SQL, and Python, with solid knowledge of Azure Databricks. A strong understanding of agile methodologies will be an added advantage.Key Responsibilities:Design, build, and maintain scalable data pipelines and ETL processes.Work with large datasets to ensure high data quality, accuracy, and performance.Develop data solutions leveraging Azure Databricks, PySpark, and SQL.Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.Troubleshoot and optimize data workflows to improve reliability and efficiency.Ensure compliance with best practices in data governance, security, and performance.Contribute to agile ceremonies and work in a collaborative, fast-paced environment.Required Skills & Qualifications:7–8 years of experience in Data Engineering.Strong programming skills in Python and SQL.Hands-on experience with PySpark for large-scale data processing.Good working knowledge of Azure Databricks.Strong problem-solving and debugging skills.Experience in designing and optimizing ETL pipelines.Knowledge of Agile methodologies (Scrum/Kanban) is a plus.Nice to Have:Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.Exposure to CI/CD practices and version control systems (e.g., Git).Experience working in a cloud-first data ecosystem.We are seeking a highly skilled Senior Data Engineer with 7–8 years of experience to design, develop, and optimize data pipelines and solutions. The ideal candidate will have expertise in PySpark, SQL, and Python, with solid knowledge of Azure Databricks. A strong understanding of agile methodologies will be an added advantage.Key Responsibilities:Design, build, and maintain scalable data pipelines and ETL processes.Work with large datasets to ensure high data quality, accuracy, and performance.Develop data solutions leveraging Azure Databricks, PySpark, and SQL.Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.Troubleshoot and optimize data workflows to improve reliability and efficiency.Ensure compliance with best practices in data governance, security, and performance.Contribute to agile ceremonies and work in a collaborative, fast-paced environment.Required Skills & Qualifications:7–8 years of experience in Data Engineering.Strong programming skills in Python and SQL.Hands-on experience with PySpark for large-scale data processing.Good working knowledge of Azure Databricks.Strong problem-solving and debugging skills.Experience in designing and optimizing ETL pipelines.Knowledge of Agile methodologies (Scrum/Kanban) is a plus.Nice to Have:Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.Exposure to CI/CD practices and version control systems (e.g., Git).Experience working in a cloud-first data ecosystem.We are seeking a highly skilled Senior Data Engineer with 7–8 years of experience to design, develop, and optimize data pipelines and solutions. The ideal candidate will have expertise in PySpark, SQL, and Python, with solid knowledge of Azure Databricks. A strong understanding of agile methodologies will be an added advantage.

Key Responsibilities:

Key Responsibilities:
  • Design, build, and maintain scalable data pipelines and ETL processes.

Design, build, and maintain scalable data pipelines and ETL processes.

  • Work with large datasets to ensure high data quality, accuracy, and performance.

Work with large datasets to ensure high data quality, accuracy, and performance.

  • Develop data solutions leveraging Azure Databricks, PySpark, and SQL.

Develop data solutions leveraging Azure Databricks, PySpark, and SQL.

  • Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.

Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.

  • Troubleshoot and optimize data workflows to improve reliability and efficiency.

Troubleshoot and optimize data workflows to improve reliability and efficiency.

  • Ensure compliance with best practices in data governance, security, and performance.

Ensure compliance with best practices in data governance, security, and performance.

  • Contribute to agile ceremonies and work in a collaborative, fast-paced environment.

Contribute to agile ceremonies and work in a collaborative, fast-paced environment.

Required Skills & Qualifications:

Required Skills & Qualifications:
  • 7–8 years of experience in Data Engineering.

7–8 years of experience in Data Engineering.

  • Strong programming skills in Python and SQL.

Strong programming skills in Python and SQL.

  • Hands-on experience with PySpark for large-scale data processing.

Hands-on experience with PySpark for large-scale data processing.

  • Good working knowledge of Azure Databricks.

Good working knowledge of Azure Databricks.

  • Strong problem-solving and debugging skills.

Strong problem-solving and debugging skills.

  • Experience in designing and optimizing ETL pipelines.

Experience in designing and optimizing ETL pipelines.

  • Knowledge of Agile methodologies (Scrum/Kanban) is a plus.

Knowledge of Agile methodologies (Scrum/Kanban) is a plus.

Nice to Have:

Nice to Have:
  • Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.

Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.

  • Exposure to CI/CD practices and version control systems (e.g., Git).

Exposure to CI/CD practices and version control systems (e.g., Git).

  • Experience working in a cloud-first data ecosystem.

Experience working in a cloud-first data ecosystem.

I'm interested{{criteriaIndex}} {{field}} {{captialize(condition)}} ( ) {{cxPropHelpText}} {{cxPropField.consent_details.description}} {{record.Posting_Title}} {{record.Posting_Title}} {{record.Posting_Title}} {{candidate.Email}} {{topMessage}} Job Details {{topMessage}} previous next {{title}} {{score}}% Job Description {{unescape(sanitizeHTML(descriptionHTML))}} {{message}} Step {{curStepInMandatorySecPrompt}}/{{totalNumOfStepsInMandatorySecPrompt}} For privacy and security purposes, please go through the following points and provide consent. Accept Decline {{getI18n("zr.careers.publicpage.meta.joblisting")}} {{getI18n("zr.careers.publicpage.meta.joblisting")}} {{cxPropAssessment.name}} {{getI18n("zr.questionnaire.question.information.heading")}} {{getI18n("zr.cw.autofill.head")}} {{cxPropBodyMessage}} {{getI18n("zr.careers.autopopulate.success")}} {{getI18n("zr.cw.apply.sign",cxPropCompanyInfo.name)}} {{if(cxPropDetails.applyWithSeekButton,cxPropDetails.applyWithSeekButton.buttonLabel,getI18n("zr.quickapply.apply.toggle","Seek"))}} {{getI18n('zr.eeo.questionnaire.portal.maintitle')}} {{botName}} {{getI18n("zr.zia.sb.consent.msg1")}}{{criteriaIndex}}{{criteriaIndex}}{{field}} {{captialize(condition)}}{{field}}{{captialize(condition)}}( )(){{cxPropHelpText}}{{cxPropHelpText}}{{cxPropField.consent_details.description}}

{{cxPropField.consent_details.description}}

{{record.Posting_Title}}{{record.Posting_Title}}{{record.Posting_Title}}

{{record.Posting_Title}}

{{record.Posting_Title}}{{record.Posting_Title}}

{{record.Posting_Title}}

{{record.Posting_Title}}
  • {{record.Posting_Title}}

{{record.Posting_Title}}

{{record.Posting_Title}}{{candidate.Email}}{{candidate.Email}}{{candidate.Email}}

{{candidate.Email}}

{{topMessage}}Job DetailsJob DetailsJob DetailsJob Details{{topMessage}}previousnext{{title}} {{score}}% Job Description {{unescape(sanitizeHTML(descriptionHTML))}}{{title}} {{score}}% Job Description {{unescape(sanitizeHTML(descriptionHTML))}}{{title}} {{score}}%

{{title}}

{{title}}{{score}}%Job Description {{unescape(sanitizeHTML(descriptionHTML))}}

Job Description

{{message}}

Step {{curStepInMandatorySecPrompt}}/{{totalNumOfStepsInMandatorySecPrompt}}

For privacy and security purposes, please go through the following points and provide consent.Accept DeclineAccept DeclineAcceptDecline{{getI18n("zr.careers.publicpage.meta.joblisting")}}{{getI18n("zr.careers.publicpage.meta.joblisting")}}
  • {{getI18n("zr.careers.publicpage.meta.joblisting")}}
{{getI18n("zr.careers.publicpage.meta.joblisting")}}{{getI18n("zr.careers.publicpage.meta.joblisting")}}{{getI18n("zr.careers.publicpage.meta.joblisting")}}
  • {{getI18n("zr.careers.publicpage.meta.joblisting")}}
{{getI18n("zr.careers.publicpage.meta.joblisting")}}{{cxPropAssessment.name}} {{getI18n("zr.questionnaire.question.information.heading")}}{{cxPropAssessment.name}} {{getI18n("zr.questionnaire.question.information.heading")}}{{cxPropAssessment.name}}

{{getI18n("zr.questionnaire.question.information.heading")}}

{{getI18n("zr.cw.autofill.head")}} {{cxPropBodyMessage}} {{getI18n("zr.careers.autopopulate.success")}}

{{getI18n("zr.cw.autofill.head")}}

{{cxPropBodyMessage}}

{{getI18n("zr.careers.autopopulate.success")}}

{{getI18n("zr.cw.apply.sign",cxPropCompanyInfo.name)}}

{{getI18n('zr.eeo.questionnaire.portal.maintitle')}}{{botName}}{{botName}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}{{getI18n("zr.zia.sb.consent.msg1")}}

Similar jobs