Technology Lead - Azure and Databricks Admin

InfosysCalgarySenior
Active

Job description

We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments. Required Qualifications:

We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.Databricks Platform AdministratorAzure DatabricksAWS and GCP Databricks environmentsRequired Qualifications:
  • Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
  • Candidate must be located within commuting distance of Calgary, Alberta or be willing to relocate to the areas.
Calgary, Alberta
  • Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
  • At least 4 years of Information Technology experience
Key Responsibilities Platform AdministrationKey Responsibilities Platform AdministrationKey ResponsibilitiesKey ResponsibilitiesKey ResponsibilitiesPlatform AdministrationPlatform AdministrationPlatform Administration
  • Administer and manage Databricks workspaces across development, test, and production environments.
Administer and manage Databricks workspaces across development, test, and production environments.Administer and manage Databricks workspaces across development, test, and production environments.Administer and manage Databricks workspaces across development, test, and production environments.Administer and manage Databricks workspaces across development, test, and production environments.
  • Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
  • Manage Unity Catalog implementation, governance, and access controls.
Manage Unity Catalog implementation, governance, and access controls.Manage Unity Catalog implementation, governance, and access controls.Manage Unity Catalog implementation, governance, and access controls.Manage Unity Catalog implementation, governance, and access controls.
  • Perform platform upgrades, maintenance, and health checks.
Perform platform upgrades, maintenance, and health checks.Perform platform upgrades, maintenance, and health checks.Perform platform upgrades, maintenance, and health checks.Perform platform upgrades, maintenance, and health checks.
  • Support multi-workspace and multi-region deployments.
Support multi-workspace and multi-region deployments.Support multi-workspace and multi-region deployments.Support multi-workspace and multi-region deployments.Support multi-workspace and multi-region deployments.Cluster ManagementCluster ManagementCluster ManagementCluster ManagementCluster Management
  • Create, configure, and manage all-purpose and job clusters.
Create, configure, and manage all-purpose and job clusters.Create, configure, and manage all-purpose and job clusters.Create, configure, and manage all-purpose and job clusters.Create, configure, and manage all-purpose and job clusters.
  • Implement cluster policies to control costs and ensure compliance.
Implement cluster policies to control costs and ensure compliance.Implement cluster policies to control costs and ensure compliance.Implement cluster policies to control costs and ensure compliance.Implement cluster policies to control costs and ensure compliance.
  • Optimize cluster sizing and auto-scaling configurations.
Optimize cluster sizing and auto-scaling configurations.Optimize cluster sizing and auto-scaling configurations.Optimize cluster sizing and auto-scaling configurations.Optimize cluster sizing and auto-scaling configurations.
  • Monitor cluster utilization, performance, and cost efficiency.
Monitor cluster utilization, performance, and cost efficiency.Monitor cluster utilization, performance, and cost efficiency.Monitor cluster utilization, performance, and cost efficiency.Monitor cluster utilization, performance, and cost efficiency.
  • Troubleshoot cluster startup, termination, and execution issues.
Troubleshoot cluster startup, termination, and execution issues.Troubleshoot cluster startup, termination, and execution issues.Troubleshoot cluster startup, termination, and execution issues.Troubleshoot cluster startup, termination, and execution issues.Security & GovernanceSecurity & GovernanceSecurity & GovernanceSecurity & GovernanceSecurity & Governance
  • Implement RBAC and workspace access controls.
Implement RBAC and workspace access controls.Implement RBAC and workspace access controls.Implement RBAC and workspace access controls.Implement RBAC and workspace access controls.
  • Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.
Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.
  • Manage service principals, managed identities, and secret scopes.
Manage service principals, managed identities, and secret scopes.Manage service principals, managed identities, and secret scopes.Manage service principals, managed identities, and secret scopes.Manage service principals, managed identities, and secret scopes.
  • Implement Unity Catalog governance and data security policies.
Implement Unity Catalog governance and data security policies.Implement Unity Catalog governance and data security policies.Implement Unity Catalog governance and data security policies.Implement Unity Catalog governance and data security policies.
  • Support audit logging, compliance, and platform security reviews.
Support audit logging, compliance, and platform security reviews.Support audit logging, compliance, and platform security reviews.Support audit logging, compliance, and platform security reviews.Support audit logging, compliance, and platform security reviews.Cloud IntegrationCloud IntegrationCloud IntegrationCloud IntegrationCloud Integration
  • Integrate Databricks with: Azure Data Lake Storage (ADLS)AWS S3Azure Key VaultAWS Secrets ManagerSQL DatabasesEvent HubKafkaSnowflakePower BI
Integrate Databricks with:Integrate Databricks with:Integrate Databricks with:Integrate Databricks with:
  • Azure Data Lake Storage (ADLS)
Azure Data Lake Storage (ADLS)Azure Data Lake Storage (ADLS)Azure Data Lake Storage (ADLS)Azure Data Lake Storage (ADLS)
  • AWS S3
AWS S3AWS S3AWS S3AWS S3
  • Azure Key Vault
Azure Key VaultAzure Key VaultAzure Key VaultAzure Key Vault
  • AWS Secrets Manager
AWS Secrets ManagerAWS Secrets ManagerAWS Secrets ManagerAWS Secrets Manager
  • SQL Databases
SQL DatabasesSQL DatabasesSQL DatabasesSQL Databases
  • Event Hub
Event HubEvent HubEvent HubEvent Hub
  • Kafka
KafkaKafkaKafkaKafka
  • Snowflake
SnowflakeSnowflakeSnowflakeSnowflake
  • Power BI
Power BIPower BIPower BIPower BIMonitoring & PerformanceMonitoring & PerformanceMonitoring & PerformanceMonitoring & PerformanceMonitoring & Performance
  • Monitor platform health and resource utilization.
Monitor platform health and resource utilization.Monitor platform health and resource utilization.Monitor platform health and resource utilization.Monitor platform health and resource utilization.
  • Establish proactive alerting and monitoring mechanisms.
Establish proactive alerting and monitoring mechanisms.Establish proactive alerting and monitoring mechanisms.Establish proactive alerting and monitoring mechanisms.Establish proactive alerting and monitoring mechanisms.
  • Analyze Spark job performance and recommend optimizations.
Analyze Spark job performance and recommend optimizations.Analyze Spark job performance and recommend optimizations.Analyze Spark job performance and recommend optimizations.Analyze Spark job performance and recommend optimizations.
  • Troubleshoot platform incidents and performance bottlenecks.
Troubleshoot platform incidents and performance bottlenecks.Troubleshoot platform incidents and performance bottlenecks.Troubleshoot platform incidents and performance bottlenecks.Troubleshoot platform incidents and performance bottlenecks.
  • Support SLA-driven production environments.
Support SLA-driven production environments.Support SLA-driven production environments.Support SLA-driven production environments.Support SLA-driven production environments.DevOps & AutomationDevOps & AutomationDevOps & AutomationDevOps & AutomationDevOps & Automation
  • Automate Databricks administration using: TerraformAzure DevOpsGitHub ActionsDatabricks CLIREST APIs
Automate Databricks administration using:Automate Databricks administration using:Automate Databricks administration using:Automate Databricks administration using:
  • Terraform
TerraformTerraformTerraformTerraform
  • Azure DevOps
Azure DevOpsAzure DevOpsAzure DevOpsAzure DevOps
  • GitHub Actions
GitHub ActionsGitHub ActionsGitHub ActionsGitHub Actions
  • Databricks CLI
Databricks CLIDatabricks CLIDatabricks CLIDatabricks CLI
  • REST APIs
REST APIsREST APIsREST APIsREST APIs
  • Manage CI/CD pipelines for Databricks deployments.
Manage CI/CD pipelines for Databricks deployments.Manage CI/CD pipelines for Databricks deployments.Manage CI/CD pipelines for Databricks deployments.Manage CI/CD pipelines for Databricks deployments.
  • Support Infrastructure as Code (IaC) initiatives.
Support Infrastructure as Code (IaC) initiatives.Support Infrastructure as Code (IaC) initiatives.Support Infrastructure as Code (IaC) initiatives.Support Infrastructure as Code (IaC) initiatives.Operational SupportOperational SupportOperational SupportOperational SupportOperational Support
  • Participate in incident, problem, and change management activities.
Participate in incident, problem, and change management activities.Participate in incident, problem, and change management activities.Participate in incident, problem, and change management activities.Participate in incident, problem, and change management activities.
  • Support patching, upgrades, certificate renewals, and maintenance activities.
Support patching, upgrades, certificate renewals, and maintenance activities.Support patching, upgrades, certificate renewals, and maintenance activities.Support patching, upgrades, certificate renewals, and maintenance activities.Support patching, upgrades, certificate renewals, and maintenance activities.
  • Create and maintain SOPs, runbooks, and technical documentation.
Create and maintain SOPs, runbooks, and technical documentation.Create and maintain SOPs, runbooks, and technical documentation.Create and maintain SOPs, runbooks, and technical documentation.Create and maintain SOPs, runbooks, and technical documentation.
  • Collaborate with engineering, infrastructure, security, and application teams.
Collaborate with engineering, infrastructure, security, and application teams.Collaborate with engineering, infrastructure, security, and application teams.Collaborate with engineering, infrastructure, security, and application teams.Collaborate with engineering, infrastructure, security, and application teams.Required SkillsDatabricksRequired SkillsDatabricksRequired SkillsRequired SkillsRequired SkillsDatabricksDatabricksDatabricks
  • Databricks Workspace Administration
Databricks Workspace AdministrationDatabricks Workspace AdministrationDatabricks Workspace AdministrationDatabricks Workspace Administration
  • Unity Catalog
Unity CatalogUnity CatalogUnity CatalogUnity Catalog
  • Cluster Management
Cluster ManagementCluster ManagementCluster ManagementCluster Management
  • Cluster Policies
Cluster PoliciesCluster PoliciesCluster PoliciesCluster Policies
  • Instance Pools
Instance PoolsInstance PoolsInstance PoolsInstance Pools
  • Jobs & Workflows
Jobs & WorkflowsJobs & WorkflowsJobs & WorkflowsJobs & Workflows
  • SQL Warehouses
SQL WarehousesSQL WarehousesSQL WarehousesSQL Warehouses
  • Databricks CLI
Databricks CLIDatabricks CLIDatabricks CLIDatabricks CLI
  • REST APIs
REST APIsREST APIsREST APIsREST APIs
  • Delta Lake
Delta LakeDelta LakeDelta LakeDelta LakeCloud PlatformsCloud PlatformsCloud PlatformsCloud PlatformsCloud Platforms
  • Azure Databricks (Preferred)
Azure Databricks (Preferred)Azure Databricks (Preferred)Azure Databricks (Preferred)Azure Databricks (Preferred)
  • AWS Databricks
AWS DatabricksAWS DatabricksAWS DatabricksAWS Databricks
  • GCP Databricks
GCP DatabricksGCP DatabricksGCP DatabricksGCP DatabricksPreferred Skill and ExperiencePreferred Skill and ExperiencePreferred Skill and ExperiencePreferred Skill and ExperiencePreferred Skill and Experience
  • Experience with enterprise-scale Azure Databricks environments.
Experience with enterprise-scale Azure Databricks environments.Experience with enterprise-scale Azure Databricks environments.Experience with enterprise-scale Azure Databricks environments.Experience with enterprise-scale Azure Databricks environments.
  • Strong understanding of data lake architecture and governance frameworks.
Strong understanding of data lake architecture and governance frameworks.Strong understanding of data lake architecture and governance frameworks.Strong understanding of data lake architecture and governance frameworks.Strong understanding of data lake architecture and governance frameworks.
  • Experience supporting production environments with strict SLA requirements.
Experience supporting production environments with strict SLA requirements.Experience supporting production environments with strict SLA requirements.Experience supporting production environments with strict SLA requirements.Experience supporting production environments with strict SLA requirements.Soft SkillsStrong analytical and troubleshooting skills.Excellent communication and stakeholder coordination.Ability to work in a global delivery model.Documentation and knowledge‑sharing mindset. ExperienceSoft SkillsStrong analytical and troubleshooting skills.Excellent communication and stakeholder coordination.Ability to work in a global delivery model.Documentation and knowledge‑sharing mindset. ExperienceSoft SkillsSoft SkillsSoft SkillsStrong analytical and troubleshooting skills.Strong analytical and troubleshooting skills.Excellent communication and stakeholder coordination.Excellent communication and stakeholder coordination.Ability to work in a global delivery model.Ability to work in a global delivery model.Documentation and knowledge‑sharing mindset.Documentation and knowledge‑‑sharing mindset.ExperienceExperienceExperience
  • 8-10 years of experience in IT Industry
8-10 years of experience in IT Industry8-10 years of experience in IT Industry8-10 years of experience in IT Industry8-10 years of experience in IT Industry
  • 5+ years of Databricks Administration experience.
5+ years of Databricks Administration experience.5+ years of Databricks Administration experience.5+ years of Databricks Administration experience.5+ years of Databricks Administration experience.
  • Prior experience in large enterprise or managed services engagements preferred.
Prior experience in large enterprise or managed services engagements preferred.Prior experience in large enterprise or managed services engagements preferred.Prior experience in large enterprise or managed services engagements preferred.Prior experience in large enterprise or managed services engagements preferred.
  • Exposure to Infosys‑scale delivery environments is an advantage.
Exposure to Infosys‑scale delivery environments is an advantage.Exposure to Infosys‑scale delivery environments is an advantage.Exposure to Infosys‑scale delivery environments is an advantage.Exposure to Infosys‑scale delivery environments is an advantage.Infosys‑scale delivery environmentsThe job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements

About Us Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem. EEO Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.

About UsEEO