M01 - Cloud Logging & Data Platform Engineer

Overview We are looking for a Logging & Data Platform Engineer to design, build, and operate the logging and operational data-platform capabilities supporting the future SSOE platform. You will build the platform that collects, transports, stores, indexes, searches, and serves operational data across MOE's technology environment, spanning on-premise infrastructure, networks, applications, GCC, AWS, Azure, and hybrid environments. What You Will Be Working On As a Logging & Data Platform Engineer, you will build the shared platform capabilities that enable engineering and operations teams to reliably collect and use operational data at scale.You will work across logging, telemetry ingestion, data movement, storage, search, retention, and platform integration.The role requires an engineer who understands both traditional enterprise infrastructure and modern cloud-native architectures and can design solutions that work securely and reliably across environment boundaries.You will work closely with the Observability Engineer on telemetry requirements and with Data Engineering & Analytics on shared data-platform capabilities and integration patterns. Key Responsibilities Logging Platform Engineering Design, build, and operate logging capabilities across on-premise infrastructure, networks, applications, cloud platforms, and hybrid environments Collect logs from servers, network devices, applications, containers, databases, security appliances, cloud services, and other infrastructure sources Build scalable log ingestion, routing, enrichment, storage, indexing, search, and retrieval capabilities Define structured logging standards, schemas, metadata, tagging, and correlation conventions across services Design appropriate retention, archival, lifecycle, and deletion policies for different classes of operational data Support correlation between logs, metrics, events, and traces using common identifiers and telemetry standards Work with the Observability Engineer to implement platform capabilities supporting end-to-end service observability Data Collection & Integration Design secure and resilient data movement between on-premise environments, GCC, and approved external services Implement collection and forwarding patterns appropriate to different infrastructure, application, network, and security environments Design for intermittent connectivity, network constraints, buffering, retry, back-pressure, and recovery between environments Build event-driven and streaming patterns for moving operational data between producers and consumers Integrate legacy and enterprise systems with modern cloud-native platform capabilities Define clear interfaces and integration patterns between logging, observability, data engineering, and application platforms Data Platform Engineering Build shared platform capabilities for ingesting, storing, processing, querying, and serving operational data Design scalable storage and query architectures appropriate to data volume, access patterns, retention requirements, and cost Build ingestion, filtering, enrichment, and transformation pipelines for operational data Provide APIs, query interfaces, or other serving mechanisms for authorised downstream consumers Support operational datasets consumed by the User Portal, dashboards, reporting, automation, and Data Engineering & Analytics Define schemas and data contracts for shared platform interfaces Ensure platform changes remain backwards compatible or are coordinated with downstream consumers Cloud & Platform Engineering Design solutions using cloud-native logging, streaming, storage, search, and data capabilities Build infrastructure and platform configuration using Infrastructure as Code Automate build, test, deployment, configuration, and platform changes through CI/CD Design for scalability, resilience, high availability, recoverability, and operational simplicity Monitor platform capacity, performance, reliability, and cost Security & Governance Enforce MOE and Government data-classification requirements Design secure routing and storage of operational data across security zones and environment boundaries Apply appropriate encryption, access controls, authentication, and authorisation Ensure logging pipelines do not unnecessarily expose credentials, secrets, or sensitive information Implement audit-trail preservation and appropriate retention controls Ensure data-residency requirements are considered when routing operational data between on-premise, GCC, cloud, and SaaS environments Participate in security, architecture, and operational-readiness reviews Reliability & Operations Define SLOs and operational health indicators for logging and data-platform services Build monitoring, alerting, failure detection, retry, and recovery into platform components Monitor ingestion health, processing latency, data loss, storage utilisation, search performance, and platform availability Participate in incident investigation, root-cause analysis, and post-incident reviews Participate in operational support and on-call responsibilities for owned services Maintain architecture documentation, operational procedures, and runbooks What We Are Looking For Experience Minimum 3–5 years of experience in cloud engineering, platform engineering, DevOps, SRE, logging engineering, data platform engineering, or a related discipline At least 2 years of hands-on experience building or operating production logging, telemetry, or data-platform capabilities Demonstrated experience working with AWS and/or Azure cloud-native services Experience integrating on-premise and cloud environments, or operating systems in a hybrid environment Experience with production data ingestion, streaming, routing, storage, indexing, or search platforms Experience implementing Infrastructure as Code and CI/CD for production environments Experience designing systems for scalability, resilience, security, and operational support

Overview

Overview

We are looking for a Logging & Data Platform Engineer to design, build, and operate the logging and operational data-platform capabilities supporting the future SSOE platform.

You will build the platform that collects, transports, stores, indexes, searches, and serves operational data across MOE's technology environment, spanning on-premise infrastructure, networks, applications, GCC, AWS, Azure, and hybrid environments.

What You Will Be Working On

What You Will Be Working On

• As a Logging & Data Platform Engineer, you will build the shared platform capabilities that enable engineering and operations teams to reliably collect and use operational data at scale.
• You will work across logging, telemetry ingestion, data movement, storage, search, retention, and platform integration.
• The role requires an engineer who understands both traditional enterprise infrastructure and modern cloud-native architectures and can design solutions that work securely and reliably across environment boundaries.
• You will work closely with the Observability Engineer on telemetry requirements and with Data Engineering & Analytics on shared data-platform capabilities and integration patterns.

Key Responsibilities

Key Responsibilities

Logging Platform Engineering

• Design, build, and operate logging capabilities across on-premise infrastructure, networks, applications, cloud platforms, and hybrid environments
• Collect logs from servers, network devices, applications, containers, databases, security appliances, cloud services, and other infrastructure sources
• Build scalable log ingestion, routing, enrichment, storage, indexing, search, and retrieval capabilities
• Define structured logging standards, schemas, metadata, tagging, and correlation conventions across services
• Design appropriate retention, archival, lifecycle, and deletion policies for different classes of operational data
• Support correlation between logs, metrics, events, and traces using common identifiers and telemetry standards
• Work with the Observability Engineer to implement platform capabilities supporting end-to-end service observability

Data Collection & Integration

• Design secure and resilient data movement between on-premise environments, GCC, and approved external services
• Implement collection and forwarding patterns appropriate to different infrastructure, application, network, and security environments
• Design for intermittent connectivity, network constraints, buffering, retry, back-pressure, and recovery between environments
• Build event-driven and streaming patterns for moving operational data between producers and consumers
• Integrate legacy and enterprise systems with modern cloud-native platform capabilities
• Define clear interfaces and integration patterns between logging, observability, data engineering, and application platforms

Data Platform Engineering

• Build shared platform capabilities for ingesting, storing, processing, querying, and serving operational data
• Design scalable storage and query architectures appropriate to data volume, access patterns, retention requirements, and cost
• Build ingestion, filtering, enrichment, and transformation pipelines for operational data
• Provide APIs, query interfaces, or other serving mechanisms for authorised downstream consumers
• Support operational datasets consumed by the User Portal, dashboards, reporting, automation, and Data Engineering & Analytics
• Define schemas and data contracts for shared platform interfaces
• Ensure platform changes remain backwards compatible or are coordinated with downstream consumers

Cloud & Platform Engineering

• Design solutions using cloud-native logging, streaming, storage, search, and data capabilities
• Build infrastructure and platform configuration using Infrastructure as Code
• Automate build, test, deployment, configuration, and platform changes through CI/CD
• Design for scalability, resilience, high availability, recoverability, and operational simplicity
• Monitor platform capacity, performance, reliability, and cost

Security & Governance

• Enforce MOE and Government data-classification requirements
• Design secure routing and storage of operational data across security zones and environment boundaries
• Apply appropriate encryption, access controls, authentication, and authorisation
• Ensure logging pipelines do not unnecessarily expose credentials, secrets, or sensitive information
• Implement audit-trail preservation and appropriate retention controls
• Ensure data-residency requirements are considered when routing operational data between on-premise, GCC, cloud, and SaaS environments
• Participate in security, architecture, and operational-readiness reviews

Reliability & Operations

• Define SLOs and operational health indicators for logging and data-platform services
• Build monitoring, alerting, failure detection, retry, and recovery into platform components
• Monitor ingestion health, processing latency, data loss, storage utilisation, search performance, and platform availability
• Participate in incident investigation, root-cause analysis, and post-incident reviews
• Participate in operational support and on-call responsibilities for owned services
• Maintain architecture documentation, operational procedures, and runbooks

What We Are Looking For

What We Are Looking For

Experience

• Minimum 3–5 years of experience in cloud engineering, platform engineering, DevOps, SRE, logging engineering, data platform engineering, or a related discipline
• At least 2 years of hands-on experience building or operating production logging, telemetry, or data-platform capabilities
• Demonstrated experience working with AWS and/or Azure cloud-native services
• Experience integrating on-premise and cloud environments, or operating systems in a hybrid environment
• Experience with production data ingestion, streaming, routing, storage, indexing, or search platforms
• Experience implementing Infrastructure as Code and CI/CD for production environments
• Experience designing systems for scalability, resilience, security, and operational support