Cloud Principal Hardware Engineer

MicrosoftAliso Viejo, CA,US

Full-time

Posted 13 hours agoStill open, checked 2h ago

About the job

Lead hands-on failure analysis across server components: characterize failure modes from fleet telemetry and logs and confirm root cause with repair results and supplier failure analysis. Define the signals, thresholds, targeted diagnostics and repair recipes, and work with component teams for the root cause analysis. Improve isolation accuracy so each repair address the source of failures: improve efficacy of the detection and repair services. Partner with software engineers and data scientists on detection rules and prediction models; set precision targets and validation gates before a rule or model takes action on production nodes. Use AI agents and machine learning to automate failure analysis, live-site triage and supplier RCA follow-up, and to shorten the time from a new failure signature to a detection rule. Lead root cause and corrective actions with server components, including firmware fixes, and feed the learnings back into the platform. Lead as the senior hardware escalation point for live-site incidents, from triage through root cause to fleet-wide mitigation. Make new server generations and AI/GPU platforms platform-ready before they reach production, with detection, isolation and repair coverage at launch. Track and improve metrics & mentor engineers across the team. Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 7+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 8+ years technical engineering experience OR equivalent experience 6+ years of hands-on failure analysis, debug or reliability engineering for server or datacenter hardware, covering two or more server component types 3+ years leading root cause analysis and corrective actions with hardware suppliers, OEMs or ODMs 2+ years using AI or machine learning in hardware engineering work, such as failure analysis, telemetry analysis or automation 10+ years in server hardware reliability, failure analysis or fleet resiliency at cloud or datacenter scale Technical depth in the failure modes of at least one server component type, with working knowledge across the rest of the server (BMC, PCIe, firmware) Experience turning failure signatures into automated detection rules, fault codes or repair procedures for a large fleet Experience with predictive maintenance or anomaly detection on hardware telemetry, including setting precision targets and automated repair actions Proficiency with Kusto (Azure Data Explorer) or SQL, and Python or PowerShell, for large-scale telemetry analysis Experience with NPI and hardware qualification for server platforms Clear written and verbal communication, including presenting technical risk and trade-offs to leadership