DeCaf: Diagnosing & Triaging Performance Issues in Large-Scale Cloud Services
Large scale cloud services use Key Performance Indicators (KPIs) for tracking and monitoring performance. They usually have Service Level Objectives (SLOs) baked into the customer agreements which are tied to these KPIs. Dependency failures, code bugs, infrastructure failures, and other problems can cause performance regressions. It is critical to minimize the time and manual effort in diagnosing and triaging such issues to reduce customer impact. Large volumes of logs and mixed type of attributes (categorical, continuous) make any automated or manual diagnosis non-trivial.
This work presents the design, implementation and experience from building and deploying DeCaf, a system for automated diagnosis and triaging of KPI issues using service logs. It uses machine learning along with pattern mining to help service owners automatically root cause and triage performance issues. We present the learnings and results from case studies on two large scale cloud services at Microsoft where DeCaf has successfully diagnosed 10 known and 31 unknown issues so far. DeCaf also automatically triages the identified issues by leveraging historical data. Our key insights are that for any such diagnosis tool to be effective in practice, it should a) scale to large volumes of service logs and attributes, b) support different types of KPIs and ranking functions, c) be integrated into the DevOps processes.
Sat 11 JulDisplayed time zone: (UTC) Coordinated Universal Time change
01:05 - 02:05
|Tailoring Programs for Static Analysis via Program TransformationTechnical|
|Phoenix: A Tool for Automated Data-Driven Synthesis of Repairs for Static Analysis ViolationsDemo|
|BCFA: Bespoke Control Flow Analysis for CFA at ScaleTechnical|
|On the Recall of Static Call Graph Construction in PracticeTechnical|
Li Sui Massey University, New Zealand, Jens Dietrich Victoria University of Wellington, Amjed Tahir Massey University, George Fourtounis University of AthensPre-print
|DeCaf: Diagnosing & Triaging Performance Issues in Large-Scale Cloud ServicesSEIP|
Software Engineering in Practice
Chetan Bansal Microsoft Research, Sundararajan Renganathan Stanford University, Ashima Asudani Microsoft, Olivier Midy Microsoft, Mathru Janakiraman AmazonPre-print
|mCoq: Mutation Analysis for Coq Verification ProjectsDemo|