Skip to content

SLI and SLO

To do:

  • Burn rates
    • Fast burn
    • Slow burn
  • GCP examples
  • Why should we care about SLI and SLO?

SLI / SLO Monitoring

  • What is an unhappy user?
  • Why are there unhappy users?
  • What do unhappy users do?

Metrics of unhappiness

  • Good metrics of variance (correlation) are necessary to identify unhappiness
  • Good threshold values capture the beginning of unhappiness and the end of happiness. They do not begin after the variance has started and end before the variance ends
  • Tickets are trailing indicators, they are reactive, not proactive

Service Level Indicators (SLI)

  • SLI says how good a system is doing right now
  • Calculating SLI only works well with lots of traffic.
  • SLI: (good events / valid events ) x 100%
  • 3-5 SLIs per user journey
    • Examples:
      • You can log in
      • You can log in with the correct user
      • You can post messages
  • More SLIs is not always better. If you have many SLIs, it will confuse you. The SLIs will start to control other SLIs
  • You want to carefully scope your SLI’s to only look at data that is relevant to the SLI. If you include too much in the scope of the SLI, you drown out the interesting SLI data from the bad SLI data.

Example: SLI Menu

  • Request / Response
    • Availability
      • Availability is the proportion of valid requests server successfully
      • The profile page should load successfully
        • How do we define success?
        • Where is the success / failure recorded?
      • Simple example: The proportion of HTTP GET requests for /profile/{user} or /profile/{user}/avatar served successfully
      • Medium example: The proportion of HTTP GET requests for /profile/{user} or /profile/{user}/avatar that have 2XX, 3XX, 4XX (excl. 429) status
      • Detailed example: The proportion of HTTP GET requests for /profile/{user} or /profile/{user}/avatar that have 2XX, 3XX, 4XX (excl. 429) status measured at the load balancer
    • Latency
      • Latency is the proportion of valid requests served faster than a threshold
      • The profile page should load quickly
        • How do we define quickly?
        • When does the timer start / stop?
      • Example: The proportion of HTTP GET requests for /profile/{user} served faster than a threshold
    • Quality
      • Todo write
    • Data Processing example SLIs:
      • Coverage
      • Correctness
      • Freshness
      • Throughput
    • Storage example SLIs:
      • Throughput
      • Latency

Service Level Objectives (SLO)

  • Set a reliability target for an SLI.
    • Means that SLOs are inherently dependent of existing SLIs
  • Your objectives should have both a target and a measurement window Users are a bad measurement of an SLO:
  • User expectations are strongly tied to past performance.
  • Users only expect a service to work as well as as it has been working before. They do not care about any SLA’s
Example SLO table
Service
Web: User Profile
Web: User Profile

Example User Journey

  1. Fetch list of SKUs from the API server (purchase options)
  2. Fetch SKU details from Play Store (Purchase option details)
  3. User launches Play billing flow (User starts purchase option)
  4. Send token to API server (Play Store sends purchase token to API server
  5. Verify that the token sent by Play Store is valid

Where should you measure your SLOs?

  • We don’t care about canceled purchases as much as we do about successful purchases
  • SLO’s for successful purchases need to be more meaningful than SLO’s for started/canceled purchases