HERMES Network anomaly detection · M-Lab

How to read HERMES

HERMES turns raw M-Lab speed tests into anomaly events — moments when a group of users sharing a source AS and city saw their connection get measurably worse (higher latency, lower throughput) versus their own normal baseline. For each event it then tries to answer the hard question: which part of the network is the source? This page explains how it does that and how to read every chart you'll see.

Group one source AS + source city measuring to a destination, tracked over time. The unit everything is computed on.
Baseline that group's typical RTT / throughput over a trailing window — the "normal" each day is compared against.
Event a day on which the group's measurements drifted far enough from baseline to be flagged as anomalous.
Tomography the inference step that pinpoints the source to a specific AS or a link within / between ASes.
The three-step workflow
  1. Browse & filter — Overview. Set a date range and optional filters (source AS, city, anomaly type, scope, IP version), then scan the events table, the country map, and the events-explained-by-entity bar to see where and what is breaking.
  2. Drill into one event — Event Detail. Click any row in the table. You get the full evidence for that single event: the measurements, the statistical tests that flagged it, and the two tomographies that identify the source.
  3. Zoom back out — Population View. See every event in your current window as one population: how severe, how widespread, how often HERMES could identify a source.
Reading the Overview

The events table is one row per detected event. Key columns:

  • Daily Median vs Baseline (Throughput Mbps, RTT ms) — the day-of value next to the group's normal. A drop in throughput or a jump in RTT is the anomaly.
  • Responsible entity — the entity the multi-granularity set-cover blamed for this group's anomaly (the same one shown in Event Detail), e.g. AS3257 · Chicago. Its granularity and interdomain/intradomain scope show as tags.

The Tags column summarises why a row was flagged:

latency day-of RTT rose > 5 ms above baseline, on ≥ 80% of samples.
throughput day-of throughput fell below baseline, on ≥ 80% of samples.
distance a day-of measurement travelled farther than any baseline measurement — a hint the path detoured.
edgenodeASmetroIXP  the granularity of the responsible entity that the set-cover attributed this group to.
path-local  a singleton — this group's anomaly wasn't correlated with any other group, so it was attributed to its own path's finest entity (an edge) rather than a shared culprit. Most events are path-local; edge is reserved for correlated link culprits.
interdomainintradomain  whether the responsible scope crosses an AS boundary or stays inside one.
reroutecongestion  a conclusive temporal-tomography verdict — the path either changed (reroute) or stayed the same but degraded (congestion). Absent when the verdict is indeterminate. Filterable via Temporal verdict.

The Culprit granularity and Scope filters in the toolbar let you zoom straight into a class of cases — e.g. only metro-wide outages, or only interdomain edges.

Events by country shades each country by event count. Events explained by entity ranks the ASes / segments that tomography flagged as the source most often across the window. Both honour the filters above.

Reading Event Detail — card by card
Measurement flow

A Sankey of how this group's measurements flow through the network. Red marks the sub-group that carried the events.

RTT, throughput, distance & loss over time

Each metric over time for the group. The orange band is the period of interest (the event day); the horizontal line is the baseline. A ★ star marks tests that also carry a reverse traceroute (only ~25% do). Click any point to load that exact measurement into the cards below.

Detection statistics — day-of vs baseline

The actual tests that flag an event. A row passes (pass) or not ():

  • Anomalous-sample ratio ≥ 80% — most samples moved the wrong way, not just the median.
  • Median RTT shift > +5 ms / throughput below baseline — the central tendency moved meaningfully.
  • Welch t-test p < 0.05 and Mann-Whitney p < 0.05 — the day-of vs baseline difference is statistically significant (parametric and rank-based).

The bold Anomaly verdict row is the same rule that listed the event in the Overview.

Correlation tomography — responsible entity

A single mixed-granularity set-cover attributes each anomalous group to exactly one responsible entity, picking the right level for the evidence — one of:

  • edge a link between two ⟨AS, metro⟩ nodes · node one ⟨AS, metro⟩ · AS a whole network · metro a whole city · IXP an exchange.

How it chooses (shown in the card so you can audit it):

  • Eligibility — an entity counts only if a material share of its own paths are anomalous (purity ≥ 10%) and it's statistically significant (Fisher's-exact p, odds ratio).
  • Coverage-first — among eligible entities it picks the one explaining the most still-unexplained anomalies, then repeats on what's left.
  • Zoom out only when broad — it reports a coarser entity (a whole AS or metro) only when that entity is still high-purity on the remaining anomalies and spans ≥ 2 distinct sub-entities; otherwise it keeps the finer one. So a metro is flagged only when several of its ASes are affected, an AS only when it's broadly bad. The card's drill trail shows when a coarser candidate was considered and rejected.
  • Path-local — a small singleton tail (groups not pooled with anyone) is attributed on their own path at the finest level. Lower confidence than a correlated entity.

The funnel shows the co-affected groups ( = yours) converging on the entity; purity = anomalous ÷ (anomalous + healthy) paths through it. Click any group in the funnel to drill into that event.

Temporal tomography — usual vs day-of path

Did the event come from the route changing or the same route degrading? The card leads with a one-line summary verdict:

  • Reroute — the path changed; the summary names the links traffic diverted onto and stopped using, with the before → after share of paths.
  • Congestion in place — the path stayed the same but got slower, pointing at load rather than a routing change.
  • Path stable / Inconclusive — no material change, or not enough path data to classify.

Below the summary, the What changed table lists every link the path gained or dropped (biggest swing first); the full forward/reverse path diagrams are tucked into a collapsible section. A ✓ agrees chip shows when the path change lands on the same place the correlation tomography blamed.

Logical & geographic paths

The forward and reverse paths drawn two ways: logical (AS / hop sequence, with reached vs not-reached hops) and geographic (the same hops on a world map). Use these to sanity-check that the source segment sits where the path actually goes.

Reading Population View

Every event in the window as one population. The summary cards:

  • Events — count in the window. Throughput drop / RTT increase — median severity, with p95 for the tail.
  • IPs / group — median distinct users behind a group (a proxy for blast radius).
  • Inter-domain — share of classified events whose source crosses an AS boundary.
  • Source identified — share of events where tomography actually identified a source (vs. inconclusive).

Below: a per-day event-count trend and distributions (throughput drop, RTT delta, unique IPs on a log scale, and inter- vs intra-domain split) across all events.

Glossary
AS / ASN
Autonomous System — a network operator (e.g. an ISP) identified by a number. Paths cross one or more ASes.
Intradomain vs Interdomain
A responsible edge whose two ⟨AS, metro⟩ ends are in the same AS vs. ones that span two different ASes.
RTT
Round-trip time in ms — the latency metric. Higher is worse.
Throughput
Download speed in Mbps. Lower is worse.
Baseline vs day-of
The group's trailing-window normal vs. the value measured on the event day.
Reverse traceroute
The path back from the destination, available for only ~25% of tests; lets HERMES attribute the source to the return direction, not just the forward one.
Granularity
The level of the responsible entity: edge (link), node (⟨AS, metro⟩), whole AS, metro, or IXP. The set-cover picks the coarsest level that's still broad and high-purity.
Purity
Of all paths crossing an entity, the share that are anomalous — anomalous ÷ (anomalous + healthy). An entity is eligible only at ≥ 10%.
Demoted-from / drill trail
A coarser candidate that was considered but rejected in favour of the finer entity actually reported (because it wasn't broad enough).
Attribution method
correlation (the set-cover pooled this group with others) or path-local (a singleton, attributed on its own path).
Interpreting responsibly
  • Reverse-path evidence exists for only ~25% of tests — absence of a reverse finding isn't absence of a reverse problem.
  • Tomography narrows correlation toward cause, but a weak or path-local verdict is a lead, not a proof.
  • Geolocation can be wrong. If a city or country looks off, use the ⚑ flag control on the measurement to submit a correction.