Real User Monitoring: From Beacons to Percentiles

What you'll learn: By the end of this lesson you will be able to explain what a real user monitoring (RUM) system collects and why the unit is a page view, design the page-side sampling and beacon sending, aggregate beacons into a 75th percentile per route template and device class, handle duplicates, out-of-order reports, sampling weights and late-arriving data, explain why percentiles must never be averaged, and judge when your own RUM, the Chrome UX Report or a lab test is the right source.

A restaurant can learn about its food in two ways. A critic visits once, at a fixed table, and writes a careful review. Or every guest gets a comment card on the way out. The cards are messier: only some guests fill them in, a few fill in two, some arrive by post a week later. But they describe what guests actually experienced. The critic is a lab test. The cards are real user monitoring.

Each visit to a page measures its own loading, responsiveness and layout stability, then posts a small report as the visitor leaves. Collecting the reports is the easy part. Turning thousands of messy ones into one trustworthy number per page and device is the subject of this lesson.

The running example is the product page of an online store, measured with the three Core Web Vitals from Loading Performance and Core Web Vitals: Largest Contentful Paint (LCP), Interaction to Next Paint (INP) and Cumulative Layout Shift (CLS). That lesson shows a small pipeline end to end. This one covers what happens once the numbers arrive.

The core idea

1. The unit is a page view, and the answer is a distribution. One visit gives one value per metric. You summarise many visits, and web.dev recommends the 75th percentile: "a good threshold to measure is the 75th percentile of page loads, segmented across mobile and desktop devices."
2. Collection is lossy by design. Only a sample of visits report, reports are sent as the page is hidden (when delivery is least reliable), and some arrive twice, out of order or hours late.
3. Correct aggregation is a handful of small rules. One value per page view and metric. Windows chosen by when the visit happened. Weights that undo uneven sampling. Percentiles computed from page views, never averaged from other percentiles.

The 75th percentile (p75) is the value that 75 percent of page views are at or below. The Chrome UX Report API says the same: "75% of page loads experienced the given metric at or less than this value." It is a target for most visitors, not the average one.

What a RUM system collects

For each metric: its name, value and an identifier. The web-vitals library gives your callback a Metric with name, value, rating, delta, id and navigationType. Its README calls the id "a unique ID representing this particular metric instance", usable to deduplicate. At the time of writing the README documents version 5: onCLS works in Chromium only, while onINP, onLCP, onFCP and onTTFB work in Chromium, Firefox and Safari.

Then come the dimensions you slice by, kept few and low in variety:

• Route template, such as /product/:slug, not the raw URL. Raw URLs make thousands of tiny segments and may carry personal data in query strings. The Chrome UX Report also strips query strings and fragments so visits to one page group together.
• Device class. The rule that puts a visit in a class is your own decision.
• Release identifier. web.dev advises a unique version identifier per deployment so a regression can be tied to a change.

web.dev also warns against tracking more than you need. Send no personal data.

Figure 1. Where each decision lives

How to read this: the left zone decides what is measured, whether this visit reports at all, and when to send. The right zone never sees the visits that were sampled out, so it must correct for them. Dashed arrows are fire-and-forget.

Getting reports out of the page

MDN says the most reliable moment to send is the visibilitychange event: unload and similar events are unreliable, "especially on mobile", and unload is incompatible with the back/forward cache. navigator.sendBeacon() returns true when the browser queued the data, not when the server received it, and queued data is limited to 64 KiB.

This sketch applies those rules. It is not run in the lab, which has no browser. The sample rate, release string and device rule are illustrative.

import { onCLS, onINP, onLCP } from "web-vitals";
import type { Metric } from "web-vitals";

const SAMPLE_RATE = 0.1; // illustrative: one visit in ten reports
const RELEASE = "2026-11-08.1"; // illustrative: normally injected by the build

if (Math.random() < SAMPLE_RATE) {
  const viewStartedAt = Date.now(); // coarse: only used to choose a day
  const seqById = new Map<string, number>();
  const queue: object[] = [];

  const report = ({ name, value, id }: Metric): void => {
    const seq = (seqById.get(id) ?? 0) + 1; // newer reports for one id get a higher number
    seqById.set(id, seq);
    queue.push({
      name, value, id, seq,
      url: location.pathname, // no query string, no fragment
      device: innerWidth < 768 ? "phone" : "desktop", // a crude, illustrative rule
      release: RELEASE,
      viewStartedAt,
      sampleRate: SAMPLE_RATE,
    });
  };

  onLCP(report);
  onINP(report);
  onCLS(report);

  addEventListener("visibilitychange", () => {
    if (document.visibilityState === "hidden" && queue.length > 0) {
      navigator.sendBeacon("/rum", JSON.stringify(queue.splice(0)));
    }
  });
}

The sampling draw happens once per page view, so a visit reports a coherent set. seq is this lesson's own convention: the README offers two ways to cope with repeated reports for one id (overwrite the value, or send delta and sum by id), and a per-id counter makes overwriting safe when requests are reordered. The collector parses the body as JSON itself, since MDN does not say which Content-Type a string body gets.

From beacons to a number

One value per page view and metric

The collector keeps one entry per metric and id. A retried request repeats a beacon: ignore it. A page that reports CLS early and again at the end sends two reports for one id: keep the higher seq, whichever arrives first. The result then does not depend on delivery order, and delivering everything twice changes nothing.

Figure 2. One page view's beacons, as the collector sees them

How to read this: time runs downward. The aggregator keeps the newest report per metric and id, ignores repeats, and counts what it drops. The counts matter as much as the numbers.

Windows follow the visit, not the arrival

A beacon belongs to the window in which its page view began, not the one in which it arrived. A phone offline for three hours delivers a 23:00 report at 02:00, and arrival time would put yesterday's visit in tomorrow's numbers. A window cannot stay open forever, so you choose an allowed lateness: after the window ends plus that allowance, later beacons are dropped and counted. Until then the window is provisional; after it, closed. The allowance trades freshness against completeness, with a bias: the latest arrivals are often the visits with the worst connections.

Figure 3. What the aggregator does with each beacon

How to read this: every beacon ends in one of five outcomes, each with a counter. A sudden rise in late or invalid is an incident in the pipeline, not in the page.

Segments and "not enough data"

Group the kept page views by route template, device class and, when comparing deployments, release. Each group gets its own p75 and rating. A group with too few page views has an unstable percentile, so report none. The Chrome UX Report applies the same idea: a page or origin must be "sufficiently popular", though Chrome does not publish the number. The lab uses 30, an illustrative value.

Sampling needs weights

If every group is sampled at one rate, a plain percentile works. If rates differ, say you keep half of the rare tablet visits, a plain percentile over-counts tablets. Weight each page view by 1 / sampleRate. The weighted nearest-rank percentile sorts the values and walks up the cumulative weight until it reaches 75 percent of the total. With equal weights it is the ordinary nearest-rank percentile: rank = ceil(0.75 x n).

Never average percentiles

The p75 of a union cannot be rebuilt from its groups' p75 values, because it depends on how many page views each group holds and on the shape inside each. Compute any combined number from page views or merged histograms. The Chrome UX Report adds a caution: its percentiles come from the full dataset, so they may differ from one calculated from its binned histogram. If you store histograms to save space, decide how much approximation you accept.

Three illustrations

1. A minimal example (illustrative values). Six LCP values in milliseconds: 1,900, 2,100, 2,300, 2,600, 3,100, 3,900. Rank = ceil(0.75 x 6) = 5, so p75 is 3,100 ms. The "good" threshold is 2,500 ms, so the rating is "needs improvement". The median (rank 3) is 2,300 ms and would pass. A team that reports the median believes the page is fine while a quarter of visitors wait over three seconds.

2. A realistic example: one day of the product page. The lab below generates a day of seeded, invented traffic. In the generator phones are 62 percent of visits and desktops 33 percent. Per device, LCP p75 is 3,846 ms for phones and 2,417 ms for desktops: desktops pass, phones do not. One combined row (3,320 ms) says "needs improvement" and hides which population to work on.

3. A failure case: the average of percentiles. In the same data the mean of the three per-device LCP p75 values is 3,042 ms, while the p75 of all page views together is 3,320 ms. The first number is lower, looks healthier, and is wrong.

Choosing the source

web.dev's lab-versus-field guide lists why field and lab disagree: screen size changes which element is largest, networks differ, personalisation and A/B tests differ per visitor, lab runs start with a cold cache, and real users scroll and interact. Use field data to decide what to fix and lab data to find out why.

Build it: aggregate beacons into percentiles

Goal. Run a deterministic aggregator on a day of invented traffic and watch it deduplicate, weight by sample rate, segment and handle late beacons.

Setup. The lab is examples/frontend/labs/rum-aggregator: pure TypeScript, no browser. It needs Node.js (captured on Node 26.12.1, which runs TypeScript files directly).

cd examples/frontend
pnpm install
cd labs/rum-aggregator
pnpm test          # 27 tests
node src/demo.ts   # prints the output below

Steps.

1. Read section 1. Of the 2,389 beacons, some are updates (a final CLS replacing an early one), some are ignored repeats, some are late. These counters are what you would graph to watch the pipeline itself.
2. In section 2, compare phone and desktop for each metric.
3. In section 3, find the two segments that report "insufficient data" (fewer than 30 page views).
4. In section 4, compare the weighted and unweighted p75. Tablets are 20 percent of sampled views but about 5 percent of estimated real visits.
5. In section 5, compare the mean of the per-device p75 values with the p75 of everyone.
6. In section 6, the last line shows the window with no lateness limit: the late beacons would have moved the phone p75 from 3,846 to 3,881 ms. They come from slower visits.

Expected output, captured from a real run (Node 26.12.1, 2026-11-08). The traffic is seeded and invented.

=== 1. What happened to each beacon
beacons received:  2389
accepted:          2121   (first report for a page view and metric)
updated:           198   (a newer report replaced an older one)
ignored:           38   (duplicates and older reports that arrived late)
late:              32   (arrived after the window closed)
invalid:           0

=== 2. 75th percentile per metric and device (window closed)
metric template        device   views est.views     p75  rating             good/ni/poor
CLS    *               desktop    188      1880   0.037  good               96%/4%/0%
CLS    *               phone      380      3800   0.090  good               80%/18%/2%
CLS    *               tablet     139       278   0.053  good               91%/7%/1%
INP    *               desktop    188      1880     160  good               86%/14%/0%
INP    *               phone      380      3800     341  needs-improvement  42%/49%/9%
INP    *               tablet     139       278     254  needs-improvement  61%/36%/3%
LCP    *               desktop    188      1880    2417  good               78%/19%/3%
LCP    *               phone      380      3800    3846  needs-improvement  42%/35%/23%
LCP    *               tablet     139       278    2864  needs-improvement  63%/27%/10%

=== 3. LCP per template and device (segments under 30 views report no percentile)
metric template        device   views est.views     p75  rating             good/ni/poor
LCP    /cart           desktop     29       290     n/a  insufficient data  -
LCP    /cart           phone       58       580    2405  good               78%/19%/3%
LCP    /cart           tablet      25        50     n/a  insufficient data  -
LCP    /product/:slug  desktop    118      1180    2748  needs-improvement  69%/25%/5%
LCP    /product/:slug  phone      228      2280    4507  poor               30%/38%/32%
LCP    /product/:slug  tablet      81       162    3342  needs-improvement  52%/31%/17%
LCP    /search         desktop     41       410    2143  good               90%/10%/0%
LCP    /search         phone       94       940    3153  needs-improvement  51%/37%/12%
LCP    /search         tablet      33        66    2325  good               82%/18%/0%

=== 4. Sampling: phones and desktops sampled at 10%, tablets at 50%
LCP p75 over all devices, weighted by 1 / sample rate: 3320
LCP p75 over all devices, every sampled view counted once: 3253
tablet share of sampled views: 20%, of estimated real views: 5%

=== 5. Averaging per-device percentiles is not the percentile of everyone
per-device LCP p75 values: desktop 2417, phone 3846, tablet 2864
mean of those p75 values:          3042
p75 of all page views together:    3320

=== 6. Late-arriving data moves the numbers until the window closes (LCP p75, phone)
at midnight                    provisional  views  378  p75 3846
after the lateness allowance   closed       views  380  p75 3846
beacons dropped as late after the window closed: 32
same window with no lateness limit: views 384, p75 3881

Break it. In weightedPercentile, change const target = p * total; to const target = p * sorted.length;. The percentile ignores what each sample stands for, and two sampling tests fail. Revert it.

The lab runs the collector's logic only. The generator's rates of duplicates, reordering and delay are made up to exercise the rules, not measured.

In practice: a dashboard that improved when the collector broke

The situation

An online store's team watches a daily dashboard of LCP, INP and CLS per page template and device class. Phones are most of the traffic. A release that moves a phone p75 past its "good" threshold is rolled back.

Starting architecture

Figure 4. The existing pipeline

How to read this: the two warning boxes are the weak points. The collector reports nothing about what it drops, and the daily job closes its window at midnight with no allowance for late data.

Options considered

The decision

Rollout and verification

1. Run the new aggregator beside the old job for two weeks and explain every per-segment difference.
2. Check estimated traffic. The sum of weights per device should follow the store's own traffic within an accepted range.
3. Compare direction with the Chrome UX Report for the origin, remembering it covers only some Chrome users over 28 days.
4. Test the failure on purpose: pause the queue consumer for an hour in staging and confirm the alerts fire.

The 2 a.m. failure

At 02:10 an alert says phone LCP p75 improved sharply overnight. It was a fault: the queue consumer stalled after a deploy and recovered hours later. Beacons that arrived on time came from fast connections; the slow ones arrived after the window closed.

First three checks: (1) estimated page views per device against previous days: phones dropped far more than desktops; (2) the late counter, which spiked; (3) collector and queue lag. Mitigation: mark the window provisional and keep it out of release decisions. Follow-up: alert on pipeline counters before metric movements, and treat a large improvement alongside a drop in estimated volume as a pipeline incident first.

Architect's notes

• Decide the segment list before the dashboard. Template, device class and release are enough to start; each dimension multiplies the segments.
• Give the pipeline its own SLOs: counts of accepted, ignored, late and invalid beacons, and lag. See observability and SLOs.
• Keep the sampling decision in one place and send the rate with every beacon.
• Make the minimum sample size explicit and show "insufficient data" instead of a number.
• Use RUM to choose what to fix, then lab tools and attribution to find out why. The metrics are covered in LCP optimization, INP and CLS.

Common mistakes

• Averaging percentiles across segments. Compute from page views or merged histograms.
• One number for all devices. Phones and desktops are different populations.
• Letting arrival time choose the window.
• Raw URLs as segments. They explode in number and may carry personal data.
• Sending on unload. Use visibilitychange.
• Ignoring sampling rates when they differ.
• Trusting a metric move before checking beacon volume.
• Reading true from sendBeacon as delivery. It means queued.

Key takeaways

• RUM measures real visits; the unit is a page view, read at the 75th percentile per segment.
• Send once per page view when the page is hidden, with a single sampling draw and no personal data.
• Keep one value per metric and id, so arrival order and duplicates do not matter.
• Window by when the visit began, add an allowed lateness, and label open windows provisional.
• Weight sampled views by 1 / sampleRate when rates differ.
• Never average percentiles. Show "insufficient data" for small segments.
• Count what the pipeline drops, and check those counts before believing a metric move.

Practice exercises

Exercise 1, Starter (30 min): Add two more values to the minimal example, work out the nearest-rank p75 by hand, then check it with weightedPercentile. Does the rating change?

Exercise 2, Intermediate (2 hours): Add a release comparison to the lab's report that flags a regression only when both segments reach the minimum sample and the p75 crosses a threshold. Test the too-few-views cases.

Exercise 3, Advanced (half day): Replace the stored page views with fixed histogram buckets per segment. Compare the bucketed p75 with the exact one over the seeded day and write down the error your bucket width allows.

Related reading

In this course:

• Loading Performance and Core Web Vitals: the three metrics, lab versus field, and a small pipeline.
• Interaction to Next Paint and Responsiveness: how INP is built, including its page-view percentile.
• LCP optimization and CLS and layout stability.

Elsewhere on the site:

• System Design: Observability and SLOs: percentiles, SLOs and alerting on symptoms.

Official documentation

• web.dev: Best practices for measuring Web Vitals in the field: percentiles, sending, release identifiers and data volume.
• web.dev: Web Vitals: the 75th percentile guidance and the "good" targets.
• web-vitals README (GitHub): the Metric fields, id, delta summing, batching and browser support.
• MDN: Navigator.sendBeacon(): queuing, the 64 KiB limit and visibilitychange.
• Chrome for Developers: CrUX API and CrUX methodology: p75, form factors, the 28-day aggregate, eligibility.
• web.dev: Why lab and field data can be different