Above a set number of sessions, Analytics answers custom questions from a sample and scales it up, which distorts small counts. Use shorter date ranges, standard reports or prepared views when the figure matters.

A small line above the graph

Ask Google Analytics a large question, such as three years of visits with a segment applied, and a line may appear near the top of the report saying it is based on a share of sessions, with a percentage.

Most people never notice it. The graph looks the same and the table still has numbers in every row.

But those numbers were calculated from a sample of visits and multiplied up to estimate the whole. For large figures that is usually close enough. For small ones it can be badly wrong.

Whether it matters depends on what you are about to decide.

Why Analytics samples

The standard reports, opened as they are, come from totals Analytics has already calculated. They are not sampled, however long the date range.

A question it has not prepared for has to be worked out on demand from the individual sessions: a segment, a secondary dimension, a custom report, an unusual combination.

Above a certain number of sessions in the date range, several hundred thousand in the free version, Analytics works from a subset instead and scales up the result. Reports stay fast, at the cost of precision.

What triggers it

The last is the one that catches growing businesses. A report that ran unsampled two years ago can start showing the notice with nothing changed except the traffic.

Small numbers suffer most

Sampling estimates the whole from a part. If a site had two thousand quote requests in three years, a ten percent sample sees about two hundred, and multiplying by ten lands close to the truth.

If one segment had twelve quote requests, the sample might hold one, or none, or three. Multiplied by ten, that becomes ten, zero or thirty.

Visit totals hold up well. Conversions within a narrow segment, which are often exactly the figures a decision rests on, do not.

When it matters

It matters when a decision rests on a small group: whether phone visitors request quotes, whether visitors from one town justify an advertising budget, whether a page added last year produces enquiries.

It matters for comparisons across years, because the difference between two estimates can be larger than the real change.

And it matters when a figure goes into a report for a partner, a lender or a board, who will read it as exact.

It matters less for broad questions such as which months are busiest, where the numbers are large and the shape of the graph is what you need.

A worked example

A brick and stone supplier near a growing suburb sells to masons, builders and homeowners from a yard stacked with pallets. Its site has a large gallery of brick colours and a form for requesting samples.

Before paying for a gallery that works better on phones, the owner asked Analytics for sample requests from phone visitors, month by month, from early 2012 to May this year.

The report carried the sampling notice. Several months showed no requests from phones at all, and others showed improbable jumps.

The owner nearly concluded that phone visitors rarely asked for samples. The office manager pointed out that the request emails in the inbox told a different story.

When the report was rerun one quarter at a time, the notice disappeared, and the monthly figures matched the emails closely. The gaps had come from the sample, not from customers.

Shorter ranges that avoid it

The simplest fix is to ask the same question over a shorter period. A quarter or a month at a time usually stays under the threshold for a small business site.

Export each period and add them up in a spreadsheet. It takes longer, and the result is a count rather than an estimate.

If the notice still appears on a single month, the site is busy enough that the next two options are worth setting up.

Reports that are never sampled

Before building a segment, check whether a standard report already answers the question. Mobile visits, traffic sources, landing pages and goal completions all have reports of their own.

Opened without a segment or secondary dimension, those reports are calculated in advance and stay unsampled even over several years.

That often means asking a slightly different question, such as goal completions by device from the mobile report, rather than a segment of phone visitors applied to the goals report.

Views set up for the question

A view is a filtered copy of the data, collected from the day it is created. A view that includes only the shop section, or only visitors from Canada, has unsampled standard reports of its own.

Views do not work backwards, so they help next year's questions rather than this year's. Set them up now for the comparisons you know you will want.

The precision setting beside the notice can also be moved towards greater precision, which uses the largest sample allowed. It reduces the problem without removing it.

The counter-case

Most small business sites will rarely see the notice. A typical local business site stays under the threshold for most of the ranges people actually look at.

A sampled report is also fine for a broad question about large numbers, and treating every sampled figure as useless would be its own mistake.

The paid version of Analytics raises these limits considerably, at a price aimed at large companies. For everyone else, look for the notice before relying on a small number, and shorten the range when it appears.

Before the next long report

  1. Look for the notice above every custom report.
  2. Try a standard report before building a segment.
  3. Shorten the date range until the notice goes.
  4. Add the periods up in a spreadsheet.
  5. Check small counts against your own records.
  6. Create views now for next year's questions.
  7. Label sampled figures as estimates in reports.

Step five is the safeguard. Enquiry emails, booking records and invoices are counts, not estimates, and an Analytics figure that disagrees with them badly is the one to distrust.

Adding together visits that Analytics splits across several lines is covered in Facebook visits scattered across several referrers.

Frequently asked questions

What does the sampling notice mean?

That the report was calculated from a share of sessions and scaled up to estimate the whole. The percentage shown says how large a share was used.

Which reports are not sampled?

Standard reports opened as they are, without a segment or secondary dimension. They are calculated in advance, so they stay unsampled even over several years.

Why are small numbers less reliable?

A sample of a rare event may catch one occurrence or none, and scaling that up turns it into a large number or zero. Visit totals hold up far better.

How do I avoid sampling?

Shorten the date range to a month or a quarter, export each period and add them up. Or answer the question from a standard report without a segment.

Do filtered views help with sampling?

Yes, for future questions. A view filtered to one part of the site has unsampled standard reports of its own, but it only collects data from the day it is created.

Will a small business site be sampled?

Rarely, for ordinary date ranges. It becomes likely as traffic grows and ranges stretch over years, which is why growing businesses should check for the notice.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Deciding on figures from a sampled report?

Send us the question you are asking Analytics. We will get you an unsampled answer and check it against your records.

Start a Conversation