Give every product and category one clean address: point canonical tags at it, keep session codes out of links, tell Search Console which parameters only sort or track, and make the catalogue's own links use the clean version.

One product, many addresses

A shop sells a few hundred products. Search Console, formerly Webmaster Tools, reports tens of thousands of its pages in Google's index.

The extra pages are the same products under other addresses. The software adds a session code for visitors without cookies, a sort order when someone sorts by price, and a page size when someone picks forty-eight per page.

Each combination makes a new address. To Google, each address is a page to fetch, and it cannot tell in advance that they all show the same thing.

So it spends its visits to the shop on copies, while new products wait to be found.

Where the extra addresses come from

The first is the worst, because a session code is different for every visit. Google's crawler arrives without the cookie, so it can be handed a fresh code, and a fresh set of addresses, every time it comes.

What it costs

Google spends a limited amount of time crawling any one site. Time spent fetching the same product under forty addresses is time not spent finding the one added this morning.

Links are split as well. People who link to or share a product use whichever address they happen to see, so the page's standing is divided among the copies.

And Google chooses which copy to show in results. Without guidance it may pick one with a sort order or a session code in it, which looks odd in the results and can drop the visitor into an expired session.

Canonical tags

A canonical tag is a line in a page's code naming the preferred address for its content. Every version of a product page, whatever is added to its address, carries the same tag pointing at the clean one.

Google treats it as a strong hint rather than a command, and generally gathers the copies together under the address it names.

Most current shop software can add canonical tags, sometimes through a setting nobody switched on. Check a product page's source for rel="canonical", and confirm it names the clean address rather than the one you arrived on.

The URL parameters setting

Search Console has a setting, URL Parameters under the Crawl menu, where you tell Google what each parameter in your addresses does.

A parameter that only tracks, such as a session code, can be marked as not changing the content. One that only sorts can be marked as sorting, and Google told not to crawl the sorted copies.

Google warns that a wrong setting can drop real pages from its index. Change it carefully, one parameter at a time, after the canonical tags are in place.

A worked example

A vintage clothing store in a neighbourhood of cafes and record shops sells dresses, coats, denim and accessories from the 1950s to the 1990s, over the counter and through an online store of a few hundred one-of-a-kind pieces.

Each piece sells once, often within days of being listed, so new pieces need to be found quickly. Instead, a newly listed coat often did not appear in search results for weeks.

Search Console showed tens of thousands of indexed addresses for a few hundred pieces, and its HTML Improvements report listed hundreds of duplicate page titles.

The software was adding session codes for visitors without cookies, a separate address for every sort and every filter by decade, size and colour, and two category paths to each piece.

The fixes went in over a week. Session codes now live in cookies only, a setting in the software. Canonical tags point every version of a piece at one clean address, and the filter links lead to clean category pages.

Sort and page size parameters were marked in Search Console, and the sitemap now lists only clean addresses for pieces still for sale.

Cleaner links throughout

Canonical tags and settings tidy up after the copies. Better links stop many of them being created.

Menus, category pages and related product links should all use the clean address. Sort and filter options can be built so they do not create a new address for every combination.

Tracking tags belong only on links from outside the site, such as an email or an ad. Added to internal links, they create copies and overwrite Analytics' record of where visitors really came from.

And the sitemap, the list of pages a site hands to Google, should contain only clean addresses of products actually for sale.

Checking the result

Search Console's Index Status report charts the number of indexed pages week by week. On a shop full of copies, that figure is far higher than the number of real pages.

The HTML Improvements report lists duplicate titles, which often point straight at the parameter causing them.

Expect the numbers to fall slowly. Google revisits old addresses over weeks and months, not days.

Do not block them first

The tempting shortcut is to block every address with a parameter in robots.txt, the file that tells crawlers where not to go.

Blocked pages cannot be crawled, so Google never sees their canonical tags, and addresses it already knows can linger in its index. Fix the tags and links first, and block only what has no value at all.

The counter-case

A shop with a few dozen products and no sorting or filtering may have very few copies, and the canonical tags its software already adds may be enough.

Some filtered pages are worth having in search, such as a category narrowed to a brand people look for by name. Those deserve their own clean address and a proper title rather than a tag pointing elsewhere.

Everywhere else the rule holds: one product, one clean address, and every link, tag and setting pointing at it.

The order to do it in

  1. Compare indexed pages with your real product count.
  2. Move session codes into cookies only.
  3. Add canonical tags naming each clean address.
  4. Point menus and filters at clean addresses.
  5. Keep tracking tags off internal links.
  6. Mark sort and size parameters in Search Console.
  7. List only clean addresses in the sitemap.

Step two stops the endless supply. Every other fix tidies copies that already exist; a session code in the address creates new ones each time Google calls.

Tidying the old addresses a site has already piled up follows the same logic as two thousand redirect rules read on every visit.

Frequently asked questions

Why does Google find so many copies?

Shop software adds session codes, sort orders, page sizes, filters and tracking tags to addresses. Each combination is a separate address to Google, though the product shown is the same.

Do duplicate addresses hurt a shop?

They use up the time Google spends crawling the site, split links between copies and can put an odd address in search results, while new products wait to be found.

What does a canonical tag do?

It names the preferred address for a page's content. Every version of a product page carries the same tag, and Google generally gathers the copies under that address.

Where is the URL parameters setting?

In Search Console, formerly Webmaster Tools, under the Crawl menu. It tells Google what each parameter does. Change it carefully, since a wrong setting can drop real pages.

Should I block parameter addresses in robots.txt?

Not first. Blocked pages cannot be crawled, so Google never sees their canonical tags. Fix the tags and links, and block only addresses with no value at all.

Why keep tracking tags off internal links?

They create extra copies of pages, and in Analytics they overwrite where the visitor really came from. Use them only on links from emails, ads and other sites.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

More indexed pages than products?

Tell us your shop's address, and we will find the parameters creating copies and the order to fix them in.

Start a Conversation