SEO Strategy · 4 min read
Testing for Duplicate Content Now Google's Cache Is Gone
Short answer
Since Google stopped offering cached page links in 2024, test duplicate content with Search Console's URL Inspection tool (compare user-declared and Google-selected canonicals), exact-phrase searches, a site crawler with near-duplicate detection and a canonical tag audit. Fix duplicates with rel="canonical", 301 redirects or consolidation.
Key takeaways
- URL Inspection shows which URL Google picked as canonical; a mismatch with your declared canonical is the clearest duplicate signal you can get.
- Crawlers with near-duplicate detection find the internal duplicates that searches miss, such as parameter URLs and printer versions.
- Google treats redirects and rel="canonical" as strong canonical signals and sitemaps as weak ones, and does not recommend noindex for choosing a canonical within one site.
For years, a quick duplicate content test meant opening Google's cached copy of a page to see which version Google had stored. That option ended in 2024, when Google retired cached page links and the cache: operator. The question still matters: which URL does Google treat as the main version? Here are the tools that answer it now, and the fixes.
What replaced Google's cache for duplicate checks?
Search Console's URL Inspection tool is the closest replacement, because it shows the URL Google selected as canonical.
Google stopped showing cached versions of web pages in 2024. Since September 2024, the About this result panel in Google Search can link to the Internet Archive's Wayback Machine for older copies, which helps with history but tells you nothing about Google's index.
URL Inspection does. For any URL in a property you verify, it reports whether the URL is on Google, the user-declared canonical (what your page says) and the Google-selected canonical (what Google chose after finding similar pages). Google notes the indexed view is not a live test, and the live test does not check duplicate status. So use the indexed view for canonical questions.
How do you test for duplicate content step by step?
Combine Search Console, targeted searches and a crawl, because each catches a different kind of duplication.
- Inspect your key URLs in Search Console. If the Google-selected canonical differs from the one you declared, Google sees a duplicate and preferred another version.
- Check the Page indexing report for "Duplicate without user-selected canonical" and "Duplicate, Google chose different canonical than user" groups.
- Search a distinctive sentence from the page in quotation marks. Other domains showing that sentence are copies or syndication.
- Run a site crawler with near-duplicate detection and list pairs above your similarity threshold.
- Audit canonical tags across templates: missing, self-conflicting, pointing to redirects or pointing to noindex pages.
- Compare titles and descriptions across the crawl. Identical titles across many URLs are a fast pointer to duplicate templates; the title and meta checker helps rewrite them.
What kinds of duplicate content does each method find?
Internal duplicates show up in crawls and URL Inspection; external copies show up in exact-phrase searches.
| Duplicate type | Common cause | Best detection method |
|---|---|---|
| Parameter URLs | Sorting, filters, tracking tags | Crawler, Page indexing report |
| HTTP vs HTTPS, www vs apex | Missing redirects | Crawler, URL Inspection |
| Printer or AMP versions | Old templates | Crawler |
| Near-duplicate location or product pages | Templated copy with swapped names | Crawler similarity score |
| Syndicated articles | Partner republishing | Exact-phrase search |
| Scraped copies | Content theft | Exact-phrase search |
For syndication, ask partners to add a canonical or a clear link to your original.
How do you fix duplicate content?
Pick one preferred URL per piece of content, then send every strong signal toward it.
Google's canonicalization documentation ranks the methods: redirects and rel="canonical" are strong signals, sitemap inclusion is a weak one, and stacking them improves the chance your preferred URL is shown.
- Use a 301 redirect when the duplicate URL has no reason to exist, such as an old HTTP or www version. Our guide to 301 vs 302 redirects covers the choice.
- Use rel="canonical" when users still need the duplicate, such as a filtered product list.
- Use noindex for pages that should not appear in search at all, but do not use it to choose between duplicates on your own site; Google says it does not recommend that.
- Consolidate near-duplicate pages into one stronger page when they compete for the same query, and redirect the rest.
- Link internally to the preferred URL only, and list only that URL in your sitemap.
Hypothetical example: a retailer has 40 city pages that differ only by the city name. Crawling shows 95% similarity across all of them. Rather than canonicalizing them to one page (which would hide them all but one), the better fix is to keep the cities with real local inventory and content, rewrite those with specifics, and merge the rest into a regional page with redirects.
How often should you check?
Check after every template change or migration, and review the Page indexing report monthly.
Duplicates usually come from development changes, such as a new filter or a staging site left open. A monthly look at Search Console's duplicate groups catches them early. Our technical SEO audit includes this check, and our SEO QA checklist lists the release-day tests that prevent it.
Frequently asked questions
Does the cache: operator still work in Google?
No. Google began retiring cached page links in early 2024, and the cache: operator went with them during 2024. For a historical copy of a page, Google Search's About this result panel has linked to the Internet Archive's Wayback Machine since September 2024.
Is duplicate content a Google penalty?
Ordinary duplication, such as URL parameters or printer pages, is handled by canonicalization: Google picks one version to show. The cost is wasted crawling and split signals. Scraped or copied content at scale is a different issue covered by spam policies.
Should I use noindex or canonical for duplicate pages?
Use rel="canonical" or a 301 redirect for duplicates on your own site. Google's documentation says it does not recommend noindex to prevent selection of a canonical page within a single site.
How similar do two pages have to be to count as duplicates?
Google does not publish a threshold. Crawler tools usually flag pages above a similarity score you set, often 85 to 90 percent. Review flagged pairs by hand, because templated pages can score high and still serve different purposes.