Introduction
Duplicate content refers to blocks of content on the internet that is the same or ‘appreciably similar’ in more than one URL (or web page). These duplicate content pages could be on the same site (same domain) or on different sites (different domains) and may be due to syndicated content.
Google, Bing and the other major search engines are able to detect content that is ‘appreciably similar’ and often won’t index duplicate content as it is not a good user experience to show the same content from multiple websites to users who perform a search.
Search engines try to attribute the ranking value to the primary source of the data and not to the sites with the duplicate content. These sites therefore may not rank well and in extreme cases may even be removed entirely from the Google index, in which case they will no longer be visible on Google. Only the primary source of the content will rank and be displayed in search results.
This duplicate content may be unintentional and may result from
- Different instances of the same content for different devices (e.g. desktop and mobile views of the data);
- Printer formatted versions of web page content;
- Store items offered on different pages of a site or on different sites.
It is important to be aware that duplicate content is not a violation of the Search Engine guidelines, rather is creates a case where the duplicated content will not perform as well in search and will not receive as much traffic. The first site to display the content is the most likely to be indexed by search engines will always have a strong ranking advantage.
First to post will usually have a strong ranking advantage
Syndicated content
This article deals with syndicated products or store items offered on a number of different sites. A supplier or manufacturer of the products may provide these products on a website and then offer the same products to different distributors for presentation and sale on their own websites at the same time. Products displayed by a Franchise would be a typical example of the products discussed in this article.
Image 1 shows a schematic of the Supplier – Distributors relationship discussed. The products with their title, description, price etc. are the content for the page on each the site. Each page is represented by a unique address or URL so that everything provided by the Supplier apart from the URL is ‘appreciably similar’ to that offered by the distributors.
When it comes to ranking and being found on the web for the content (products), search engines like Google will always try to show the version of the content that is most appropriate to the users’ search.
If the content on the supplier and each of the distributors sites is the same (i.e. appreciably similar) and there is no indication of which is the primary page then the search engine will most likely show the content from only one of the sites (usually the Supplier) and the other sites will not appear in searches.
A canonical URL can be used to identify the preferred page and search engines recommend the practice of creating Canonical links. However, in the case of syndicated products the agents are not likely to be pleased if their products never show up in searches.
Canonical Links
A canonical link is a means of identifying the URL that is most representative (or preferred version) of a set of duplicate pages. This can be used for pages (URL’s) on the same site or pages on different sites.
Canonical links are not meant for the exclusion of pages, they are meant to manage the choice of the canonical page by the Search Engine. In the absence of a canonical link Search Engines attempt to identify the canonical copy. Google advises however that even if a Canonical links is provided,a different page may be chosen for other reasons like performance or speed .
A noindex tag is used to exclude pages from search engines. These should never be used on the same page as a canonical link and should not be seen as an alternative to a canonical link.
The canonical link is created in the head section of the site and has the format
<link href=”url_of_the_preferred_page” rel=”canonical” />
This link should be created in the head section of every duplicate version of the page.
How to Deal with Syndicated Products
In the case of syndicated content, there are 3 main options available for the distributor’s pages (the site displaying the syndicated content).
- Display the content exactly as received:
In this instance, the content will be the same as the content on the suppliers site and it is unlikely that the content will be displayed by Google in search results unless the distributor is first to present the content. - Display the content exactly as received but attribute the content to the supplier through the use of a canonical link:
This approach tells search engines that the content is known to be duplicate content and identifies the canonical version of the page (as that of the supplier). The down-side for the distributor is that most of the ranking power is attributed to the canonical version. However, the advantage of this approach is that the duplicate content will still be considered to be valid. As Google always tries to display the content that is most relevant to the users’ search (i.e. location, language, search phrase etc.), the sites with the duplicate content may still appear in relevant searches. - Add value to the content provided:
The most effective approach for dealing with syndicated content is for the distributor to rewrite the content and / or add value to the content displayed on the distributors site (for example by adding reviews, frequent questions or other content). In this instance, the distributors page will be able to rank in it’s own right and can compete with the suppliers’ page in search.
There is a fourth (and very effective) approach that expands on option 2 and can lead to good ranking for the distributor. This approach involves the distribution displaying the products with canonical links to the suppliers’ pages (option 2) and then creating list pages that combine these products onto one or more list pages (see the image). Even though the individual products are identical to the suppliers products and may not rank on their own, combining these on a separate list page makes the list pages unique to the distributor. The list page will rank for the aggregated product content and any other unique content added to the page. Ideally the list page will include a unique title and description in addition to the listed products which will be combined using some logic which adds value to the products.
Links from the list page to the individual product pages will increase the click through rate and this too will have a positive impact on ranking. This is a very powerful approach where the combined products are have more ranking power than the individual components.
The combining of products can be more powerful than the sum of the individual products
There is another potential reason for distributors adopting this fourth approach. Remember that while canonical links indicate a preference, Google advise that they may still set a different page as the preferred page if it has better performance and speed. Therefore, if the list page created by an distributor drives traffic to the individual product pages, this constitutes improved performance and if the distributor has optimised his / her site for speed, it may be argued that the product pages on the distributors site may become the pages that feature in the searches despite any canonical links created.
Conclusions
- Only one version of duplicate content is usually displayed by search engines.
- Canonical links can be used to identify the ‘preferred’ version of the content.
- The supplier of the content is most likely to rank well for the syndicated content if it is exactly the same as the content on the distributors’ sites. The most effective approach for distributors is to add value to the content provided by a supplier by re-writing the content or by adding reviews, additional information etc.
- If the distributor must duplicate the content provided by the supplier, then a very powerful approach is to set up canonical links which point back to the suppliers page and to create unique list pages where value is added to the products by grouping these on separate pages.