How Bing Discovery “Steals” Organic Traffic from Google Search

According to Sparktoro, Google Images is the second largest Google search engine with a whopping volume of more than X4 YouTube searches, crowning it the third most popular search engine in the world.

Please read and have a look at these numbers again.

where americans search the web

Why this is so interesting? Because it proves just how much image optimization can sometimes be overlooked and as a result, some unexpected competitors can use it for their advantage.

In the following post, I will explain in detail how Bing piggybacks on the low competition of Google’s image search and how to get huge amounts of organic traffic.

Meet Bing Discovery

First thing first, how did I find out about it?

During the last couple of months I have started to noticed more and more organic results are coming from a strange source called bing.com/discover/

I was not familiar with this URL family, so I used SEMRUSH to check the organic traffic for this folder. As you can see from the graph below, the first traffic data had started on in April 2017:

organic traffic driven from Google to Bing

Why this is VERY interesting?

  • On this magnificent graph trend you see above, we notice a gigantic increase starting September  – This is organic traffic driven from Google to Bing.
  • Later in this post, I will try to analyze and give my assumptions on what exactly happened during September 2018 (i.e. – the drop on the graph below) . Even when analyzing the overall Bing traffic that comes from google.com we can see how the numbers are relatively high, compared to the overall numbers:organic traffic driven from Google to bing - 2For example, the estimated organic traffic driven from Google to Bing in January 2019 is almost 22%. That’s HUGE.
  • Why should Google send organic traffic to other search engines (or search results pages)? Moreover, Google has noted several times that search results are often considered as low-quality content. To be fair, I don’t see these results as low quality, and if the users had thought the same, I would not have expected these numbers to last.
  • Direct competition with Pinterest – if we go back to the old history of 2012, Bing introduced a way to explore Pinterest board with Bing. Seems that a few years later, they were actually trying to replicate many SEO practices of Pinterest and as the graphs show – it worked like a charm. Bing was clever enough to index their search result pages with highly targeted long-tails, and in this way to be much more like Pinterest. Later on, I will try to explain in detail HOW they did it.

 

Universal Search vs. Google Images

First, it is important to spot the difference between traffic and rankings from Google universal search and google images.

Google universal search is the main SERP you will get in Google for any given query. It blends results from other Google search verticals such as Google News, Google local/maps, YouTube and yes, Google Images as well.

It is very common to see results from Google Images within the rest of the universal/blended search results. It all depends on the query.

So for example, searching for cute cats (because, well, why not?), will definitely display some of those:

universal search results for cute cats

According to SEMRUSH, Google Images sends Bing a sheer number of up to almost 16,000,000 visitors per month (!):

google Images sends Bing up to almost 16,000,000 visitors per month

To sum up the confusion – there is a strong correlation between ranking high on the first 10 results in the universal search results and being able to rank high on Google images. Bing Discovery is performing well on both.

 

How Bing Discovery was optimized for search

Here goes the really interesting part – how Bing SEO’s were able to rank on Bing Discovery?

  1. Keyword research – there is no doubt; these highly optimized pages for specific keywords have not come out of the blue. A few examples:
    bing.com/discover/christmas-borders-for-word
    bing.com/discover/power-window-wiring-diagram
    bing.com/discover/i-love-you-coloring-pagesThis is NOT by accident. Conducting keyword research for Google Images can be a little bit trickier. For a search engine that technically contains any given photo in the world, it can be a challenge to conduct a Keyword research about ANYTHING. What I will do in this case is to conduct extensive competitor research for a mega player such as Pinterest and “steal” keyword ideas from them.
  2. Internal links and hierarchy – how these pages are being crawled and indexed in the first place? I was digging deep, but could not find internal links or HTML sitemap leading to all the links. If we compare it to Pinterest, who are doing a great job at tagging the pins/images as internal links for categories/tag pages:pinterest tagging the pins/images as internal links for categories/tag pages:What I could find is internal links to main and generic categories such as travel, recipes, and animals:generic categoriesIf you click on one of them it will take you to the main category:main categoriesHowever, I could not find a clear internal links structure in here. Checking Bing’s robots.txt file:Bing's robots.txt fileAnd indeed the following sitemap.xml – http://cached.blob.core.windows.net/tmp/sitemap_all_v2.xml contains 6 sitemap.xml had been extracted to see the number of submitted URL’s (using xml site extractor):xml sitemap extractorEvery sitemap.xml contained 45,000 URL’s (besides the last one who contains only 11,336 URL’s). Just as a reminder – the maximum number of submitted URL’s is 50,000.  In total – that is 261,336 submitted pages.
    How many of these pages are indexed in Google?More than expected: 415,000 pages:Why is the number of indexed pages exceeding the number of submitted URL’s?Maybe there are some hidden sitemap.xml files that I have not found, maybe there is a hidden sitemap.html with additional URLs, and maybe there are some duplicate pages.

    But still, it is perfectly normal.

    It is important to emphasize – this is NOT an image sitemap.xml . These pages should be indexed on the main index.

  3. Handling thin content – this is smart, I really like what they have done here. Surprisingly, some of the queries could not match with any given images, which means that these pages are empty. For example:sorry we could't find any streams results right nowI was curious about it, so I searched for this query on Google to find these pages:site search operatorThese pages are deploying Meta Robots noidex, nofollow:noindex, nofollow
  4. Optimization – Let’s have a look on how most of the Meta Titles are constructed: Looking a bit familiar? Here is the Pinterest optimization for most of the pages:pinterest meta titles

 

Why there was such a massive increase during the last months?

I have to note that without any access to their Google Analytics and Search Console (although we still have alternatives to Google fetch and render) and without any information about what changes were made, it is quite challenging to run into conclusions.

      • As you might know, you can use Wayback Machine to find old robots.txt files. It was a pleasant surprise to find out you can also view historic sitemap.xml files!It enables us to monitor and check for any past changes.First, I tried to compare between the different robots.txt files with diffchecker.There were some minor changes, but the main index sitemap.xml file was still there. So now I was looking for any old versions of this file.Up to April 2018, this was the message:Which let me think that Google might have difficulties processing it.The first record we have of a valid sitemap.xml was in August 2018:sitemaps xml waybackmachineSurprisingly, this time the index sitemap contained 20 (!) sitemaps in comparison to only 6 at the moment.Drumroll – The sitemap.xml index was replaced between September 2018 and January 2019 (unfortunately, these are the months with no records in the web archive)
      • As I don’t have access to the index status, I wanted to check if the alleged issue with the sitemap.xml had any influence on the number of indexed pages.While nobody has a way of knowing this for sure without any access, there is a way around it.I used SEMRUSH to download the list of ranked keywords and then remove duplicates just to get the total number of ranked pages – the result was 7,023 pages that are ranked. While checking the historic data from April 2018, the difference was not that significant – 6,134 pages.My assumption that the indexation issue fix had contributed to this, might not have a 100% fit.I had to dig even deeper.
      • Links – what about their internal links and external links graph? Perhaps something changed there?Nope. Even with Ahref’s internal backlinks checker, I was not able to find anything:

       

      • Panda algorithm – my last hint was hanging on something important I mentioned before – currently, every page that does not have any content (“sorry we couldn’t find any results”) is removed from the index.My assumption is that thousands of pages with thin content (or should I say zero content) are bound to signal “bad quality”.Adding to that, we are optimizing our crawl budget by encouraging Google to crawl and index only the important, valuable pages.So by “noindexing” those pages (plus removing them from the gigantic sitemap.xml file) I believe that the quality update through the last Q of 2018 might had a positive effect.

      TL;DR

      I have to emphasize, despite the provocative headline, Bing does not “steal” anything, this is not black hat SEO in any way and I find it 100% legit. On the contrary, I find it brilliant.

      I have decided to write and analyze this case because I have found it fascinating, and I believe others will do as well.

      So, what is your take? Why do you think Bing traffic (both in Google images and Google Universal search) had skyrocketed? I would love to hear your thoughts.

Posted by Roey Skif

7 comments

  1. Gefen

    Nice one,

    Couldn’t get a screen shot into the comment, so:

    When commanding – site:https://www.bing.com/discover in google images, there is one big thing in common (not sure if it’s a coincidence):

    ~90% of the results are BEST X…(Ctrl+F counts 182 in the first scroll page)

    When looking at the DOM of Bing it seems that it is built as a list (“li” attributes) while Pinterest doesn’t.

    Also Bing UX is more like a simple list experience while Pinterest is complex and interactive…

    Made me feel it’s also about intent.

  2. Daily Search Forum Recap: April 12, 2019 – AdvertisSEO

    […] How Bing Discovery “Steals” Organic Traffic from Google Search, TL;DR SEO […]

  3. Daily Search Forum Recap: April 12, 2019 | The SEO Dr.

    […] How Bing Discovery “Steals” Organic Traffic from Google Search, TL;DR SEO […]

  4. Dylan

    Fascinating read. Just so I understand: Because the deep internal links are missing (ie only category pages available) you think the only way Google discovers the belly/long tail pages (“I love you colouring pages”) are via XML sitemaps (and possibly over time external links)? In my opinion that’s the most interesting part because I’ve always thought internal links are one of the strongest mechanisms to help rank content. I suppose the Authority of Bing’s site helps it rank via XML sitemaps alone. Great article, and good digging! Thanks.

    1. Roey Skif

      Thanks Dylan!

      After doing my research and digging more into the website, I can tell that it seems there are no internal links to these pages
      As you can see in here (any many other instances) you can still rank without internal links, especially if you have a high authority site and you try to rank for long tails with no competition

      However, I wouldn’t recommend it, internal links are super important

  5. Pete

    Hi there – it’s really difficult to actually see any of the charts embedded in this article as they are too small to properly read the figures and there is no way to expand to view them full sized.

    1. Roey Skif

      Thanks, Pete for your feedback
      I’ve fixed the sizes today, hope it looks better

Leave a Reply