How to find & fix broken external links (the right way)

2. 301 redirects to the homepage

As stated by Google do not ever link all your 404 pages to the homepage by default. If there is no relevancy, Google will consider them as soft 404’s and devalue them.

Do the same process mentioned above in Screaming Frog to follow the redirects final destination and check if the redirected path makes sense. Is it a product page or an old blog post? Does it make sense to redirect it to your homepage, or maybe there might be other pages that will be more relevant?

For example, think about a subcategory page that does not exist anymore – will it be better to redirect it to the homepage or a similar/closer category page?

3. Soft 404 pages

What are soft 404 pages?

The definition of soft 404 page is “not found” page which returns a 200 header instead of 404 (or 410 on some instances).

How can you check if you have soft 404’s on your site?

On the domain, type a random string after the root domain:
Example.com/jajaja

Check the header for this page (I use Ayima chrome extension) – if the header returns 404 you can easily find them with Ahrefs and redirect the:

However, if you see a 200 header, it means your pages are being considered as soft 404 and you will not be able to identify broken links if there are any.

Step 1 – fix it. Ask your developer to return 404 header for all error pages.

Step 2 – export the list from the Ahrefs tool again (this time only with the 200 headers) and run them with Mode -> List

Step 3 – Check all page titles – you will expect to see the same title for your 404 page (404 not found or something similar).

Another option is to check the error pages by their H1:

Filter it by the H1 column in the screaming frog and you are basically done.

Check them by either the Meta Title tab or the H1:

Step 4 – After identifying all soft 404’s, you can redirect them to the most relevant pages.

4. Page is currently not in our index

According to ahrefs, that indicates that the target page is not in their index.

While I have not found any issues that should prevent these pages from getting indexed, we need to find a solution for this.
It is problematic because we currently do not know what is the status code of those pages – are they 200, 404 or being appropriately redirected?

The solution – export the list from ahrefs and crawl it in Screaming Frog. You will get the status codes for every URL on the list which will give us the information which URL’s need to be redirected.

5. Robots.txt

What if I will tell you about a scenario that all your broken links are being redirected properly, but still do not pass any value? (AKA link juice)

This can happen much more often than you would expect, in case of overlooked deployment of the robots.txt file.

In case you disallow specific folders or URLs via the robots.txt, Google will not crawl it and will not pass the link juice

So how can you check it?

Either via Google Search Console’s robots.txt tester (which will make this job a little tedious), or via SEOBOOK robots.txt analyzer.

After adding your robots.txt you can export the list of backlinks and check if some of the rules block any of these urls.

If some lines are blocking the redirects, make sure you remove it (but carefully).

6. Meta robots noindex, nofollow

Similar to section no. 5 (regarding robots.txt), this one can be easily missed, as most of the link analysis tools do not state the status of the robots.txt nor the Meta Robots tag.

So even if these pages return a 200 status code, it still does not guarantee Google will index and crawl them.

Let’s say you have dozens of out-of-stock product pages that you have decided to block with noindex a while ago. How can you make sure you’re not missing any of those?

Back to the Screaming Frog again, after crawling the list of given URLs, go to the “directives” column and filter by noindex or nofollow:

If you find pages with backlinks and contradicting Meta Robots tag (noindex/nofollow or both) make sure to decide what to do with this page:

• Remove the Meta robots tag – in case you find this pages useful
• Perform a 301 redirect to the most relevant pages

7. Unknown old domains

You are starting to work with a new client and their link graph is nothing to brag about. But wait!

You do not need any fancy SEO tools to check it. Just ASK your client if there are any old domains or properties, it can be either their main website or other old domains that are in their disposal.

You will be surprised to find how many times clients don’t understand the importance behind their old domains and do not even bother to mention it.

And NO, you will not see it in the Ahrefs or any other given tool if nobody had bothered to redirect them.

Lately, I myself even accidentally found an old domain for one of my clients. In their mind, it was an old website, and they were not slightly aware of its potential (although that’s always the first question I ask).

The result? They forgot to renew the domain ownership, someone else had bought it, and just like that – more than 400 relevant referring domains were flushed down the drain:

8. Toxic links

Before blindly asking to redirect 404 pages, you should first ask yourself –

It’s not only the NUMBER of referring domains, but also their QUALITY.

Take into account that in some instances, if a given page returns a 404 header (or 410 for that matter), maybe it was intentionally.

Perhaps the former SEO company was smart enough to remove some old pages that had toxic spammy links that had been built manually in the past.

So the idea is to really analyze those links BEFORE deciding whether to redirect them or not.

Make sure you understand well what is an unnatural link before taking action (although this guide is originally from 2013, it is still relevant to 2018).