&Latest tutorials

How to Plan a PPC Campaign

Whether you decide to embark on a pay-per-click (PPC) advertising campaign to add an extra string to your digital marketing bow, or because you want to use it as an alternative keyword research tool in today’s world of keyword (not provided), the ability to plan and manage your campaign holds the key to whether you achieve your objectives.

learn more
How to Tell People you’ve linked to them

If you’re crediting other websites with links out from your own, the onus is on you to tell them you’ve done it, otherwise your generosity may never be discovered!

learn more

Latest tweets

What are Crawl Errors?

The crawl errors analysis feature within Google Webmaster Tools will show you any errors that Google’s search spider – GoogleBot – has encountered when trying to crawl your site for indexing.

There are a number of common errors that you’ll encounter when looking at your crawl errors. These are listed below:


  • Soft 404 Errors– When the server does not return a proper 404 error.
  • Not Found Errors– When a page is ‘missing’ from your website; the most common type of 404 error.
  • 400 Errors– When the application used to access a specific webpage has accessed a page incorrectly or a request was somehow corrupted during the process.
  • 401 Errors – Usually, these are login errors caused by individuals trying to access pages without permission.
  • 403 Errors – Signifying a ‘forbidden’ page.Similar to 401 Errors, but there is no login function, so 403 errors will appear to everyone if a page cannot be accessed.
  • 500 Errors – Generic errors that could point to a range of internal server errors.This might mean your server is overloaded – common if you are on a shared server or have too much content for your server to handle – or that load balancers have failed, although there are several other possibilities, too.
  • Sitemap Errors – Errors with your Sitemap.xml file (we look at sitemaps in more detail later).
  • Timed Out – Either the server is busy or your load times are terrible, meaning GoogleBot has given up trying to load your page, which is never a good sign.


Site Errors

Within the crawl errors feature is a section for site errors. This page will tell you the specific pages where GoogleBot found a problem, enabling you to go into your site and fix them.

With 404 errors being the most common, what can you do to fix them?

404 Errors: Top Tips

Remember that a 404 error on your website might not be your fault.

For example, if you have a page on your website with the URL http://www.yourwebsite.com/blog, you might get a 404 error because someone has tried to access http://www.yourwebsite.com/blogg, either because of his or her own typo or from a link pointing to your site elsewhere on the internet.

The best thing to do here is to set up a 301-redirect for these pages to the ones the internet user was probably looking for. You should also ensure you have optimised your 404 page with links to your main pages, so anyone who does get a 404 error can still get onto your site without having to look at what they may have done wrong. This is much better than a blank screen that simply says, “The server returned a 404 error.”

Mark as Fixed

When you have dealt with each individual issue, you can “mark as fixed” to remove them from your Google Webmaster Tools account. It is worth using the ‘Fetch as Google’ function, too, so you can double check if you really have fixed the issue. If you haven’t, it will simply reappear the next time Google visits and crawls your site.

Written by
Posted · Jan 14, 2014
Categories · Indexing & Crawling.
Related Articles · Does My Hosting Plan Matter in Terms of SEO?. How to Install WordPress Plugins. How do 404 Errors Impact SEO?. What Is a Technical SEO Audit and It's Importance?. How to Optimise Your 404 Page. What Is the Value of Webmaster Tools?. How to check sitemap details?. How to generate and submit sitemaps?. The Importance of XML Sitemaps.

Do you like this post?


Increase your sales - get a FREE quote

Our clients, and what they say