Crawl Error

Author Topic: Crawl Error  (Read 270 times)

Offline logosoukigniteTopic starter

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 17
  • Karma: 0
  • Ecommerce solutions for small businesses
    • Ecommerce Solutions for Small Business - IGNITE
Crawl Error
« on: 06-02-2020, 09:29:47 »
Crawl Errors page provides details about the URLs that Google could not successfully crawl or that returned an HTTP error code.


Offline brill_mindz

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 4
  • Posts: 60
  • Karma: 0
    • Best Mobile App development company in Saudi Arabia
Re: Crawl Error
« Reply #1 on: 06-03-2020, 02:33:10 »
Crawl errors occur when a search engine tries to reach a webpage but fails to do so. There are two types of crawl errors:

Site errors (which mean your entire site can’t be crawled)
URL errors (which relate to one specific URL per error)
Site Errors in Google Search Console

DNS Errors
A DNS (Domain Name System) error means that a search engine can’t communicate with your server either because the server is down or because there's an issue with the DNS routing to your domain. Usually, this is a temporary issue.

Server Errors
When you see this type of error for your URLs, it means that Googlebot wasn’t able to access your URL, the request timed out, or your site was busy. This can also mean that your website has so many visitors that the server just couldn’t handle all the requests. Most of these errors are defined as 5xx status codes.

Robots failure
Before Googlebot crawls your website, it tries to crawl your robots.txt file to see if there are any pages you’d rather not have indexed. If that bot can’t reach the robots.txt file (i.e. robots.txt file doesn’t return a 200 or 404 HTTP status code), Google will postpone the crawl rather than risk crawling URLs that you don’t want to be crawled. That’s why you always want to make sure the robots.txt file is available.

URL Errors in Google Search Console

Common URL Errors
Common URL errors occur when a search engine tries to crawl a specific web page. Among them might be an occasional DNS error or server error for that specific URL.

Mobile-Only URL Errors
This type of error refers to the crawl failure that occurs on smart devices. If your website is responsive, mobile-only URL errors are unlikely to appear.

The most usual mobile-only URL errors are faulty redirects. Some websites use different URLs for desktop and smartphone users. A faulty redirect occurs when a desktop page incorrectly redirects mobile users to a page not relevant to their query.

Offline Nizam18

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 13
  • Karma: 0
  • Gender: Male
    • Digital Transformation Company
Re: Crawl Error
« Reply #2 on: 06-15-2020, 11:45:54 »
Crawl errors occur when a search engine tries to reach a page on your website but fails at it. Let’s shed some more light on crawling first. Crawling is the process where a search engine tries to visit every page of your website via a bot. A search engine bot finds a link to your website and starts to find all your public pages from there. The bot crawls the pages and indexes all the contents for use in Google, plus adds all the links on these pages to the pile of pages it still has to crawl. Your main goal as a website owner is to make sure the search engine bot can get to all pages on the site. Failing this process returns what we call crawl errors.
Nizam Ali,
Marketing Executive - Digital Transformation Company [nofollow].

Offline eSage IT

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 1
  • Posts: 88
  • Karma: 0
  • Gender: Male
    • eCommerce Marketing Agency
Re: Crawl Error
« Reply #3 on: 06-16-2020, 05:13:00 »
Crawl errors occur when a search engine tries to reach a webpage on a website but fails to do so. Crawl errors are of different types such as Site errors, URL errors, and more.

 

Related Topics

  Subject / Started by Replies Last post
0 Replies
1040 Views
Last post 08-27-2016, 11:37:14
by aghosh
3 Replies
3969 Views
Last post 11-29-2016, 07:48:21
by damponting44
0 Replies
979 Views
Last post 10-17-2017, 02:34:40
by shabrishn
5 Replies
5453 Views
Last post 07-11-2018, 05:37:57
by organictextiles
1 Replies
2480 Views
Last post 09-29-2018, 19:25:40
by marrysmith101