Learn How to solve robots.txt error (5xx error)

[akki5577]

Hey guys, if you find 5xx (unreachable) error in your webmaster tool then do not worry I got a solution for that.
To solve error, You just need to contact your web hosting service provider and ask them for any issue. If you do not find any issue then login to your website's cpanel and open file manager. Then you need to find robots.txt file in public_html section

If you do not find the file then you must upload a new file. Please note that, If you're using WordPress, you might see a robots.txt file when you go to yoursite.com/robots.txt, but you won't be able to find it in public_html section. This is because WordPress creates a virtual robots.txt file. In this case you'll need to create a new robots.txt file.

Note : Save your robots.txt file as "robots" in your computer otherwise it will become "robots.txt.txt"

1.   Once you create the file, open your cpanel OR login to your FTP account
2.   Now go to file manager and select "Public_html"
3.   Then click on "Upload" and upload the file
4.   Now login to your webmaster account
5.   After that, "Crawl" in the left-hand sidebar.
6.   Then select "robots.txt Tester."
7.   Click "Test" , If the "Test" changes to "Allowed," that means your robots.txt is valid.

Now wait for at least 4 days to clear 5xx (unreachable) error.

[Neel Patel]

Till yet not faced 5xx errors. but this will be useful in future in case i face any of 5xx error.
Guys must read once. explained properly and easily understandable. 

Hey guys, if you find 5xx (unreachable) error in your webmaster tool then do not worry I got a solution for that.
To solve error, You just need to contact your web hosting service provider and ask them for any issue. If you do not find any issue then login to your website's cpanel and open file manager. Then you need to find robots.txt file in public_html section

If you do not find the file then you must upload a new file. Please note that, If you're using WordPress, you might see a robots.txt file when you go to yoursite.com/robots.txt, but you won't be able to find it in public_html section. This is because WordPress creates a virtual robots.txt file. In this case you'll need to create a new robots.txt file.

Note : Save your robots.txt file as "robots" in your computer otherwise it will become "robots.txt.txt"

1.   Once you create the file, open your cpanel OR login to your FTP account
2.   Now go to file manager and select "Public_html"
3.   Then click on "Upload" and upload the file
4.   Now login to your webmaster account
5.   After that, "Crawl" in the left-hand sidebar.
6.   Then select "robots.txt Tester."
7.   Click "Test" , If the "Test" changes to "Allowed," that means your robots.txt is valid.

Now wait for at least 4 days to clear 5xx (unreachable) error.

[kavyasharma561]

The robots.txt Tester tool shows you whether your robots.txt file blocks Google web crawlers from specific URLs on your site. For example, you can use this tool to test whether the Googlebot-Image crawler can crawl the URL of an image you wish to block from Google Image Search.

[evejones]

Hey guys, if you find 5xx (unreachable) error in your webmaster tool then do not worry I got a solution for that.
To solve error, You just need to contact your web hosting service provider and ask them for any issue. If you do not find any issue then login to your website's cpanel and open file manager. Then you need to find robots.txt file in public_html section

If you do not find the file then you must upload a new file. Please note that, If you're using WordPress, you might see a robots.txt file when you go to yoursite.com/robots.txt, but you won't be able to find it in public_html section. This is because WordPress creates a virtual robots.txt file. In this case you'll need to create a new robots.txt file.

Note : Save your robots.txt file as "robots" in your computer otherwise it will become "robots.txt.txt"

1.   Once you create the file, open your cpanel OR login to your FTP account
2.   Now go to file manager and select "Public_html"
3.   Then click on "Upload" and upload the file
4.   Now login to your webmaster account
5.   After that, "Crawl" in the left-hand sidebar.
6.   Then select "robots.txt Tester."
7.   Click "Test" , If the "Test" changes to "Allowed," that means your robots.txt is valid.

Now wait for at least 4 days to clear 5xx (unreachable) error.

Errors are really disgusting and these steps fluently describe how to overcome them with robots.txt. Thanks a lot for this!

[richardmsmith]

Robots file is for stopping the search engine to crawling the website. It is not showing the error. If you find this one than may be its another problem.

[organictextiles]

Hi

The robots.txt Tester tool shows you whether your robots.txt file blocks Google web crawlers from specific URLs on your site. For example, you can use this tool to test whether the Googlebot-Image crawler can crawl the URL of an image you wish to block from Google Image Search.