Cant find robots.txt file

Started by ofpeb777, 06-13-2016, 12:25:55

Previous topic - Next topic

ofpeb777Topic starter

Hi,

Does anyone know where one's robots.txt file would most likely be found. I know that its suppose to be in the root directory. I would think it would be in a txt file, but i can't find it and it's presently set up to not allow my iweb site to be  crawled. "Disallow /" I need to get rid of the '/' but i can't find the file in file manager. I've searched online to no avail as all anyone talks about is - just go to http://yoursite.com/robots.txt. I've done that and know that it's set up to not allow the site to be crawled. I've talked to bluehost where the site lives and they can't seem to help either.

Thanks, joe
  •  


RH-Calvin

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.


TomClarke

Robots.txt file usually resides in your site's root folder. You will need to connect to your site using an FTP cliebt or by using cPanel file manager to view it. It is just like any ordinary text file, and you can open it with a plain text editor like Notepad. If you don't have a robots.txt file in your site's root directory, then you can always create one. All you need to do is create a new text file on your computer and save it as rebots.txt. Next, simply oupload it to your site's root folder.


SEO.Ninja

Yes, you can also find robots.txt file.
The website you are trying to open the file Robots.txt, Please open the site after that use "site URL /robots.txt Please put on your browser will open the file Robots.txt
Example:
You want to check Robots.txt file of the site "www.abcsite.com".
Please open in your browser: www.abcsite.com/robots.txt

PrimoPierotz

Quote from: randiv on 07-08-2016, 00:00:37
Thanks, Now a days It is working or not ? I think It is compulsory or not, please share some valuable information ??

It isn't compulsory, but it can help robots to analyze your pages in the way you prefer.