Cant find robots.txt file

Author Topic: Cant find robots.txt file  (Read 2892 times)

Offline ofpeb777Topic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You 0
  • Posts: 1
  • Karma: 0
Cant find robots.txt file
« on: 06-13-2016, 12:25:55 »
Hi,

Does anyone know where one's robots.txt file would most likely be found. I know that its suppose to be in the root directory. I would think it would be in a txt file, but i can't find it and it's presently set up to not allow my iweb site to be  crawled. "Disallow /" I need to get rid of the '/' but i can't find the file in file manager. I've searched online to no avail as all anyone talks about is - just go to http://yoursite.com/robots.txt. I've done that and know that it's set up to not allow the site to be crawled. I've talked to bluehost where the site lives and they can't seem to help either.

Thanks, joe


Offline RH-Calvin

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 77
  • Posts: 1143
  • Karma: -1
  • Gender: Male
    • Cheap VPS Hosting
Re: Cant find robots.txt file
« Reply #1 on: 06-16-2016, 01:25:19 »
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

Offline TomClarke

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 44
  • Posts: 893
  • Karma: 0
  • Gender: Male
    • Custom Writing Service & Content Marketplace
Re: Cant find robots.txt file
« Reply #2 on: 06-21-2016, 01:44:53 »
Robots.txt file usually resides in your site's root folder. You will need to connect to your site using an FTP cliebt or by using cPanel file manager to view it. It is just like any ordinary text file, and you can open it with a plain text editor like Notepad. If you don't have a robots.txt file in your site's root directory, then you can always create one. All you need to do is create a new text file on your computer and save it as rebots.txt. Next, simply oupload it to your site's root folder.

Your ideas, our writings - Writology.com

Offline SEO.Ninja

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 23
  • Posts: 206
  • Karma: 0
  • Gender: Female
    • SEO Ninja
Re: Cant find robots.txt file
« Reply #3 on: 07-07-2016, 06:33:52 »
Yes, you can also find robots.txt file.
The website you are trying to open the file Robots.txt, Please open the site after that use "site URL /robots.txt Please put on your browser will open the file Robots.txt
Example:
You want to check Robots.txt file of the site "www.abcsite.com".
Please open in your browser: www.abcsite.com/robots.txt

Offline heyzal

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 1
  • Posts: 51
  • Karma: 0
  • Gender: Male
    • viaje a la india
Re: Cant find robots.txt file
« Reply #4 on: 07-08-2016, 00:00:37 »
Yes, you can also find robots.txt file.
The website you are trying to open the file Robots.txt, Please open the site after that use "site URL /robots.txt Please put on your browser will open the file Robots.txt
Example:
You want to check Robots.txt file of the site "www.abcsite.com".
Please open in your browser: www.abcsite.com/robots.txt


Thanks, Now a days It is working or not ? I think It is compulsory or not, please share some valuable information ??


Offline PrimoPierotz

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 36
  • Karma: 0
    • World Open News
Re: Cant find robots.txt file
« Reply #5 on: 07-09-2016, 10:09:26 »
Thanks, Now a days It is working or not ? I think It is compulsory or not, please share some valuable information ??

It isn't compulsory, but it can help robots to analyze your pages in the way you prefer.

 

Related Topics

  Subject / Started by Replies Last post
24 Replies
3793 Views
Last post 05-10-2012, 01:38:00
by ring2012
1 Replies
1294 Views
Last post 06-16-2014, 13:10:28
by Johnny
0 Replies
697 Views
Last post 04-02-2016, 04:57:02
by chinmay.sahoo
2 Replies
2625 Views
Last post 11-25-2016, 04:39:23
by CreativeDreamrz
0 Replies
125 Views
Last post 01-17-2017, 06:39:23
by jeyavinoth