Cant find robots.txt file

Author Topic: Cant find robots.txt file  (Read 1946 times)

Offline ofpeb777Topic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You 0
  • Posts: 1
  • Karma: 0
Cant find robots.txt file
« on: 06-13-2016, 12:25:55 »
Hi,

Does anyone know where one's robots.txt file would most likely be found. I know that its suppose to be in the root directory. I would think it would be in a txt file, but i can't find it and it's presently set up to not allow my iweb site to be  crawled. "Disallow /" I need to get rid of the '/' but i can't find the file in file manager. I've searched online to no avail as all anyone talks about is - just go to http://yoursite.com/robots.txt. I've done that and know that it's set up to not allow the site to be crawled. I've talked to bluehost where the site lives and they can't seem to help either.

Thanks, joe


Offline RH-Calvin

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 52
  • Posts: 1018
  • Karma: -1
  • Gender: Male
    • Cheap VPS Hosting
Re: Cant find robots.txt file
« Reply #1 on: 06-16-2016, 01:25:19 »
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

Offline TomClarke

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 17
  • Posts: 896
  • Karma: 0
  • Gender: Male
    • Custom Writing Service & Content Marketplace
Re: Cant find robots.txt file
« Reply #2 on: 06-21-2016, 01:44:53 »
Robots.txt file usually resides in your site's root folder. You will need to connect to your site using an FTP cliebt or by using cPanel file manager to view it. It is just like any ordinary text file, and you can open it with a plain text editor like Notepad. If you don't have a robots.txt file in your site's root directory, then you can always create one. All you need to do is create a new text file on your computer and save it as rebots.txt. Next, simply oupload it to your site's root folder.

Your ideas, our writings - Writology.com

Offline SEO.Ninja

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 3
  • Posts: 109
  • Karma: 0
  • Gender: Female
    • SEO Ninja
Re: Cant find robots.txt file
« Reply #3 on: 07-07-2016, 06:33:52 »
Yes, you can also find robots.txt file.
The website you are trying to open the file Robots.txt, Please open the site after that use "site URL /robots.txt Please put on your browser will open the file Robots.txt
Example:
You want to check Robots.txt file of the site "www.abcsite.com".
Please open in your browser: www.abcsite.com/robots.txt

Offline heyzal

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 39
  • Karma: 0
  • Gender: Male
    • viajes a la india
Re: Cant find robots.txt file
« Reply #4 on: 07-08-2016, 00:00:37 »
Yes, you can also find robots.txt file.
The website you are trying to open the file Robots.txt, Please open the site after that use "site URL /robots.txt Please put on your browser will open the file Robots.txt
Example:
You want to check Robots.txt file of the site "www.abcsite.com".
Please open in your browser: www.abcsite.com/robots.txt


Thanks, Now a days It is working or not ? I think It is compulsory or not, please share some valuable information ??


Offline PrimoPierotz

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 24
  • Karma: 0
    • World Open News
Re: Cant find robots.txt file
« Reply #5 on: 07-09-2016, 10:09:26 »
Thanks, Now a days It is working or not ? I think It is compulsory or not, please share some valuable information ??

It isn't compulsory, but it can help robots to analyze your pages in the way you prefer.

 

Related Topics

  Subject / Started by Replies Last post
24 Replies
3327 Views
Last post 05-10-2012, 01:38:00
by ring2012
32 Replies
7527 Views
Last post 04-24-2016, 20:52:14
by nguyenmailan
1 Replies
1120 Views
Last post 06-16-2014, 13:10:28
by Johnny
0 Replies
496 Views
Last post 04-02-2016, 04:57:02
by chinmay.sahoo
2 Replies
298 Views
Last post 11-25-2016, 04:39:23
by CreativeDreamrz