Discussion in 'SEO & Traffic Discussion' started by fish2012, Sep 27, 2012.
I would like to know that what is robot.txt and why it use in web pages.... thanks
Quit spamming the forum so people see your "live links".... Read the forum rules and follow the like the rest of us need to.. They should ban you but probable wont..
Spammer, don't use live links. Moderators please step in. I have seen this fellow posting live spamming links in other threads also. Take action!.
Robots.txt is a file through which also you can guide search engines crawl or not to crawl certain sections of your website.
Google specifically follows instructions given in this robots.txt file..
Robot.txt is a map through which you show your pages to google and increase index rate
Why ask here, just Google it F.C.O.L.
Robot.txt is a file which gives instructions to the Google about the website that what section or part of the site must crawl and other not. There are some disallow options available which shows to robots of search engines that this part is official and not for their use so they don’t have to crawl that part.
Robots.txt is a text file you put on your site to tell search robots that which website page you dont want to crawl.
instead of doing discussion about spamming the forum or not . i will share the ans.
Robots.txt is a text file that tells search engine spiders , also known as search engine robots, which parts of your website they can enter and which parts they can’t .
Robots.txt are files you place in the root directories of the pages you don't want Google to crawl which means they won't index these pages and none of them will be found in the search results. Best example would be a Members page or download page. If a member has to make a purchase to go to a downloads page, make sure the page has a robots.txt file in the root folder so the big 'G' won't allow people to find it in search engines, stopping users from going direct to your download pages and taking all your products without a purchase.
Hope this info helps.
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit. It is important to explain that robots.txt is not a way from preventing search engines from crawling your site.
Robots.txt are files contained in the root directory of a site that allows you to control certain pages to be indexed by the search engine spiders.
What I do know, robots.txt is a list of commands used to instruct robots search engines (like Google, Bing, Yahoo, etc.) index file or files on the server. I think this is important, because Search Engines in mengcrawling a website it blindly. and sometimes there is a folder in your website or confidential files that do not want us to tell the search engines. (because if the index by search engines, it can be seen also by others).
Robots.txt - a text file located on a site that is intended for search engine robots. In this file, the webmaster can specify the index of the site for all the robots at once, and for each search engine individually.
Use any text editor (such as Notepad or WordPad'om), create a file called "robots.txt" and fill it out according to the rules. After this, you must download the file to the root directory of your website.
Admin, please close this thread. It's just being used for spam. Look at the repeat answers from new members with just 2 or 3 posts.
Robots.txt can be called a file that allows you to adjust the flow of the search engine robots (search engines) on our site. For web robots.txt which we can arrange to index and which ones should not. and we also can avoid duplicate content.
A Robot.txt file turns all your .txt files into robots Happy New Year.
Robots.txt is a text file which describe spiders of search engine and crawls of your website.
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note â€œPlease, do not enterâ€ on an unlocked door â€“ e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naÃ¯ve to rely on robots.txt to protect it from being indexed and displayed in search results.
Whilst it doesn't happen very often it is useful to check the robots.txt of sites for download directories. Sometimes unsuspecting site owner will put dl directories in the robots.txt file to prevent them being indexed. There are of course other ways to protect downloads (not against seasoned BHT'ers of course).
Separate names with a comma.