Robots Prohibiting Protocol
There may be areas of your site that you do not want to emerge in search engine listings, such as customer information, or other responsive data. Indeed, there may be reasons why you do not want your site to become visible at all in some or all search engines. If this is the case, the way to avoid being listed unintentionally is to employ a Robots Exclusion Protocol (REP). This is a normal text file that contains commands to robots about whether you will allow them to index parts of your site or not, and it is the first thing a robot will check when it arrives at your site.
It is in the benefit of the robot not to waste its time indexing your site if there is no need, and also not to index all on the site if there are large areas that are not relevant, and so nearly all robots are happy to follow the instructions in the REP. This means that the robots do not squander their bandwidth, nor do they waste yours.
The instructions in a REP are written in a method resembling a coding language, although in practicality they are just variables composed by the robot, and translated into commands by the robot. This means that the commands have to be written in a specific format in order to be recognized by the robot.
It is in the benefit of the robot not to waste its time indexing your site if there is no need, and also not to index all on the site if there are large areas that are not relevant, and so nearly all robots are happy to follow the instructions in the REP. This means that the robots do not squander their bandwidth, nor do they waste yours.
The instructions in a REP are written in a method resembling a coding language, although in practicality they are just variables composed by the robot, and translated into commands by the robot. This means that the commands have to be written in a specific format in order to be recognized by the robot.
Labels: search engine optimization, seo
posted by jarabni @ 1:36 AM,
0 Comments: