With the Internet
developing at an incredible range, it's coming more and more difficult to get a
high page ranking. There are 100s of dynamics that serve engines decide how to
rank a page. Learned how some of these dynamics work, will definitely help you
increase your sites exposure on the Internet. One of the dynamics that you need
to take into condition is "meta robots". Trust it or not, but you can
really control the robots that crawl your sites in a number of directions,
which will undoubtedly help your ranking.
Meta robots are unique
to each page on a website, and are used by search engines to direct crawlers on
a single page. For the most part you want as many crawlers as possible to
access your site; however, there are certain instances where you may not want
pages or specific areas of your site being crawled. Robot tags will allow you
to keep control of what is being scanned by search engines, and preventing them
from returning search query results, while still being able to transfer their
link value to a different pages.
Why would you prefer to
block a page from being scanned, you are wondering? Here are a some reasons:
for security, you may want to protect private information such as contact
information, and addresses. Block spam crawlers from trying to collect
sensitive information.
Mostly, you'll use the
default values of meta name="robots" content="index,
follow", which is the same thing as having no meta robots tag in the
least. For duplicate pages you will want to use meta name="robots"
content="noindex, follow", basically you do not want the duplicate
page indexed, but you want the links on your page to hold value. With
user-generated content, meta name="robots" content="index,
nofollow", would be appropriate. The page will still be indexed, but the
links will hold no value. When trying to protect private information, meta
name="robots" content="noindex, nofollow", would be used to
block everything.
To get started you'll
need to create a robots.txt file, that lists all your pages you want to use the
meta robot tag for. Than insert the appropriate tags mentioned above on each
page. Every spider or bot that enters your site should read the robot.txt file
which will tell them what not to index and where not to go on your website.
0 comments:
Post a Comment