Proper Robot Tag UsageThere is a special HTML meta tag to tell robots not to index the content of a page, and/or not scan it for links to follow.
Example Usage: <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
There are two important considerations when using the robots Meta Tag.
- robots can ignore your Meta Tag. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
- the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (maybe on some other site), and so still arrives at your undesired page.
How to write a Robots Meta Tag
Where to put it
What to put into it
- NOINDEX, FOLLOW
- INDEX, NOFOLLOW
- NOINDEX, NOFOLLOW