I've used "ROBOTS.txt" files before to restrict crawlers from indexing certain pages on a site... but was wondering if telling them to index the rest (which I guess is being done here) is actually recommended... and via the actual source code like this.. instead of a separate text file? Any SEO advantage to this?
@lex
-- This list is a free service of LassoSoft: http://www.LassoSoft.com/ Search the list archives: http://www.ListSearch.com/GoLive/Browse/ Manage your subscription: http://www.ListSearch.com/GoLive/
Search
Lasso Programming
This site manages and broadcasts several email lists pertaining to Lasso Programming and technologies related and used by Lasso developers. Sign up today!