There is a site (that shall remain nameless on this forum!) called www.thesite.com.
When I Google it, the Google results page gives a nicely laid out presentation of the site's sections, with links to each section, like this:
[U]Sitename - Your guide to blah blah[/U]
Welcome to Sitename, blah blah...
www.sitename.com/ - 32k - [U]Cached[/U] - [U]Similar Pages[/U]
[U]Section1[/U] [U]Section4[/U]
[U]Section2[/U] [U]Section5[/U]
[U]Section3[/U] [U]Section6[/U]
How do I achieve this for my site? Is it something on their site?
Also, they appear to be missing a robots.txt.
www.thesite.com/robots.txt yields 404 doc not found.
I thought a robots.txt would be mandatory for a site like this (its a major site).
Thanks any advice!