Member Avatar for nissy

What is Robot. Txt?
What is the use of it?

  • 5 Contributors
  • 4 Replies
  • 194 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by almostbob
Member Avatar for Mudassir Hasan

More details about Robot.Txt can be found at here moz(dot)com/learn/seo/robotstxt

Member Avatar for mindstreak

Robots.txt can be used to specify which webpages can be crawled by specific search engines.Also it can specify whether a webpage can be crawled by search engine or not.

Member Avatar for almostbob

Note the Difference

robot.txt is a an exercise in futility

robots.txt is a file containing control directives to search robots

summer semester has started,
the beginner html course questions have been asked again.

be worried,
if the coursework texts have got the name of such a basic element wrong,
the rest of the course will be total crap

Edited by almostbob
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.