SMO

Robots.txt Celebrates 20 Years of Blocking Search Engines

Today is the 20th anniversary of the robots.txt directive being available for webmasters to block search engines from crawling their...

Sourabh Jul 1, 2014 1 min read