White House site hides Iraq-related pages
As revealed by Atrios, the White House is using its robots.txt file to hide dozens of folders with iraq in their names from Google and other search engines.An example of the content affected is the White House Special Report Operation Iraqi Freedom. Neither the main page nor any of the other pages in this report are cached by Google.
For those unfamiliar with robots.txt, it's part of the Robots Exclusion Protocol, a way to tell search engines which parts of your site they should never index.
Add a Comment
All comments are moderated before publication. These HTML tags are permitted: <p>, <b>, <i>, <a>, and <blockquote>. This site is protected by reCAPTCHA (for which the Google Privacy Policy and Terms of Service apply).
Social Media
Subscriptions
Blogroll
Projects
Statistics
This blog has been published since Nov. 7, 1999 (a span of 9,239 days).
Thank you for visiting Workbench.
