Google search bot (Googlebot) parses robots.txt file to find excluded sections of the site like any good webbots.

Allen from CenterNetworks made a startling discovery.

Yahoo! spent the July 4 weekend in discussions with its lead adviser, Goldman Sachs, and potential bid partners including Time Warner in a desperate bid to defend itself from shareholders meeting on August 1 to vote on whether to re-elect their board.