On Mon, Oct 20, 2008 at 12:13:23 -0400,
Mike McLean <mikem(a)redhat.com> wrote:
These links are all reachable via the web ui, any crawler might will hit
them. I suggest adding a robots.txt to keep crawlers out.
Or meta tags directed at robots. Doing things that way has some advantages
over robots.txt.