change request - make robots.txt work on fho

[Matt_Domsch at Dell.com]

+1

--
Matt Domsch
Technology Strategist
Social Media and Community Professional
Dell | Office of the CTO

From: infrastructure-bounces at lists.fedoraproject.org [mailto:infrastructure-bounces at lists.fedoraproject.org] On Behalf Of Ricky Elrod
Sent: Tuesday, May 10, 2011 1:02 PM
To: Fedora Infrastructure
Subject: change request - make robots.txt work on fho

Wanted to do this before freeze, but never had a chance -- let's get robots.txt working on hosted1 and try to bring down cpu load and improve page load times a bit.

Thoughts/+1's please. robots.txt is already there and has been for a long time, but nothing has told apache to use it, because apache requests go straight to trac.

robots.txt is set to block crawlers from accessing fh.o/*/browser/* (which is the trac source code browser) -- as per https://fedorahosted.org/fedora-infrastructure/ticket/1848



diff --git a/configs/web/fedorahosted.org.conf b/configs/web/fedorahosted.org.conf
index d0f7139..8f1f2e1 100644
--- a/configs/web/fedorahosted.org.conf
+++ b/configs/web/fedorahosted.org.conf
@@ -2,6 +2,9 @@
     ServerName fedorahosted.org<http://fedorahosted.org>
     ServerAlias www.fedorahosted.org<http://www.fedorahosted.org>

+    # Make robots.txt be used.
+    Alias /robots.txt /srv/web/fedorahosted.org/robots.txt<http://fedorahosted.org/robots.txt>
+
     Redirect 301 / https://fedorahosted.org/
 </VirtualHost>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20110510/117f3a6f/attachment-0001.html