db-koji01 slowness

Stephen John Smoogen smooge at gmail.com
Thu Apr 16 20:26:47 UTC 2015


+100 from me.

On 16 April 2015 at 14:23, Patrick Uiterwijk <puiterwijk at redhat.com> wrote:

> +1 from me, though you might need to add a <Directory /var/www/html> with
> allow from all.
>
>
> ----- Original Message -----
> > ok. Next thing... I notice that some queries hitting particularly hard
> > are from spiders.
> >
> > Back in the puppet days, koji disallowed all spidering.
> >
> > I never have seen useful search results from it.
> >
> > So, I'd like to block all the spiders and see if that helps out load
> > issues.
> >
> > robots.txt would have:
> >
> > User-agent: *
> > Disallow: /
> >
> > +1s?
> >
> > kevin
> > --
> > diff --git a/roles/koji_hub/files/kojiweb.conf
> > b/roles/koji_hub/files/kojiweb.conf index 86abd2e..8334274 100644
> > --- a/roles/koji_hub/files/kojiweb.conf
> > +++ b/roles/koji_hub/files/kojiweb.conf
> > @@ -6,6 +6,8 @@ KeepAlive On
> >  Alias /koji "/usr/share/koji-web/scripts/wsgi_publisher.py"
> >  #(configuration goes in /etc/kojiweb/web.conf)
> >
> > +Alias /robots.txt /var/www/html/robots.txt
> > +
> >  <Directory "/usr/share/koji-web/scripts/">
> >      Options ExecCGI
> >      SetHandler wsgi-script
> > diff --git a/roles/koji_hub/tasks/main.yml
> > b/roles/koji_hub/tasks/main.yml index 0b6cd82..a5fc795 100644
> > --- a/roles/koji_hub/tasks/main.yml
> > +++ b/roles/koji_hub/tasks/main.yml
> > @@ -171,6 +171,13 @@
> >    notify: restart httpd
> >    when: env != "staging"
> >
> > +- name: koji robots.txt config
> > +  copy: src=robots.txt dest=/var/www/html/robots.txt
> > +  tags:
> > +  - config
> > +  - koji_hub
> > +  notify: restart httpd
> > +
> >  - name: kojira log dir
> >    file: dest=/var/log/kojira owner=root group=root mode=0750
> > state=directory tags:
> >
> > _______________________________________________
> > infrastructure mailing list
> > infrastructure at lists.fedoraproject.org
> > https://admin.fedoraproject.org/mailman/listinfo/infrastructure
> _______________________________________________
> infrastructure mailing list
> infrastructure at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure
>



-- 
Stephen J Smoogen.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150416/27058236/attachment-0001.html>


More information about the infrastructure mailing list