Retrospective freeze break: Block riddler.io

[Pierre-Yves Chibon]

On Fri, Apr 17, 2015 at 01:36:50PM +0200, Patrick Uiterwijk wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> Hi,
> 
> Earlier today, the riddler.io bot pulled Bodhi under by performing
> a lot of heavy requests.
> 
> I know from experience that this bot does NOT listen to robots.txt,
> even though the website says it does (it hit my personal webserver
> as well at an earlier time), so I have currently blocked it by user
> agent.

Is this something we could report to them?

>  1 files changed, 3 insertions(+), 1 deletions(-)
> [puiterwijk at lockbox01 files]$ git show HEAD
> commit a635e91849f29dd3c146b8067f602871ff959f00
> Author: Patrick Uiterwijk <puiterwijk at redhat.com>
> Date:   Fri Apr 17 11:29:48 2015 +0000
> 
>     Block the Riddler.io bot from accessing bodhi
>     
>     This bot does NOT follow robots.txt, even though it announces that
>     it does, and it hits Bodhi so much that it pulls the bodhi servers
>     under.
>     
>     An email has been sent to the maintainer of the bot, but for the
>     time being, let's block it from using any bodhi resources.
>     
>     Signed-off-by: Patrick Uiterwijk <puiterwijk at redhat.com>
> 
> diff --git a/roles/bodhi/base/files/bodhi-app.conf b/roles/bodhi/base/files/bodhi-app.conf
> index 3e10a59..3f44964 100644
> - --- a/roles/bodhi/base/files/bodhi-app.conf
> +++ b/roles/bodhi/base/files/bodhi-app.conf
> @@ -13,6 +13,8 @@ Alias /updates/tg_widgets/tgmochikit/packed/MochiKit/MochiKit.js /usr/lib/python
>  
>  <Directory /usr/share/bodhi>
>    WSGIProcessGroup bodhi
> - -  Order deny,allow
> +  SetEnvIf User-Agent Riddler GoAway=1
> +  Order allow,deny
>    Allow from all
> +  Deny from env=GoAway
>  </Directory>

+1 for me, thanks!

Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150417/df5238b3/attachment.sig>