Why is fedoraproject.org only indexed by Google?

Tue Jan 27 16:29:23 UTC 2015

On 01/26/2015 09:46 AM, Kevin Fenzi wrote:
> I think we added the Crawl-delay several years ago when we were having
> storage issues. We could definitely try removing it and see if things
> improve.

10 seconds may be on the high side, but you may still want to keep it and lower 
it to 1 to 5 seconds. Bing Bot hits my sites like it's running a benchmark.