ftp 421 errors when retrieving development files from download.fedora.redhat.com

[Michael Schwendt]

On Thu, 26 Apr 2007 08:56:36 +1000, David Timms wrote:

> Hi,
> 
> After manually ftping stuff {13 files: rescue iso, pxe kernel, repodata, 
> stage2.img} from development last few weeks, I created a shell script 
> that did the same. I used urlgrabber to get the files I'm interested in 
> - see attach.
> 
> What I noticed was that some {4} downloads failed with python traceback eg:
> Traceback (most recent call last):
>    File "/usr/bin/urlgrabber", line 124, in ?
>      main()
>    File "/usr/bin/urlgrabber", line 120, in main
>      filename = urlgrab(url,filename=file,**kwargs)
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 597, in urlgrab
>      return default_grabber.urlgrab(url, filename, **kwargs)
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 927, in urlgrab
>      return self._retry(opts, retryfunc, url, filename)
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 845, in _retry
>      r = apply(func, (opts,) + args, {})
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 913, in retryfunc
>      fo = URLGrabberFileObject(url, filename, opts)
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 1001, in __init__
>      self._do_open()
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 1068, in _do_open
>      fo, hdr = self._make_request(req, opener)
>    File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 
> 1184, in _make_request
>      raise URLGrabError(4, _('IOError: %s') % (e, ))
> urlgrabber.grabber.URLGrabError: [Errno 4] IOError: [Errno ftp error] 
> 421 There are too many connections from your internet address.
> 
> Now, in terms of this, the machine making the connection {and there 
> isn't any other machine powered on within my NATed IP address}, was 
> running just the script using urlgrabber, and it is used sequentially 
> {one download finishes before the next one starts}, rather than in parallel.
> 
> My guess is that the download site is taking a huge hit, which is fair 
> enough, but if that is the case, the message should be "too many 
> connected users" rather than ~" ... from your internet address".
>
> Am I just doing something silly ? {other than not using an in country 
> mirror - which is often a few days late in it's content}

There is a difference between the number of users connected to the FTP
service and the number and frequency of connection attempts.

With FTP as the protocol, I would stay logged in and fetch all needed
files before leaving the single session finally. FTP clients such as lftp
are scriptable and can do that.