Link checking for fedora-web

Paul W. Frields stickster at gmail.com
Wed Aug 19 19:24:30 UTC 2015


There's a useful utility called `linkchecker` that will verify links
in our site.  It's packaged in Fedora already (yay!) so this Makefile
goo should invoke it properly.  The resulting CSV file can be loaded
into LibreOffice (or whatever) and filtered to find 404 or other
errors.

I used this process recently to detect dead sponsor site links:
<https://git.fedorahosted.org/cgit/fedora-web.git/commit/?id=361b47a42a>

A few notes:

* This process downloads a few 10's of MiB, so if you're on a heavily
  metered connection you may want to leave this process to someone
  else.

* Very large downloads will generate a timeout error, which is normal.
  The configuration file has been set so that the rule won't download
  enormous images.

* The httplib3 library on which linkchecker relies gets cranky about
  verification of HTTPS links, so you'll see numerous warnings, which
  can be safely ignored.




More information about the websites mailing list