Link checking for fedora-web
Paul W. Frields
stickster at gmail.com
Wed Aug 19 19:24:30 UTC 2015
There's a useful utility called `linkchecker` that will verify links
in our site. It's packaged in Fedora already (yay!) so this Makefile
goo should invoke it properly. The resulting CSV file can be loaded
into LibreOffice (or whatever) and filtered to find 404 or other
errors.
I used this process recently to detect dead sponsor site links:
<https://git.fedorahosted.org/cgit/fedora-web.git/commit/?id=361b47a42a>
A few notes:
* This process downloads a few 10's of MiB, so if you're on a heavily
metered connection you may want to leave this process to someone
else.
* Very large downloads will generate a timeout error, which is normal.
The configuration file has been set so that the rule won't download
enormous images.
* The httplib3 library on which linkchecker relies gets cranky about
verification of HTTPS links, so you'll see numerous warnings, which
can be safely ignored.
More information about the websites
mailing list