The State of Publishing Documentation in Fedora

[Stephen Wadeley]

On 02/11/2015 03:53 PM, Pete Travis wrote:
> Hi all,
>
> I've given a lot of thought to our publishing situation.  The current
> solution, using an unmaintained version of Publican on an unsupported
> Fedora release, is untenable. The site we have been maintaining in
> web.git is incompatible with current versions of publican, besides being
> massive and prone to breakage.  We have been working towards a
> replacement, slowly.
>
> The proposed successor to web.git is an RPM based system where Publican
> creates SRPMs of the guides, Fedora's koji buildsystem builds packages
> from the RPMs, and a VM instance installs the package to build the
> site.  Creating the SRPM would be analagous to the `publican build
> --embedtoc ...` from the old process, and when the VM installs the
> package, it does the `publican install-book` step of the process.  We
> currently have the koji infrastructure in place, a mostly viable way for
> newly built packages to automatically be installed on the VM, and a
> functional but mostly vanilla site frontend.  Along the way, I've
> identified a few caveats to this method, to which I'll add some general
> observations:
>
>    - Each language of each release of each guide is a unique package.  To
> publish anything new, we must coordinate with releng to have the package
> defined in koji.  Releng is already overburdened.
>    - Each language must have a separately maintained revision history.
>    - The buildroot shared by koji and the VM required updated versions of
> publican and publican's dependencies, and now must be maintained there
> (which is currently not happening afaik).
>    - None of the breakage problems we've dealt with in web.git will go
> away.  Unusual language codes, bad draft procedures, wonky *_Info.xml
> issues, and whatever black magic that ends with my manually running
> sqlite invocations against the site db - that's all still there.
>    - The site frontend needs to be completely redesigned from the ground
> up, and we have not demonstrated the motivation to see that through.
> There are intricacies here that we didn't discover from a
> straightforward following of the publican user's guide, and many
> subjectively frustrating surprises.
>    - A publican website can only publish publican-friendly content.  To
> the best of my knowledge, that means docbook only.  Our contributor base
> is declining, and prospective contributors have consistently
> demonstrated a lack of interest in writing docbook.
>    - A publican website does not provide an effective presentation of
> many smaller articles.  I've been disagreed with on this point, and will
> concede that it's definitely possible to have 200 things under that F21
> category, but I maintain that presentation in that way would be inimical
> to reader browsing.
>
> With all this in mind, I've decided to be honest with myself, and you,
> in saying that I'm not motivated to make the publican website frontend
> work.  Something must be done to deal with our site's incompatibility
> with current Publican, but from a usage perspective, I feel like there's
> a very high upfront investment - and a moderate continuing investment in
> maintaining the el6-docs koji repo - with no substantial gain in the
> product delivered.  For normal usage, we've simply replaced `publican
> build;publican install-book` with `publican package;koji build`.  Rudi
> had initially volunteered to do *all* of the setup to make this work,
> but he's a busy guy and it really isn't fair to expect him to deal with
> that level of implementation for us.
>
> We've tossed around ideas in #fedora-docs, and in discussing it, I've
> developed a wishlist for an improved publishing platform:
>
>    - We use release-based branches to maintain content for a given Fedora
> release.  Anything committed to these branches is intended for
> publication, so that should happen automatically.
>    - It should be simpler to make drafts available.  If we can automate
> publishing on release branches, we can do the same with master.
>    - We don't do as good of a job as we should with pushing and pulling
> strings from translation.  Pushes and pulls could be automated.
>    - There should be an effective way to publish and organize sundry
> articles.
>    - The publishing platform should be able to process arbitrary markup
> formats.  I think docbook is great, and publican does a good job with
> it, but there will be room for more contributions if that isn't mandatory.
>    - The frontend of the site should be adaptable.  docs.fp.o would
> ideally use the same design elements as other official Fedora sites for
> a more unified appearance.
>    - Some validation on commits would be nice.
>
> The working theory at this point is to use a continuous integration
> system like Jenkins to automate builds.  I've had a Jenkins instance
> running locally this week, with new builds triggered via fedmsg signals
> generated by our commits.  This part works smoothly, with the exception
> of a fedwatch bug that I've crudely worked around, but triggers using
> python-fedmsg really wouldn't be too difficult.  The infra folks are
> also working on a Jenkins setup, so there's potential to share
> experience and buildslaves.  Turning that into a browseable frontend is
> more immediately viable than you might think;
> http://sourceforge.net/projects/jenkinscat/ is designed just for that
> and can be customized for our needs.  Jenkins plugins to cycle strings
> from Zanata, or some other method, could come down the road.  There's
> room for other enhancements, too.
>
> The CI idea is something I'm excited about, and motivated to work on.
> I'd love to spend that time working with others, if you're interested.
>
> --Pete
>

Hello Pete

I had an idea to test CI in one VM sending docs to another VM running a 
webserver, all in a local data centre. Not had the time yet. I would be 
interested to work with you if no one else picks up the gauntlet.

Regards
-- 
Stephen Wadeley
Content Author | Red Hat, Inc.