The State of Publishing Documentation in Fedora

Thu Feb 12 00:55:22 UTC 2015

On Wed, Feb 11, 2015 at 4:06 PM, Jeff Fearn <jfearn at redhat.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 02/12/2015 12:53 AM, Pete Travis wrote:
> > Hi all,
> >
> > I've given a lot of thought to our publishing situation.  The current
> solution, using an unmaintained version of Publican on an unsupported
> Fedora release, is untenable. The site we have been maintaining in web.git
> is
> > incompatible with current versions of publican, besides being massive
> and prone to breakage.  We have been working towards a replacement, slowly.
> >
> > The proposed successor to web.git is an RPM based system where Publican
> creates SRPMs of the guides, Fedora's koji buildsystem builds packages from
> the RPMs, and a VM instance installs the package to build the site.
> > Creating the SRPM would be analagous to the `publican build --embedtoc
> ...` from the old process, and when the VM installs the package, it does
> the `publican install-book` step of the process.  We currently have the
> > koji infrastructure in place, a mostly viable way for newly built
> packages to automatically be installed on the VM, and a functional but
> mostly vanilla site frontend.  Along the way, I've identified a few caveats
> to
> > this method, to which I'll add some general observations:
> >
> > - Each language of each release of each guide is a unique package.  To
> publish anything new, we must coordinate with releng to have the package
> defined in koji.  Releng is already overburdened. - Each language must
> > have a separately maintained revision history. - The buildroot shared by
> koji and the VM required updated versions of publican and publican's
> dependencies, and now must be maintained there (which is currently not
> > happening afaik). - None of the breakage problems we've dealt with in
> web.git will go away.  Unusual language codes, bad draft procedures, wonky
> *_Info.xml issues, and whatever black magic that ends with my manually
> > running sqlite invocations against the site db - that's all still there.
> - The site frontend needs to be completely redesigned from the ground up,
> and we have not demonstrated the motivation to see that through. There
> > are intricacies here that we didn't discover from a straightforward
> following of the publican user's guide, and many subjectively frustrating
> surprises. - A publican website can only publish publican-friendly content.
> > To the best of my knowledge, that means docbook only.  Our contributor
> base is declining, and prospective contributors have consistently
> demonstrated a lack of interest in writing docbook.
>
> Not quite, you can ship pre-built content easily enough [1], but it
> doesn't integrate the web UI in to it for obvious reasons.
>
> 1: https://bugzilla.redhat.com/show_bug.cgi?id=1081303
>
> > - A publican website does not provide an effective presentation of many
> smaller articles.  I've been disagreed with on this point, and will concede
> that it's definitely possible to have 200 things under that F21
> > category, but I maintain that presentation in that way would be inimical
> to reader browsing.
>
> > With all this in mind, I've decided to be honest with myself, and you,
> in saying that I'm not motivated to make the publican website frontend
> work.  Something must be done to deal with our site's incompatibility with
> > current Publican, but from a usage perspective, I feel like there's a
> very high upfront investment - and a moderate continuing investment in
> maintaining the el6-docs koji repo - with no substantial gain in the product
> > delivered.  For normal usage, we've simply replaced `publican
> build;publican install-book` with `publican package;koji build`.  Rudi had
> initially volunteered to do *all* of the setup to make this work, but he's a
> > busy guy and it really isn't fair to expect him to deal with that level
> of implementation for us.
> >
> > We've tossed around ideas in #fedora-docs, and in discussing it, I've
> developed a wishlist for an improved publishing platform:
> >
> > - We use release-based branches to maintain content for a given Fedora
> release.  Anything committed to these branches is intended for publication,
> so that should happen automatically. - It should be simpler to make
> > drafts available.  If we can automate publishing on release branches, we
> can do the same with master. - We don't do as good of a job as we should
> with pushing and pulling strings from translation.  Pushes and pulls
> > could be automated. - There should be an effective way to publish and
> organize sundry articles. - The publishing platform should be able to
> process arbitrary markup formats.  I think docbook is great, and publican
> > does a good job with it, but there will be room for more contributions
> if that isn't mandatory. - The frontend of the site should be adaptable.
> docs.fp.o would ideally use the same design elements as other official
> > Fedora sites for a more unified appearance. - Some validation on commits
> would be nice.
> >
> > The working theory at this point is to use a continuous integration
> system like Jenkins to automate builds.  I've had a Jenkins instance
> running locally this week, with new builds triggered via fedmsg signals
> > generated by our commits.  This part works smoothly, with the exception
> of a fedwatch bug that I've crudely worked around, but triggers using
> python-fedmsg really wouldn't be too difficult.  The infra folks are also
> > working on a Jenkins setup, so there's potential to share experience and
> buildslaves.  Turning that into a browseable frontend is more immediately
> viable than you might think;
> > http://sourceforge.net/projects/jenkinscat/ is designed just for that
> and can be customized for our needs.  Jenkins plugins to cycle strings from
> Zanata, or some other method, could come down the road.  There's room
> > for other enhancements, too.
> >
> > The CI idea is something I'm excited about, and motivated to work on.
> I'd love to spend that time working with others, if you're interested.
>
> Don't forget that Publican offers you an exit strategy from XML!
>
> It can can convert your DocBook in to Markdown** and then you *never* have
> to use XML *ever* again! \o/
>
> ** or in fact any of the formats at [2] if you install them!
>
> 2: http://search.cpan.org/search?m=dist&q=wikiconverter&s=1&n=50
>
> Which means you can also consider switching to a real CMS like medai wiki,
> moin moin, etc.
>
> You could use publican (or pandoc???) to convert XML to medai wiki format
> (by installing HTML::WikiConverter::MediaWiki [via cpanspec]) then import
> that in to https://fedoraproject.org/wiki/
>
> Unified web space FTW.
>
> Cheers, Jeff.
>
> - --
> Jeff Fearn
> Senior Software Engineer
> Hosted & Shared Services
> Red Hat Pty Ltd
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
>
> iQEcBAEBAgAGBQJU2+BoAAoJELs3R4zxGZvK9EMIAJniylZ0V+KV88wUNaSCsGfE
> D6ZKCNvkMilcWa+VevGwhfMLW6a6hNj6t3wktzw0JXncmpflZMS1BOL+tQasSf3R
> mlEPLhYCI/HpPWa3CAknmw7vaI/hUzDziPVrubovBFzzLQtXc7F4uRbgmAmjXPig
> l9nbYt4PIfG4m1/a/tK3B14JXgyBcustTslKUgxdVZ0JSumkQm619CO9QABuVXwB
> wP2MzBM5JxA4FLHTJa+m112frw43XYPIsyJZlA/UwqZ0QLv/GP/JMF4RDRcWFIpH
> FrTw48Zyn3je/eO3ZWaiYuqdqBsdW/pXFMw4o3jfPJ0sKvYD7XR2OjwUZh993LA=
> =WNBJ
> -----END PGP SIGNATURE-----
>

It's not about getting rid of publican, or docbook.  I'm very satisfied
with the way publican renders markup, and would not like to see that go
away.  The books we have would still be written in docbook and built by
publican, but jenkins would do the building instead of koji, because it can
happen automatically that way, and the users would use something else to
browse to the books.  The publican website front end, however, doesn't do
that most of extra stuff on my wishlist.

--Pete,
using a different client because clearly F22/rawhide thunderbird is broken
here
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/docs/attachments/20150211/aafa2884/attachment.html>