docs.fedoraproject.org redesign -- very long!

Sat Apr 24 12:18:16 UTC 2010

Thank you Rudi for this explanation.
My comments are general enough, that I am not going to intersperse them
below.

I had not gotten to the stage of publishing yet and was not looking forward
to it.  I do not like the sounds of the existing option 0 and am likely to
delegate that task for the F13 User Guide to someone who has already done it
*grin*

I am sure that I can handle option 1 (git and html) but the publican
install_book option sounds even better.  Definitely keep working towards the
future with rpm builds.  Which ever option (1 or 2) is easier to implement
until the rpm build option is available worksforme.

Thanks!
-Susan

On Sat, Apr 24, 2010 at 7:11 AM, Ruediger Landmann <r.landmann at redhat.com>wrote:

> For at least the last year or so, we've been looking for ways to make
> docs.fedoraproject.org (d.fp.o) more usable for our readers and less
> onerous for our contributors. At the last docs meeting,[0] nb outlined a
> proposal to replace the current CVS mechanism, and I presented a brief demo
> of what publishing looks like with the forthcoming Publican 2.0.[1]
>
> To further the discussion, I thought I'd present the options a bit more
> completely here.
>
> Sorry this is so very wordy! I threw together some graphics to accompany
> this post that hopefully might make it a bit more digestible.
>
> Please have a read and share your thoughts.
>
> Cheers
> Rudi
>
>
> ================================
> OPTION 0 -- What we have now -- CVS + PHP
> ================================
>
> 1. Writers[2] pull down the source code of a doc from a repo (mostly git)
> and at least some part of d.fp.o site as stored in a CVS repo. Unless you're
> working on multiple docs, you only need the directory that corresponds to
> your  book, not the entire site.
>
> 2. Writers build the book from source on their local machines, and copy the
> Publican output into their local copy of the d.fp.o directory for that book.
> They also hand-edit PHP files as needed to provide the index pages.
>
> 3. Writers perform CVS voodoo to add and commit the files back up to the
> CVS repo, and tag the files LIVE. Within a few hours, the web server picks
> up any files tagged LIVE in the CVS repo and publishes them.
>
> PROBLEMS:
> 1. Editing the PHP index files by hand is onerous and seriously prone to
> error.  Everyone who has worked on this has routinely made mistakes keeping
> track of which book in which format in which language goes where.
>
> 2. The CVS process is notoriously temperamental, unforgiving, and prone to
> error. Mistakes are frequent, and correcting them is difficult and
> frustrating (and in some cases, not even possible).
>
>
> ================================
> OPTION 1 -- Proposal from Infra -- git + static HTML
> ================================
>
> 1. Writers pull down the source code of a doc from a repo (mostly git) and
> a git repo containing the entire d.fp.o website.
>
> 2. Writers build the book from source on their local machines, and copy the
> Publican output into the local copy of the git directory for that book. They
> also hand-edit html files as needed to provide the index pages.
>
> 3. Writers commit the files back up to the d.fp.o git repo. Within a few
> hours,  the web server synchronises with the repo and publishes the new
> files.
>
> ADVANTAGES:
> 1. Replacing CVS with git eliminates the single greatest source of failure
> and frustration.
>
> 2. The tools available to edit static html index files are more powerful
> and  more user-friendly than editing PHP in a text editor, so mistakes are
> less likely  to occur.
>
> DISADVANTAGES:
> 1. The initial git clone is massive -- something like 2 GB -- and this will
> only grow bigger over time.
>
> 2. Indexes are still built by hand.
>
>
> ================================
> OPTION 2 -- Publican
> ================================
> Note technology demonstration available at
> http://publictest8.fedoraproject.org/fedoradocs/public_html -- hurried,
> and slightly broken in places
>
> 1. Writers ssh into a remote system (let's call it the "build system" and
> pull  down the source code of a doc from a repo (mostly git) onto that
> system.
>
> 2. Writers build the book with Publican on the build system, and run the
>  "publican install_book" command. Publican adds the book to a local copy of
> d.fp.o and automatically rebuilds the site index.
>
> 3. At regular intervals, any new or changed files in the version of d.fp.o
> on the build server are copied to the web server, using scp or rsync or
>  something similar, initiated either on the build server end or the web
> server end.[3]
>
> ADVANTAGES:
> 1. Writers are finally completely free from the two most onerous and
>  error-prone parts of the publication process: using a version-control
> system as a publishing tool, and having to rebuild indexes by hand. Even if
> a book itself is misconfigured, screwups are contained to a single book in a
> single version in a single language, unlike some incidents we've had with
> hand-edited PHP.
>
> DISADVANTAGES:
> None of which I'm aware, however we'll need some new infrastructure in
> place  and we'll all no doubt make mistakes while getting used to a new way
> of  doing things.
>
> ================================
> OPTION FOR TOMORROW -- Publican + Koji
> ================================
>
> 1. Writers pull down the source code of a doc from a repo (mostly git)
>
> 2. Writers run "publican package" on their local systems with an option
> that  tells Publican to upload the SRPM package to a Koji instance. Koji
> builds the package and places it in a repo.
>
> 3. At regular intervals, the web server pulls any new docs packages from
> the  repo and installs them. Documentation packages built with Publican
>  automatically install into subdirectories of /var/www/html/ and regenerate
> any Publican site index present.
>
> ADVANTAGES:
> 1. Publishing is completely automated. This is actually the job that
> Publican was designed to do and pretty much how it's been running internally
> within Red Hat for  years. Writers and translators can focus on writing and
> translating and leave everything else to the machines.
>
> 2. Since the documentation is going to a repo, Fedora users world-wide
> could yum install docs packages from this repo. We might also be able to
> take advantage of the new langpacks feature in yum to help users find
> documentation in their own language.
>
> DISADVANTAGES:
> No disadvantages of which I'm aware, but the current stumbling blocks to
> implementing such a system right now are:
>
> 1. Publican doesn't yet support sending packages to arbitrary Koji
> instances! So this would require some code changes to Publican. Since
> Publican already supports Red Hat's Brew system, I suspect these are
> relatively minor.
>
> 2. We would need a Koji instance on which to build docs. Writers and
> translators who are trusted to publish docs would need to be approved as
> packagers on that Koji instance. In a perfect world, this would be the same
> Koji instance that the Fedora Project uses already to build software
> packages. However, most writers and translators do  not need to or want to
> package software. We would need some administrative changes within the
> project that would approve certain writers and translators to build docs
> packages only. We would also likely need a different  package approval
> process for docs, as docs packages would quickly overwhelm the existing
> system.[4] Failing to get these admin changes, we would need a separate Koji
> instance somewhere, and need Infra to maintain it for us.
>
>
> ================================
> Notes
> ================================
>
> [0] minutes --
> http://lists.fedoraproject.org/pipermail/docs/2010-April/012130.html
>
> [1] Publican 2.0 is still under heavy development, but I have (unsigned)
> packages for Publican 1.99 and its web component available from my
> fedorapeople page: http://rlandmann.fedorapeople.org/publican/ and you can
> find instructions for the new web stuff on Jeff Fearn's page:
> http://jfearn.fedorapeople.org -- please download and experiment! However,
> if you're involved in building or publishing F13 docs, please do not use
> Publican 1.99 for this, and in particular, do *not* update POT or PO files
> for current work with Publican 1.99!
>
> [2] I say "writers" here, but the same applies to translators, although to
> date, translators have not generally been involved in publishing their work.
> One advantage of rethinking how we publish is to allow translators and
> language teams direct control over their work, without making them rely on
> writers as "gatekeepers" for their docs.
>
> [3] Technically, there's no reason why the build server and web server need
> to be two separate machines, or why the "publican install_book" command
> couldn't write directly to a publicly accessible directory on the web
> server. I've suggested separating them mainly out of paranoia, but maybe I'm
> being overly cautious?
>
> [4] Hopefully in the context of this whole workflow, Publican's insistence
> on packaging languages one-at-a-time and on including version numbers in
> package names makes more sense -- it's these distinctions that produce the
> table of contents for the site. It also illustrates why a different package
> approval process would be necessary for docs -- 15 books in up to 12
> regularly-translated languages is up to 180 new packages for each Fedora
> release...
>
>
>
> --
> docs mailing list
> docs at lists.fedoraproject.org
> To unsubscribe:
> https://admin.fedoraproject.org/mailman/listinfo/docs
>

-- 
Susan Lauber, (RHCX, RHCA, RHCSS)
Lauber System Solutions, Inc.
http://www.laubersolutions.com
gpg: 15AC F794 A3D9 64D1 D9CE  4C26 EFC3 11C2 BFA1 0974
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/docs/attachments/20100424/1feb4e09/attachment.html 