Docs CMS ideas

[Karsten Wade]

While I've been waiting to see how the Zikula deployment went with
Fedora Insight, we've finally seen a better update to
docs.fedoraproject.org by using the native Publican site building
tools.  I've waited to push forward with a CMS solution for Fedora
Docs until we had a working solution to design against.

While Publican gives us a way to build a nicely organized tree of
content, it doesn't give other capabilities.  I don't know of any
plans to include this sort of thing in Publican natively:

* Ability to trigger (re)builds from source repositories via a webUI.

* Workflow control, either by group (writers, editors, publishers), or
  specific roles like that but per document (I can write this one, I
  can edit yours, and I can push hers to publish after you edit it.)

* WYSIWYG editor for DocBook XML from source repository.

Basically, we still have very high barriers to move from "edit stuff
on the wiki" to "manage a full guide on DocBook XML."  The main
purpose for a CMS is to get the ability to publish and tweak
docs.fedoraproject.org out of the hands of a very few, very busy
people, and in to the hands of just about anyone that this team wants
to authorize to do work there.

Talking in the meeting yesterday, I threw out some ideas around this
and agreed to bring those here.  I'm still interested in helping drive
a CMS-like project for Docs.

So I'll throw out a proposal to get us started.  This proposal exists
partially because of our commitment to using Publican to publish.
Another option is to use Publican only to generate documents, which
are then loaded and versioned in a CMS e.g. Zikula or Drupal.  Working
under this other option would be more like the CMS experience others
have -- you either work in the native editor, or you upload documents
you created elsewhere.

== Docs CMS ==

=== Terms ===

  * Publican is a command-line toolchain for building books as HTML,
    PDF, Epub, Text, etc. and publishing them to a static website with
    a navigation sidebar.  It uses books written in DocBook XML that
    exist as installable RPM packages.

  * Beacon is a wysiwyg, webUI editor that has been extended to
    support DocBook XML editing.

  * Fedora Hosted (fh.o) is a hosting solution that is the source
    repository for Fedora Docs.  Books primarily use Subversion or git
    as source control manager (SCM).

  * WebUI is a generic term for the web user interface that we have to
    write or configure to wrap around and perform certain functions
    for this CMS.  For example, it can present a button to a person
    with publishing privileges, when pressed triggers of a Publican
    build on the backend.  This webUI could be custom code written in
    Python and TurboGears, or it could be Drupal with a few custom
    modules.

  * A reader has the ability to read the website anonymously.  No
    credentials are required.

  * A writer belongs to a 'doc-writers' group and also belong to the
    group associated with each document, e.g. 'gitinstall-guide'.
    Being in the 'doc-writers' group is a prerequisite that exposes
    the Beacon editor function.

  * An editor is the same as a writer, performing a different role in
    the process.

  * A publisher is a member of the 'doc-publishers' group, which gives
    the permission to trigger special functions in the webUI - build a
    doc, rebuild a doc, install a doc package, etc.

=== WebUI architecture ===

Functions:
  0. Read
  1. Write/edit
  2. Publish

0. Read
   a. Publican produces pages and navigation at docs.fp.org.
      i. These pages are static and not wrapped by the webUI.
   b. Readers use Publican's navigation to read guides.

1. Write/edit
   a. Writer clicks "Check out document for editing" (this is NOT a
      locking mechanism.)
      i. Clicking check out triggers a 'git clone' of the document
      	 repo.
      ii. The cloned git repo is then branched locally to the webUI
      	  server, for example 'git branch quaid-20100224-UUID' with a 
	  short, sequential UUID such as 'AA00'.
      iii. Beacon is then opened using the local file check out of the
      	   XML.
      iv. Beacon saves work only in the files in the cloned repo.  An
      	  added functionality could trigger 'git commit' on save, but
   	  this might cause too many commits without a log message.
      v. The writer's training includes knowing to commit on a regular
      	 basis, in a granular fashion, when one or more edit+saves
   	 makes a good bundle for a commit.  Clicking on "Commit
   	 changes" opens a log summary window, then a 'git commit -m
   	 "$dump_from_log_summary_window" is done on the local repo.
      vi. A writer's session should persist.  When the writer returns
      	  and still has local changes committed but not pushed to the
   	  master git repo, the writer is offered those files to
   	  continue work on, or to checkout a different document.
      vii. When the writer is done with the writing session, the
      	   writer clicks a "Check in document to main repository".
   	   This triggers a 'git push' to the master repository.

2. Publish
   a. Publisher selects a document and version from a
      dropdown/checklist menu.
      i. This list is generated from installed packages on docs.fp.o,
         information extracted using 'rpm' commands.
   b. Publisher clicks "Rebuild and publish document."
      i. A koji scratchbuild is triggered, which grabs the latest
         source from the upstream repository and builds a package.
      ii. When the build is done, the package is installed from koji
      	  using yum.
   c. Publisher adds a new document to the publish list.
      i. Document must exist as package in koji already.
      ii. Publisher uses a Moksha widget to identify the meta-package
      	  OR a specific package name and version are supplied
   	  manually.
      iii. Package is installed on docs.fedoraproject.org.
      iv. Cached list of installed packages is refreshed to update
      	  document and version dropdown/checklist menus.

<eof>

- Karsten
-- 
name:  Karsten 'quaid' Wade, Sr. Community Gardener
team:                Red Hat Community Architecture 
uri:               http://TheOpenSourceWay.org/wiki
gpg:                                       AD0E0C41

[Chris Curran]

On 06/25/2010 07:20 AM, Karsten Wade wrote:
>  While I've been waiting to see how the Zikula deployment went
> with
>  Fedora Insight, we've finally seen a better update to
>  docs.fedoraproject.org by using the native Publican site building
>  tools.  I've waited to push forward with a CMS solution for Fedora
>  Docs until we had a working solution to design against.
>
>  While Publican gives us a way to build a nicely organized tree of
>  content, it doesn't give other capabilities.  I don't know of any
>  plans to include this sort of thing in Publican natively:
>
>  * Ability to trigger (re)builds from source repositories via a webUI.
>
>  * Workflow control, either by group (writers, editors, publishers), or
>     specific roles like that but per document (I can write this one, I
>     can edit yours, and I can push hers to publish after you edit it.)
>
>  * WYSIWYG editor for DocBook XML from source repository.
>
>  Basically, we still have very high barriers to move from "edit stuff
>  on the wiki" to "manage a full guide on DocBook XML."  The main
>  purpose for a CMS is to get the ability to publish and tweak
>  docs.fedoraproject.org out of the hands of a very few, very busy
>  people, and in to the hands of just about anyone that this team wants
>  to authorize to do work there.
>
>  Talking in the meeting yesterday, I threw out some ideas around this
>  and agreed to bring those here.  I'm still interested in helping drive
>  a CMS-like project for Docs.
>
>  So I'll throw out a proposal to get us started.  This proposal exists
>  partially because of our commitment to using Publican to publish.
>  Another option is to use Publican only to generate documents, which
>  are then loaded and versioned in a CMS e.g. Zikula or Drupal.  Working
>  under this other option would be more like the CMS experience others
>  have -- you either work in the native editor, or you upload documents
>  you created elsewhere.
>
>  == Docs CMS ==
>
>  === Terms ===
>
>     * Publican is a command-line toolchain for building books as HTML,
>       PDF, Epub, Text, etc. and publishing them to a static website with
>       a navigation sidebar.  It uses books written in DocBook XML that
>       exist as installable RPM packages.
>
>     * Beacon is a wysiwyg, webUI editor that has been extended to
>       support DocBook XML editing.
>
>     * Fedora Hosted (fh.o) is a hosting solution that is the source
>       repository for Fedora Docs.  Books primarily use Subversion or git
>       as source control manager (SCM).
>
>     * WebUI is a generic term for the web user interface that we have to
>       write or configure to wrap around and perform certain functions
>       for this CMS.  For example, it can present a button to a person
>       with publishing privileges, when pressed triggers of a Publican
>       build on the backend.  This webUI could be custom code written in
>       Python and TurboGears, or it could be Drupal with a few custom
>       modules.
>
>     * A reader has the ability to read the website anonymously.  No
>       credentials are required.
>
>     * A writer belongs to a 'doc-writers' group and also belong to the
>       group associated with each document, e.g. 'gitinstall-guide'.
>       Being in the 'doc-writers' group is a prerequisite that exposes
>       the Beacon editor function.
>
>     * An editor is the same as a writer, performing a different role in
>       the process.
>
>     * A publisher is a member of the 'doc-publishers' group, which gives
>       the permission to trigger special functions in the webUI - build a
>       doc, rebuild a doc, install a doc package, etc.
>
>  === WebUI architecture ===
>
>  Functions:
>     0. Read
>     1. Write/edit
>     2. Publish
>
>  0. Read
>      a. Publican produces pages and navigation at docs.fp.org.
>         i. These pages are static and not wrapped by the webUI.
>      b. Readers use Publican's navigation to read guides.
>
>  1. Write/edit
>      a. Writer clicks "Check out document for editing" (this is NOT a
>         locking mechanism.)
>         i. Clicking check out triggers a 'git clone' of the document
>         	 repo.
>         ii. The cloned git repo is then branched locally to the webUI
>         	  server, for example 'git branch quaid-20100224-UUID' with a
>  	  short, sequential UUID such as 'AA00'.
>         iii. Beacon is then opened using the local file check out of the
>         	   XML.
>         iv. Beacon saves work only in the files in the cloned repo.  An
>         	  added functionality could trigger 'git commit' on save, but
>      	  this might cause too many commits without a log message.
>         v. The writer's training includes knowing to commit on a regular
>         	 basis, in a granular fashion, when one or more edit+saves
>      	 makes a good bundle for a commit.  Clicking on "Commit
>      	 changes" opens a log summary window, then a 'git commit -m
>      	 "$dump_from_log_summary_window" is done on the local repo.
>         vi. A writer's session should persist.  When the writer returns
>         	  and still has local changes committed but not pushed to the
>      	  master git repo, the writer is offered those files to
>      	  continue work on, or to checkout a different document.
>         vii. When the writer is done with the writing session, the
>         	   writer clicks a "Check in document to main repository".
>      	   This triggers a 'git push' to the master repository.
>
>  2. Publish
>      a. Publisher selects a document and version from a
>         dropdown/checklist menu.
>         i. This list is generated from installed packages on docs.fp.o,
>            information extracted using 'rpm' commands.
>      b. Publisher clicks "Rebuild and publish document."
>         i. A koji scratchbuild is triggered, which grabs the latest
>            source from the upstream repository and builds a package.
>         ii. When the build is done, the package is installed from koji
>         	  using yum.
>      c. Publisher adds a new document to the publish list.
>         i. Document must exist as package in koji already.
>         ii. Publisher uses a Moksha widget to identify the meta-package
>         	  OR a specific package name and version are supplied
>      	  manually.
>         iii. Package is installed on docs.fedoraproject.org.
>         iv. Cached list of installed packages is refreshed to update
>         	  document and version dropdown/checklist menus.
>
>  <eof>
>
>  - Karsten
>      
Another writer and I started coding up a project to do this. The main 
issue we hit was time, which is what killed it for us. This isn't a 
difficult project but it would take at least 6 months of full time 
development.

Chris

[Nathan Thomas]

On 24/06/10 22:20, Karsten Wade wrote:
>  While I've been waiting to see how the Zikula deployment went
> with
>  Fedora Insight, we've finally seen a better update to
>  docs.fedoraproject.org by using the native Publican site building
>  tools.  I've waited to push forward with a CMS solution for Fedora
>  Docs until we had a working solution to design against.
>
>  While Publican gives us a way to build a nicely organized tree of
>  content, it doesn't give other capabilities.  I don't know of any
>  plans to include this sort of thing in Publican natively:
>
>  * Ability to trigger (re)builds from source repositories via a webUI.
>
>  * Workflow control, either by group (writers, editors, publishers), or
>    specific roles like that but per document (I can write this one, I
>    can edit yours, and I can push hers to publish after you edit it.)
>
>  * WYSIWYG editor for DocBook XML from source repository.
>
>  Basically, we still have very high barriers to move from "edit stuff
>  on the wiki" to "manage a full guide on DocBook XML."  The main
>  purpose for a CMS is to get the ability to publish and tweak
>  docs.fedoraproject.org out of the hands of a very few, very busy
>  people, and in to the hands of just about anyone that this team wants
>  to authorize to do work there.
>
>  Talking in the meeting yesterday, I threw out some ideas around this
>  and agreed to bring those here.  I'm still interested in helping drive
>  a CMS-like project for Docs.
>
>  So I'll throw out a proposal to get us started.  This proposal exists
>  partially because of our commitment to using Publican to publish.
>  Another option is to use Publican only to generate documents, which
>  are then loaded and versioned in a CMS e.g. Zikula or Drupal.  Working
>  under this other option would be more like the CMS experience others
>  have -- you either work in the native editor, or you upload documents
>  you created elsewhere.
>
>  == Docs CMS ==
>
>  === Terms ===
>
>    * Publican is a command-line toolchain for building books as HTML,
>      PDF, Epub, Text, etc. and publishing them to a static website with
>      a navigation sidebar.  It uses books written in DocBook XML that
>      exist as installable RPM packages.
>
>    * Beacon is a wysiwyg, webUI editor that has been extended to
>      support DocBook XML editing.
>
>    * Fedora Hosted (fh.o) is a hosting solution that is the source
>      repository for Fedora Docs.  Books primarily use Subversion or git
>      as source control manager (SCM).
>
>    * WebUI is a generic term for the web user interface that we have to
>      write or configure to wrap around and perform certain functions
>      for this CMS.  For example, it can present a button to a person
>      with publishing privileges, when pressed triggers of a Publican
>      build on the backend.  This webUI could be custom code written in
>      Python and TurboGears, or it could be Drupal with a few custom
>      modules.
>
>    * A reader has the ability to read the website anonymously.  No
>      credentials are required.
>
>    * A writer belongs to a 'doc-writers' group and also belong to the
>      group associated with each document, e.g. 'gitinstall-guide'.
>      Being in the 'doc-writers' group is a prerequisite that exposes
>      the Beacon editor function.
>
>    * An editor is the same as a writer, performing a different role in
>      the process.
>
>    * A publisher is a member of the 'doc-publishers' group, which gives
>      the permission to trigger special functions in the webUI - build a
>      doc, rebuild a doc, install a doc package, etc.
>
>  === WebUI architecture ===
>
>  Functions:
>    0. Read
>    1. Write/edit
>    2. Publish
>
>  0. Read
>     a. Publican produces pages and navigation at docs.fp.org.
>        i. These pages are static and not wrapped by the webUI.
>     b. Readers use Publican's navigation to read guides.
>
>  1. Write/edit
>     a. Writer clicks "Check out document for editing" (this is NOT a
>        locking mechanism.)
>        i. Clicking check out triggers a 'git clone' of the document
>        	 repo.
>        ii. The cloned git repo is then branched locally to the webUI
>        	  server, for example 'git branch quaid-20100224-UUID' with a 
>  	  short, sequential UUID such as 'AA00'.
>        iii. Beacon is then opened using the local file check out of the
>        	   XML.
>        iv. Beacon saves work only in the files in the cloned repo.  An
>        	  added functionality could trigger 'git commit' on save, but
>     	  this might cause too many commits without a log message.
>        v. The writer's training includes knowing to commit on a regular
>        	 basis, in a granular fashion, when one or more edit+saves
>     	 makes a good bundle for a commit.  Clicking on "Commit
>     	 changes" opens a log summary window, then a 'git commit -m
>     	 "$dump_from_log_summary_window" is done on the local repo.
>        vi. A writer's session should persist.  When the writer returns
>        	  and still has local changes committed but not pushed to the
>     	  master git repo, the writer is offered those files to
>     	  continue work on, or to checkout a different document.
>        vii. When the writer is done with the writing session, the
>        	   writer clicks a "Check in document to main repository".
>     	   This triggers a 'git push' to the master repository.
>
>  2. Publish
>     a. Publisher selects a document and version from a
>        dropdown/checklist menu.
>        i. This list is generated from installed packages on docs.fp.o,
>           information extracted using 'rpm' commands.
>     b. Publisher clicks "Rebuild and publish document."
>        i. A koji scratchbuild is triggered, which grabs the latest
>           source from the upstream repository and builds a package.
>        ii. When the build is done, the package is installed from koji
>        	  using yum.
>     c. Publisher adds a new document to the publish list.
>        i. Document must exist as package in koji already.
>        ii. Publisher uses a Moksha widget to identify the meta-package
>        	  OR a specific package name and version are supplied
>     	  manually.
>        iii. Package is installed on docs.fedoraproject.org.
>        iv. Cached list of installed packages is refreshed to update
>        	  document and version dropdown/checklist menus.
>
>  <eof>
>
>  - Karsten
>     
I don't have the technical know-how to comment on the details of
Karsten's suggestion, however as a relatively new contributor of limited
technical abilities (arguably the kind of user we are aiming to attract
with this) I have one or two thoughts:

1. It seems from Karsten's description that there are two things going
on here: a) a system to make it easier for new contributors to write and
edit documents (cloning repos, editing the files, pushing the edits back
to the repos), and b) a system to make it easier to publish completed
documents to the web. A web CMS might be able to tick both of these
boxes - or it might be more appropriate (and less work) to use a
separate tool for each.
 
2. Any user wanting to get involved in the Docs Project is going to have
to learn new skills, new ways of thinking and working - and this is a
good thing. However, if the learning curve is too steep, or even appears
so to a potential new contributor, they might be put off. So I guess
anything we can do to take advantage of new contributors' existing
skills and experience could make the learning process easier. A lot of
new contributors might not have had previous experience in using
command-line tools to build marked-up documents from remote repositories
- but they might have used a simple web CMS to publish their blog
articles. A familiar working environment could go a long way to reducing
the trepidation of learning new skills.

Just my two pence!

Nathan