Documentation Demo with Sphinx

[Tim Flink]

I spent a little bit of time and put together a really small demo of
what our documentation could look like using Sphinx
(http://sphinx.pocoo.org/). I stress that this is a _demo_ and I didn't
spend much time on it - the content is pretty sparse and not very good.
If we're interested in looking into this more, I can spend more time on it.

For the demo, I converted the AutoQA wiki page verbatim (other than the
added section for module docs) and generated doc for bodhi_utils. I
listed some of what I think the advantages and disadvantages of
switching to Sphinx would be.

Code:
 - https://github.com/tflink/autoqa-devel
   * To generate the HTML docs, go to the doc/ dir and type 'make html'

Demo:
 - http://tflink.fedorapeople.org/autoqa/sphinxdemo/


========================================
Advantages:
========================================
 - Keep documentation in mostly one place
   * Obviously some stuff should stay in trac (planning) and other
     stuff in MW (tie-ins with Fedora)

 - Allow for documentation changes between releases without messy
   namespace issues.
   * Instead of worrying when wiki pages will go live with not-yet-true
     information, keep the info in master and it only shows up as the
     current official documentation once we release
   * This hasn't been much of an issue for us yet, but it will be once
     we do our first major overhaul of AutoQA and break some backwards
     compatibility

 - Integration of generated API docs and written docs

 - Easier generation of an autoqa-docs package once we start packaging

 - More encouragement to clean up docstrings
   * I know that I've not been as good about good docstrings as I could
     be - having them more visible in API docs might help improve their
     quality :)

========================================
Disadvantages:
========================================
 - It's different
   * reStructuredText is nice but it's not mediawiki or moin
   * links and global substitutions are a little more difficult
   * requires compilation instead of hitting the preview button
   * Harder for people who aren't already AutoQA devs to contribute docs

 - Requires conversion
   * The conversion process for syntax is pretty straight-forward but
     it would take time.

 - Lends to a different documentation style
   * I think that Sphinx works best with more of a book-like style than
     we've been doing thus far (more atomic pages). Converting what we
     have to start would be fine but we might want to look into
     documentation refactoring.
   * pytest is a good example of what this could look like
     - http://pytest.org/latest/


Thoughts?

Tim

[Kamil Paral]

>  I spent a little bit of time and put together a really small demo of
>  what our documentation could look like using Sphinx
>  (http://sphinx.pocoo.org/). I stress that this is a _demo_ and I
>  didn't
>  spend much time on it - the content is pretty sparse and not very
>  good.
>  If we're interested in looking into this more, I can spend more time
>  on it.
>  
>  For the demo, I converted the AutoQA wiki page verbatim (other than
>  the
>  added section for module docs) and generated doc for bodhi_utils. I
>  listed some of what I think the advantages and disadvantages of
>  switching to Sphinx would be.
>  
>  Code:
>  - https://github.com/tflink/autoqa-devel
>  * To generate the HTML docs, go to the doc/ dir and type 'make html'
>  
>  Demo:
>  - http://tflink.fedorapeople.org/autoqa/sphinxdemo/ 

It really looks good. I am not sure of the immediate benefits though (when the extra work
is taken into account).

* I think these auto-generated pages are good for module documentation.
They really are. The question is whether we need it currently. The developers who checkout
out our code will probably always look at the current doc with pydoc or some IDE or
display source code. I don't see anything we currently offer to the wide public. Once
we have support for user-provided tests, I agree this is a great tool to have the docs
online.

* I think it's little too complicated for free-text documentation.
I imagine a situation when we publish documentation like "How to write AutoQA
tests" or "Depcheck help" with the new release and after a week or two we
want to change something. Add more examples, fix typos, etc. Will we have to always push
to git, tag it, deploy it? I don't think it worth it, at least currently. I understand
it's good to have "How to write AutoQA tests" for 0.5 and for 0.6 separated
and publish it only after new release. But again, I think it makes sense once we support
public API for user-provided tests. Until that time it's more of a documentation for
us than for anyone else.


Maybe we should separate documentation "for us, the developers" and for
"the others, test writers" and then talk about them separately (location,
generation, etc).

Josef has some more thoughts on this, I suppose he'll write some of them here soon.

[Josef Skladanka]

+1 on Kamil's thoughts.

The auto-generated documentation is great for libraries (i.e. an equivalent of pydoc), but
I would not support having "user" documentation done this way.

I believe that the first step should be 'consolidation' of all the documentation
we have, and sorting it into three categories
 * code documentation
 * development documentation
 * user documentation


Code documentation are docstrings + possibly stuff like "ResultsDB Scheme" and
documentation tightly related to 'understanding the code'.

Development documentation would be AutoQA architecture, ResulsDB Writeup, Depcheck
descriptions...

While by user documentation, I mean stuff like "how to install autotest"
"How to update AutoQA repoinfo.conf", but also "understanding
depcheck/upgradepath errors" and other things which are not really related to the
code itself.

I don't have any problem with code and devel doc being autogenerated (and hence
tightly coupled with autoqa releases & git), but I'd like to have the users doc on
wiki.

The only thing I'd gladly got rid of is the spread of documentation between trac &
wiki. I'd keep one, and personaly prefer wiki over trac.

J.
----- Original Message -----
>  From: "Kamil Paral" <kparal(a)redhat.com&gt;
>  To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org&gt;
>  Sent: Wednesday, July 13, 2011 10:16:10 AM
>  Subject: Re: Documentation Demo with Sphinx
>  > I spent a little bit of time and put together a really small demo of
>  > what our documentation could look like using Sphinx
>  > (http://sphinx.pocoo.org/). I stress that this is a _demo_ and I
>  > didn't
>  > spend much time on it - the content is pretty sparse and not very
>  > good.
>  > If we're interested in looking into this more, I can spend more time
>  > on it.
>  >
>  > For the demo, I converted the AutoQA wiki page verbatim (other than
>  > the
>  > added section for module docs) and generated doc for bodhi_utils. I
>  > listed some of what I think the advantages and disadvantages of
>  > switching to Sphinx would be.
>  >
>  > Code:
>  > - https://github.com/tflink/autoqa-devel
>  > * To generate the HTML docs, go to the doc/ dir and type 'make html'
>  >
>  > Demo:
>  > - http://tflink.fedorapeople.org/autoqa/sphinxdemo/
>  
>  
>  It really looks good. I am not sure of the immediate benefits though
>  (when the extra work is taken into account).
>  
>  * I think these auto-generated pages are good for module
>  documentation.
>  They really are. The question is whether we need it currently. The
>  developers who checkout out our code will probably always look at the
>  current doc with pydoc or some IDE or display source code. I don't see
>  anything we currently offer to the wide public. Once we have support
>  for user-provided tests, I agree this is a great tool to have the docs
>  online.
>  
>  * I think it's little too complicated for free-text documentation.
>  I imagine a situation when we publish documentation like "How to write
>  AutoQA tests" or "Depcheck help" with the new release and after a week
>  or two we want to change something. Add more examples, fix typos, etc.
>  Will we have to always push to git, tag it, deploy it? I don't think
>  it worth it, at least currently. I understand it's good to have "How
>  to write AutoQA tests" for 0.5 and for 0.6 separated and publish it
>  only after new release. But again, I think it makes sense once we
>  support public API for user-provided tests. Until that time it's more
>  of a documentation for us than for anyone else.
>  
>  
>  Maybe we should separate documentation "for us, the developers" and
>  for "the others, test writers" and then talk about them separately
>  (location, generation, etc).
>  
>  Josef has some more thoughts on this, I suppose he'll write some of
>  them here soon.
>  
>  _______________________________________________
>  autoqa-devel mailing list
>  autoqa-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/autoqa-devel

[John Dulaney]

>  +1 on Kamil's thoughts.
>
>  The auto-generated documentation is great for libraries (i.e. an equivalent of pydoc),
> but I would not support having "user" documentation done this way.
>
>  I believe that the first step should be 'consolidation' of all the documentation
> we have, and sorting it into three categories
>  * code documentation
>  * development documentation
>  * user documentation
>
>
>  Code documentation are docstrings + possibly stuff like "ResultsDB Scheme" and
> documentation tightly related to 'understanding the code'.
>
>  Development documentation would be AutoQA architecture, ResulsDB Writeup, Depcheck
> descriptions...
>
>  While by user documentation, I mean stuff like "how to install autotest"
> "How to update AutoQA repoinfo.conf", but also "understanding
> depcheck/upgradepath errors" and other things which are not really related to the
> code itself.
>
>  I don't have any problem with code and devel doc being autogenerated (and hence
> tightly coupled with autoqa releases & git), but I'd like to have the users doc on
> wiki.
>
>  The only thing I'd gladly got rid of is the spread of documentation between trac
> & wiki. I'd keep one, and personaly prefer wiki over trac.
>
>  J. 
Ok, kind of expanding on this, would it be possible to set up Sphinx so that
it does the autodoc thing with code and devel documentation, but also pull
from the Wiki for user doc?  The Wiki content would still have to be human
authored, but I don't see why Sphinx can't pull from that.
Am I making sense here?

Also, +1 on keeping docs in the Wiki rather than spreading them about, no
matter whether Sphinx winds up getting used or not.

John.

[Josef Skladanka]

I understand what you mean, but I'd rather not invest a huge amount of time into
hacking (eeeer... adjusting ;-)) Sphinx so it pulls data from MediaWiki. Do you think that
just a simple link to "Search for user documentation here <link>" would
not be sufficient?

J.

----- Original Message -----
>  From: "John Dulaney" <j_dulaney(a)live.com&gt;
>  To: autoqa-devel(a)lists.fedorahosted.org
>  Sent: Wednesday, July 13, 2011 3:31:50 PM
>  Subject: RE: Documentation Demo with Sphinx
>  > +1 on Kamil's thoughts.
>  >
>  > The auto-generated documentation is great for libraries (i.e. an
>  > equivalent of pydoc), but I would not support having "user"
>  > documentation done this way.
>  >
>  > I believe that the first step should be 'consolidation' of all the
>  > documentation we have, and sorting it into three categories
>  > * code documentation
>  > * development documentation
>  > * user documentation
>  >
>  >
>  > Code documentation are docstrings + possibly stuff like "ResultsDB
>  > Scheme" and documentation tightly related to 'understanding the
>  > code'.
>  >
>  > Development documentation would be AutoQA architecture, ResulsDB
>  > Writeup, Depcheck descriptions...
>  >
>  > While by user documentation, I mean stuff like "how to install
>  > autotest" "How to update AutoQA repoinfo.conf", but also
>  > "understanding depcheck/upgradepath errors" and other things which
>  > are not really related to the code itself.
>  >
>  > I don't have any problem with code and devel doc being autogenerated
>  > (and hence tightly coupled with autoqa releases & git), but I'd like
>  > to have the users doc on wiki.
>  >
>  > The only thing I'd gladly got rid of is the spread of documentation
>  > between trac & wiki. I'd keep one, and personaly prefer wiki over
>  > trac.
>  >
>  > J.
>  
>  Ok, kind of expanding on this, would it be possible to set up Sphinx
>  so that
>  it does the autodoc thing with code and devel documentation, but also
>  pull
>  from the Wiki for user doc? The Wiki content would still have to be
>  human
>  authored, but I don't see why Sphinx can't pull from that.
>  Am I making sense here?
>  
>  Also, +1 on keeping docs in the Wiki rather than spreading them about,
>  no
>  matter whether Sphinx winds up getting used or not.
>  
>  John.
>  
>  _______________________________________________
>  autoqa-devel mailing list
>  autoqa-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Tim Flink]

On 07/13/2011 07:31 AM, John Dulaney wrote:
>  
>  Ok, kind of expanding on this, would it be possible to set up Sphinx so that
>  it does the autodoc thing with code and devel documentation, but also pull
>  from the Wiki for user doc?  The Wiki content would still have to be human
>  authored, but I don't see why Sphinx can't pull from that.
>  Am I making sense here? 
That would certainly help make any packaged documentation more complete
but it would be quite a bit of work and I think that the current
perception is that we don't have enough other users to justify that.
I've seen tools for going from rST (what Sphinx uses) to mediawiki but
not the other way around.

I'm also not a fan of requiring network access in order to build the
docs. If something is going to be built locally, the source should be
stored locally or at least cached locally to allow for building without
network access after the first time.

Tim

[Tim Flink]

On 07/13/2011 02:16 AM, Kamil Paral wrote:
>  It really looks good. I am not sure of the immediate benefits though
>  (when the extra work is taken into account).
>  
>  * I think these auto-generated pages are good for module
>  documentation. They really are. The question is whether we need it
>  currently.  
No, we probably don't need the API docs right now. I don't think that
there are many other people trying to develop against AutoQA right now.

>  Will we have to always push to git, tag it, deploy it? 
No, I was thinking of making doc generation and pushing to hosting part
of a CI process that gets run on every push to the fedorahosted git
repo. Each branch's documentation would be put in a different directory
on a public facing web server.

CI is on my list of things to do once we get the hardware for a staging
environment.