[RFC] Packaging technology update

[Tommy Reynolds]

Hello, All!

As part of getting the RPM's built for the Fedora documents,  there
are a number of files necessarily that all must be consistent.  For
example, there is a revision history within the DocBook document
itself and a similar one for the RPM package.  There is an author /
editor / translator list for the DocBook information and a similar
list for the RPM.  The list goes on, but you get the idea.

Since I'm lazy and don't want to do the same task twice, I've
developed some XSLT stylesheets that will generate:

1) the <bookinfo> or <articleinfo> information in the DocBook itself;

2) the OMF file needed for the yelp integration;

3) the ".desktop" file needed for the GNOME menu system;

4) the ".spec" file needed for the RPM packaging.

All that is needed is for the authors, translators, and editors to
enter the information once in an "rpm-info.xml" file.  Then, the
document build system Makefile.common will be able to automatically
generate the derivative files as needed.

So that you can get an idea of how this will work, I've also added a
sample "rpm-info.xml" file to the Example Tutorial.  To see this
work, type the following command from the "example-tutorial/"
directory:

$ xsltproc --stringparam doctype articleinfo		\
    ../docs-common/packaging/bookinfo.xsl rpm-info.txt

When we get this worked out, we will redirect this generated XML into
a file and the base document will include it rather than having the
<articleinfo> stuff hardcoded within it.

As new translations are added to the document, the translator just
adds the new locale to the Makefile ${LANGUAGES} macro and updates
"rpm-info.xml" with the translated document name and description.
Everything else adapts itself to the new information and out comes a
localized RPM package.  This is just the plan, we're not quite there
yet, but we're very close.

The rest of this is probably of interest only to KWADE and STICKSTER, but feel
free to read along.

In the "docs-common/packaging" directory are some new files:

rpm-info.dtd	-- DTD for the "rpm-info.xml" files
bookinfo.xsl	-- XSLT stylesheet to extract the <articleinfo> stuff
desktop.xsl	-- XSLT stylesheet to build the "{docbase}.desktop" file
omf.xsl		-- XSLT stylesheet to generate the OMF file
spec.xsl	-- XSLT stylesheet to generate an RPM ".spec" file

Of these XSLT stylesheets, "spec.xsl" is the most tentative.  I'm not
clear about how to structure the multi-language RPM generation yet,
so I've put in some stubs to suggest how this technology could
generate a customized SPEC file on the fly.

Suggestions, praise and criticsm are all welcome!

Cheers

[Paul W. Frields]

On Fri, 2005-11-25 at 15:14 -0600, Tommy Reynolds wrote:
> As part of getting the RPM's built for the Fedora documents,  there
> are a number of files necessarily that all must be consistent.  For
> example, there is a revision history within the DocBook document
> itself and a similar one for the RPM package.  There is an author /
> editor / translator list for the DocBook information and a similar
> list for the RPM.  The list goes on, but you get the idea.
>
> Since I'm lazy and don't want to do the same task twice, I've
> developed some XSLT stylesheets that will generate:

[... a bunch of really cool stuff snipped ...]

Very cool!  Still looking over all of this, but I think one of the best
things about this (besides all the general coolness) is that it keeps
everything in the authoritative DocBook XML.  This is a big step in the
right direction.

[Paul W. Frields]

More thoughts on the packaging work Tommy did...

- Version numbers must be sane for revisionhistory entries.  In the
install-guide there was a 1.0rc1 followed by 1.0, which is decidedly
*not* sane.  (Ever the lemming, I then followed suit with 1.01rc1, which
I have since eliminated in favor of 1.0.1 and followed by 1.0.2.)  In
the future, no alphabetics should be used.  Runup to a X.0 should be
$(X-1).9.9 or something similar.  Remember that in RPM packaging terms,
IIRC, 0.99 > 0.12 > 0.9.9, so we should probably eliminate any weirdness
with double-digit numbering.  Using x.y.z where both y and z are less
than 10 is probably the right way to do things.  If you need something
even more minor, just make it x.y.z.n instead.

- I made just a couple cosmetic changes to spec.xsl to be consistent
with FE guidelines.

- I wonder if it would be wise to have a change to the DTD which offers
a 'release' in addition to 'version' for a 'revision', such that:

<revision date="Sat Nov 25 2005" version="0.1.3" release="1">

would be allowed.  The latest release number would be the thing that
appears in the %release tag.  The 'release' element that falls directly
inside 'rpm-info' would be eliminated.  There is always a chance that
things have to be repackaged because an OMF or .desktop file is updated
-- or even the spec template -- but not the doc content, which calls for
a release bump, not a version bump.  Is such a thing possible, Tommy?

Just some quick ideas...  I'm not really conversant with a lot of XSLT
stuff so I may piddle around with this, but not expecting great things
as a result. ;-)

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> I wonder if it would be wise to have a change to the DTD which offers
>
> a 'release' in addition to 'version' for a 'revision', such that:
>
>   <revision date="Sat Nov 25 2005" version="0.1.3" release="1">
>
> would be allowed.  The latest release number would be the thing that
> appears in the %release tag.  The 'release' element that falls directly
> inside 'rpm-info' would be eliminated.  There is always a chance that
> things have to be repackaged because an OMF or .desktop file is updated
> -- or even the spec template -- but not the doc content, which calls for
> a release bump, not a version bump.  Is such a thing possible, Tommy?

My thought was that the /rpm-info/release value would represent the
RPM packaging release cycle.  The <revision> components of the
<changelog> would generate both the RPM %changelog and the DocBook
revision log.

Hm.. would a "role='rpm'" attribute on the <changelog>/<revision>
element suffice?  I could then just skip that entry when building the
DocBook history.
> Just some quick ideas...  I'm not really conversant with a lot of XSLT
> stuff so I may piddle around with this, but not expecting great things
> as a result. ;-)

Neither am I, but if you can help get the prototype SPEC, OMF, et.
al., files right I think I can mangle the XSLT stuff enough to get
by.

Cheers

[Paul W. Frields]

On Sat, 2005-11-26 at 13:18 -0600, Tommy Reynolds wrote:
> Uttered "Paul W. Frields" , spake thus:
> > I wonder if it would be wise to have a change to the DTD which offers
> >
> > a 'release' in addition to 'version' for a 'revision', such that:
> >
> >   <revision date="Sat Nov 25 2005" version="0.1.3" release="1">
> >
> > would be allowed.  The latest release number would be the thing that
> > appears in the %release tag.  The 'release' element that falls directly
> > inside 'rpm-info' would be eliminated.  There is always a chance that
> > things have to be repackaged because an OMF or .desktop file is updated
> > -- or even the spec template -- but not the doc content, which calls for
> > a release bump, not a version bump.  Is such a thing possible, Tommy?
>
> My thought was that the /rpm-info/release value would represent the
> RPM packaging release cycle.

Ah, I see, that makes sense... IOW we would hope that if we do this
right that might never change, right?  Sorry, forget that snippet above,
then.
> The <revision> components of the
> <changelog> would generate both the RPM %changelog and the DocBook
> revision log.
> Hm.. would a "role='rpm'" attribute on the <changelog>/<revision>
> element suffice?  I could then just skip that entry when building the
> DocBook history.

Well, I think having that sort of structure makes sense, in that some
changes will be packaging only, and some changes will be document only.
It probably makes sense for the document revision history to be in the
RPM %changelog but not the other way around.  What if we were to use
some of the XSL "if" statements to perform some of that magic, and have
a hierarchy like this:

<changelog>
    <revision type="doc">
      <date>2005-11-20</date>
      <authorinitials>PWF</authorinitials>
      <details>Some guff here</details>
    </revision>
    <revision type="rpm">
      <date>Sat Nov 26 2005</date>
      <packager>
        <name>Paul W. Frields</name>
        <email>stickster at gmail</email>
      </packager>
    </revision>
  </changelog>
> > Just some quick ideas...  I'm not really conversant with a lot of XSLT
> > stuff so I may piddle around with this, but not expecting great things
> > as a result. ;-)
>
> Neither am I, but if you can help get the prototype SPEC, OMF, et.
> al., files right I think I can mangle the XSLT stuff enough to get
> by.

I am doing some DTD and XSLish reading... your new additions here have
helped make this a lot easier for me to learn a bit, so thanks for that.
I might play around with what's there, but will try not to break
anything.  ;-)

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> Well, I think having that sort of structure makes sense, in that some
> changes will be packaging only, and some changes will be document only.

You were right; I was wrong.  I fixed this your way.

There is now a "role='doc|rpm'" attribute to the changelog info.  I
ignore the "role='rpm'" entries when populating the DocBook
revhistory.

So there ;-)

[Paul W. Frields]

On Sat, 2005-11-26 at 16:03 -0500, Paul W. Frields wrote:
> > The <revision> components of the
> > <changelog> would generate both the RPM %changelog and the DocBook
> > revision log.
> > Hm.. would a "role='rpm'" attribute on the <changelog>/<revision>
> > element suffice?  I could then just skip that entry when building the
> > DocBook history.
>
> Well, I think having that sort of structure makes sense, in that some
> changes will be packaging only, and some changes will be document only.
> It probably makes sense for the document revision history to be in the
> RPM %changelog but not the other way around.  What if we were to use
> some of the XSL "if" statements to perform some of that magic, and have
> a hierarchy like this:
>
>   <changelog>
>     <revision type="doc">
>       <date>2005-11-20</date>
>       <authorinitials>PWF</authorinitials>
>       <details>Some guff here</details>
>     </revision>
>     <revision type="rpm">
>       <date>Sat Nov 26 2005</date>
>       <packager>
>         <name>Paul W. Frields</name>
>         <email>stickster at gmail</email>
>       </packager>
>     </revision>
>   </changelog>

Maybe the following might prove easier:

<changelog>
  <docrevision>
    <date>2005-11-20</date>
    <authorinitials>PWF</authorinitials>
    <details>Some guff here</details>
  </docrevision>
  <pkgrevision>
    <date>Sat Nov 26 2005</date>
    <packager>
    <name>Paul W. Frields</name>
    <email>stickster at gmail</email>
    <details>Fixed OMF</details>
    <details>Did something else important</details>
  </pkgrevision>
</changelog>

Amazing what a little DTD reading will teach ya... I'm startin' to
understand some of this here XML guff.  Unfortunately while I was
composing this you had already improved the DTD!

I'm wondering whether it makes sense to have these two elements be
different since they want for different data.  On the one hand, you
could just ask for a big author element with attributes for all of them,
as is the case now.  On the other hand, a RPM %changelog often has more
than one explanatory line for details.  I will try my hand at this one
if you agree, since I don't want to try your patience, and since I want
to see if I understand the concept well enough to DIY.  (DIM?)

I suppose, thanks to your display of XML skillz, we can even validate
this info during the build process to make sure nothing breaks!  Sweet.

[Paul W. Frields]

On Sat, 2005-11-26 at 17:04 -0500, Paul W. Frields wrote:
> Maybe the following might prove easier:
>
> <changelog>

[...snip...]

Maybe valid XML would be easier. :-D  Should read:

<changelog>
  <docrevision>
    <date>2005-11-20</date>
    <authorinitials>PWF</authorinitials>
    <details>Some guff here</details>
  </docrevision>
  <pkgrevision>
    <date>Sat Nov 26 2005</date>
    <packager>
      <name>Paul W. Frields</name>
      <email>stickster at gmail</email>
      <details>Fixed OMF</details>
      <details>Did something else important</details>
    </packager>
  </pkgrevision>
</changelog>

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> Maybe valid XML would be easier. :-D  Should read:
>
> <changelog>
>   <docrevision>
>     <date>2005-11-20</date>
>     <authorinitials>PWF</authorinitials>
>     <details>Some guff here</details>
>   </docrevision>
>   <pkgrevision>
>     <date>Sat Nov 26 2005</date>
>     <packager>
>       <name>Paul W. Frields</name>
>       <email>stickster at gmail</email>
>       <details>Fixed OMF</details>
>       <details>Did something else important</details>
>     </packager>
>   </pkgrevision>
> </changelog>

I don't really like this <docrevision> / <pkgrevision> stuff.  The
"role=doc" or "role=rpm" more succinctly captures the reason for a
particular change having been made.  I hope this is not a NIH issue,
but having frobbed around with the XSLT stylesheets a bit, I think I
like my approach better.

How about we keep the CVS versions of the DTD and XSLT stylesheets in sync?

Cheers

[Paul W. Frields]

On Sun, 2005-11-27 at 15:17 -0600, Tommy Reynolds wrote:
> Uttered "Paul W. Frields" , spake thus:
> > Maybe valid XML would be easier. :-D  Should read:
> >
> > <changelog>
> >   <docrevision>
> >     <date>2005-11-20</date>
> >     <authorinitials>PWF</authorinitials>
> >     <details>Some guff here</details>
> >   </docrevision>
> >   <pkgrevision>
> >     <date>Sat Nov 26 2005</date>
> >     <packager>
> >       <name>Paul W. Frields</name>
> >       <email>stickster at gmail</email>
> >       <details>Fixed OMF</details>
> >       <details>Did something else important</details>
> >     </packager>
> >   </pkgrevision>
> > </changelog>
>
> I don't really like this <docrevision> / <pkgrevision> stuff.  The
> "role=doc" or "role=rpm" more succinctly captures the reason for a
> particular change having been made.  I hope this is not a NIH issue,
> but having frobbed around with the XSLT stylesheets a bit, I think I
> like my approach better.

I'm not opposed to it, just wanted to float some ideas.  I'm OK with the
role attribute too... It would be nice, though, to *not* have to enter
all the extraneous name info in contexts where it won't be used.  For
example, doc revisions only need the person's initials to generate a
revision entry, where RPM revisions need the person's full name and
email.  (BTW, generating the full name from the firstname and surname
doesn't work for some people, like me for instance.)
> How about we keep the CVS versions of the DTD and XSLT stylesheets in sync?

Sure, as I noted in previous posts, I was just experimenting to see if I
understood DTDs well enough to make changes to this one and write a new
rpm-info.xml against it.  Experiment successful, now I can roll back to
the previous version, no problem, and move on from there.  I do want to
actually move the content out of attributes and into elements where it
seems more in keeping with standard XML usage, but I'll update the XSLT
as needed to do that.

What do you think about simply making the various pieces of <author>
optional so that they can be used as necessary?  Or is there a way to
make subelements dependent on the parent element's attribute?  I am
still reading up on that... In the meantime I've reverted the
aforementioned files.

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> It would be nice, though, to *not* have to enter all the extraneous
> name info in contexts where it won't be used.  For example, doc
> revisions only need the person's initials to generate a revision
> entry, where RPM revisions need the person's full name and email.
> (BTW, generating the full name from the firstname and surname
> doesn't work for some people, like me for instance.)

I know.  That's why I have all those names, name parts, initials,
combinations and permutations like "wholename" in there.  From that,
I generate the DocBook's <articleinfo> and <bookinfo> entries.

Think of the rpm-info file as sort of a database where we can
squirrel away (potentially) useful data and then produce wonderous
results without resorting to custom surgery.
> I do want to actually move the content out of attributes and into
> elements where it seems more in keeping with standard XML usage,
> but I'll update the XSLT as needed to do that.

Be very cautious here.  I used attributes because they can make
searching and selection easier.  There has long been a war between
the attribute folks and the sub-element folks, but then an emacs user
should be familiar with that sort of thing ;-)
> What do you think about simply making the various pieces of <author>
> optional so that they can be used as necessary?

If these items are optional, then we must know _now_ how we are going
to use this info _later_.  However, I *do* wish XML had some sort of
static structure declaration.

Until we get the building process up and working, we really have
nothing to streamline.  I'll listen to discussion like this later.

To me, the key point of this exercise has been that we can plop all
the packaging information into a single meta file and fairly easily
derive whatever we need from it; I hate changing the same info all
over creation.

Thanks for helping!

Cheers

[RFC] Packaging technology updateUttered "Paul W. Frields" <[e-mail]>, spake thus:

[Paul W. Frields]

On Sun, 2005-11-27 at 19:38 -0600, Tommy Reynolds wrote:
> Uttered "Paul W. Frields" , spake thus:
> > It would be nice, though, to *not* have to enter all the extraneous
> > name info in contexts where it won't be used.  For example, doc
> > revisions only need the person's initials to generate a revision
> > entry, where RPM revisions need the person's full name and email.
> > (BTW, generating the full name from the firstname and surname
> > doesn't work for some people, like me for instance.)
>
> I know.  That's why I have all those names, name parts, initials,
> combinations and permutations like "wholename" in there.  From that,
> I generate the DocBook's <articleinfo> and <bookinfo> entries.
>
> Think of the rpm-info file as sort of a database where we can
> squirrel away (potentially) useful data and then produce wonderous
> results without resorting to custom surgery.

Right, I understood this.
> > I do want to actually move the content out of attributes and into
> > elements where it seems more in keeping with standard XML usage,
> > but I'll update the XSLT as needed to do that.
>
> Be very cautious here.  I used attributes because they can make
> searching and selection easier.  There has long been a war between
> the attribute folks and the sub-element folks, but then an emacs user
> should be familiar with that sort of thing ;-)

I did google my way to some advice on this, the simplest explanation
being "if there's no required order to the content, use attributes."
That makes sense to me.

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> On the other hand, a RPM %changelog often has more than one
> explanatory line for details.  I will try my hand at this one if
> you agree, since I don't want to try your patience, and since I
> want to see if I understand the concept well enough to DIY.  (DIM?)

Go ahead and welcome.  I just didn't have time to investigate line
wrapping the %changelog entries, but I'd be happy for your to take a
swing.
> I suppose, thanks to your display of XML skillz, we can even validate
> this info during the build process to make sure nothing breaks!  Sweet.

BTW, order matters in the %changelog entries.  I always take the
version and release numbers from the very first <revision> element.
That's why there is that mandatory, only-one-alternative "ordered="
attribute; it's really documentation.

Cheers

[Paul W. Frields]

On Sun, 2005-11-27 at 09:40 -0600, Tommy Reynolds wrote:
> Uttered "Paul W. Frields" , spake thus:
> > On the other hand, a RPM %changelog often has more than one
> > explanatory line for details.  I will try my hand at this one if
> > you agree, since I don't want to try your patience, and since I
> > want to see if I understand the concept well enough to DIY.  (DIM?)
>
> Go ahead and welcome.  I just didn't have time to investigate line
> wrapping the %changelog entries, but I'd be happy for your to take a
> swing.
> > I suppose, thanks to your display of XML skillz, we can even validate
> > this info during the build process to make sure nothing breaks!  Sweet.
>
> BTW, order matters in the %changelog entries.  I always take the
> version and release numbers from the very first <revision> element.
> That's why there is that mandatory, only-one-alternative "ordered="
> attribute; it's really documentation.

Cool, that means I grokked it right.  Although that means revision
histories will be written differently than they are now, that's not a
problem.  DocBook doesn't mandate an order for them, so any way we want
to do them is acceptable.

[Tommy Reynolds]

Uttered "Paul W. Frields" , spake thus:
> Although that means revision histories will be written differently
> than they are now, that's not a problem.  DocBook doesn't mandate
> an order for them, so any way we want to do them is acceptable.

Well, we can output the changelog in any order we choose.  Instead of
an xsl:for-each construct, something along the lines of
"blah/blah/revision[count()-position()]" could reverse the order.

I figured that nicety could wait.

Cheers

[Paul W. Frields]

On Sun, 2005-11-27 at 09:56 -0600, Tommy Reynolds wrote:
> Uttered "Paul W. Frields" , spake thus:
> > Although that means revision histories will be written differently
> > than they are now, that's not a problem.  DocBook doesn't mandate
> > an order for them, so any way we want to do them is acceptable.
>
> Well, we can output the changelog in any order we choose.  Instead of
> an xsl:for-each construct, something along the lines of
> "blah/blah/revision[count()-position()]" could reverse the order.
>
> I figured that nicety could wait.

Yeah, I'm starting to realize the more reading I do that XSLT is pretty
danged powerful.  And yes, I am in fact late to the party, as usual. ;-D