L10n and Docs fallback

[Paul W. Frields]

We have several problems currently brewing with using hosted git repos
of Publican-based documents with L10n's Transifex (Tx).

1.  Publican uses multiple POT files instead of the single POT model.
    Transifex apparently does not currently handle multiple POT files
    well.

2.  The current (obsolete but soon to be replaced) Damned Lies (DL)
    used in L10n doesn't respond well to multiple POT files either.

3.  There may also be some issues in Tx about support for the ISO
    language codes used in Publican.

I've asked Dimitris Glezos to give us an idea how much work is
involved in fixing each of these factors.

Problem #1 is a high risk, and if we can't solve it quickly we have to
fall back to our original methodologies.

Problem #2 is lower risk, because DL is being obsoleted anyway, and Tx
will soon take over its duties.  Plus there is the possibility of a
low-drag workaround, which I think some folks are working on now.

Problem #3 is medium-to-high risk -- again, translators need to be
able to get at the content.

I expect Dimitris, Asgeir, and other L10n leaders will respond to this
thread with some suggestions for (1) some estimate of how big these
problems really are, (2) whether they can have the higher-risk ones
repaired in the next, say, 1-2 weeks.

-- 
Paul W. Frields                                http://paul.frields.org/
  gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233  5906 ACDB C937 BD11 3717
  http://redhat.com/   -  -  -  -   http://pfrields.fedorapeople.org/
  irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug

[Karsten Wade]

On Wed, 2008-10-01 at 15:38 -0400, Paul W. Frields wrote:
>  We have several problems currently brewing with using hosted git
> repos
>  of Publican-based documents with L10n's Transifex (Tx).
>  
>  1.  Publican uses multiple POT files instead of the single POT model.
>      Transifex apparently does not currently handle multiple POT files
>      well.
>  
>  2.  The current (obsolete but soon to be replaced) Damned Lies (DL)
>      used in L10n doesn't respond well to multiple POT files either.
>  
>  3.  There may also be some issues in Tx about support for the ISO
>      language codes used in Publican.
>  
>  I've asked Dimitris Glezos to give us an idea how much work is
>  involved in fixing each of these factors.
>  
>  Problem #1 is a high risk, and if we can't solve it quickly we have to
>  fall back to our original methodologies.
>  
>  Problem #2 is lower risk, because DL is being obsoleted anyway, and Tx
>  will soon take over its duties.  Plus there is the possibility of a
>  low-drag workaround, which I think some folks are working on now. 
For example:

http://asgeirf.fedorapeople.org/docstats/release-notes.master/

>  Problem #3 is medium-to-high risk -- again, translators need to be
>  able to get at the content.
>  
>  I expect Dimitris, Asgeir, and other L10n leaders will respond to this
>  thread with some suggestions for (1) some estimate of how big these
>  problems really are, (2) whether they can have the higher-risk ones
>  repaired in the next, say, 1-2 weeks. 
In case this isn't clear, we need someone to step-up and solve the
problems in Publican and Transifex.  I support Asgeir's reasoning for
working around DL's statistics.

We've been committed to using a good upstream product, now that Publican
fills that role, but there are still struggles in the upstream's
interaction with the wider, established communities in terms of methods,
tooling choices, and tool architecture.  All of these can be solved, and
Fedora Docs commitment is to be a good citizen in trying to get features
worked out and architectural decisions revisited and fixed at the
Publican level.

In the meantime, the higher priority parts of our mission can be
fulfilled with any DocBook XML toolchain, and we'll use whichever one is
working best within that 2 weeks.

- Karsten
-- 
Karsten Wade, Community Gardener
Dev Fu : http://developer.redhatmagazine.com
Fedora : http://quaid.fedorapeople.org
gpg key : AD0E0C41

[Jeff Fearn]

Karsten 'quaid' Wade wrote:
>  On Wed, 2008-10-01 at 15:38 -0400, Paul W. Frields wrote:
> > We have several problems currently brewing with using hosted git repos
> > of Publican-based documents with L10n's Transifex (Tx).
> >
> > 1.  Publican uses multiple POT files instead of the single POT model.
> >     Transifex apparently does not currently handle multiple POT files
> >     well.
> >
> > 2.  The current (obsolete but soon to be replaced) Damned Lies (DL)
> >     used in L10n doesn't respond well to multiple POT files either.
> >
> > 3.  There may also be some issues in Tx about support for the ISO
> >     language codes used in Publican.
> >
> > I've asked Dimitris Glezos to give us an idea how much work is
> > involved in fixing each of these factors.
> >
> > Problem #1 is a high risk, and if we can't solve it quickly we have to
> > fall back to our original methodologies.
> >
> > Problem #2 is lower risk, because DL is being obsoleted anyway, and Tx
> > will soon take over its duties.  Plus there is the possibility of a
> > low-drag workaround, which I think some folks are working on now.
>  
>  For example:
>  
>  http://asgeirf.fedorapeople.org/docstats/release-notes.master/
>  
> > Problem #3 is medium-to-high risk -- again, translators need to be
> > able to get at the content.
> >
> > I expect Dimitris, Asgeir, and other L10n leaders will respond to this
> > thread with some suggestions for (1) some estimate of how big these
> > problems really are, (2) whether they can have the higher-risk ones
> > repaired in the next, say, 1-2 weeks.
>  
>  In case this isn't clear, we need someone to step-up and solve the
>  problems in Publican and Transifex.  I support Asgeir's reasoning for
>  working around DL's statistics.
>  
>  We've been committed to using a good upstream product, now that Publican
>  fills that role, but there are still struggles in the upstream's
>  interaction with the wider, established communities in terms of methods,
>  tooling choices, and tool architecture. 
I do not know why you are flaming the publican team over this. The only 
bug raised in relation to the three problems listed above, which BTW I 
had to open myself, was the inability to generate HTML reports with two 
character language definitions.

To my knowledge all other functionality works correctly with two 
character language definitions.

You can generate non-html statistics for two character language 
definitions, in the current version, by running 'make report-total-all'.

You can get per-language non html reports by running 'make 
report-total-<lang>' e.g. 'make report-total-es'

The bug in HTML reporting has already been fixed and will appear in the 
next version of publican.

I reworked the layout since I didn't like it, although I received no 
feedback from anyone on the previous layout :/

You can see the new output at http://jfearn.fedorapeople.org/Test_Report/

>   All of these can be solved, and
>  Fedora Docs commitment is to be a good citizen in trying to get features
>  worked out and architectural decisions revisited and fixed at the
>  Publican level. 
The architectural flaw here is in transifex. Expecting translators to 
work on 300000 word po files is not realistic.

The Test Report above is for the Fedora 10 Deployment Guide, it is 
293756 words, approximately 1175 hours of translation time, per 
language, having that in one po file is not a workable solution.

>  In the meantime, the higher priority parts of our mission can be
>  fulfilled with any DocBook XML toolchain, and we'll use whichever one is
>  working best within that 2 weeks. 
Please open a bug for any issue you have with publican. Critical bugs 
will get fixed ASAP.

Cheers, Jeff.