F26 Self Contained Change: Transdiff

[Jan Kurik]

= Proposed Self Contained Change: Transdiff =
https://fedoraproject.org/wiki/Changes/Transdiff

Change owner(s):
*Sundeep Anand <suanand AT redhat DOT com>

Often even after 100% translation in Zanata, few packages do not get
build with latest translations in Fedora. This result in poor
localization experience. Transdiff is a python program to run on
products installations for tracking translations with project upstream
and generate diff reports.

== Detailed Description ==
Often even after 100% translation in Zanata, few packages do not get
build with latest translations in Fedora. This result in poor
localization experience. Transdiff is a python program to run on
products installations for tracking translations with project upstream
and generate diff reports.

== Scope ==
* Proposal owners: Complete script and package it for Fedora.

* Other developers: N/A (not a System Wide Change)

* Release engineering: N/A (not a System Wide Change)

* List of deliverables: N/A (not a System Wide Change)

* Policies and guidelines: N/A (not a System Wide Change)

* Trademark approval: N/A (not needed for this Change)

-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

[Tom Hughes]

On 10/01/17 08:39, Jan Kurik wrote:

>  Often even after 100% translation in Zanata, few packages do not get
>  build with latest translations in Fedora. This result in poor
>  localization experience. 
I don't really understand the logic here...

I would expect anything in Fedora to be built with whatever translations 
are provided in the upstream release - are you saying that packagers are 
routinely removing such translations for some reason? or just forgetting 
to package the extra translations?

Or are you proposing that packagers should be pulling translations from 
some additional source above and beyond what upstream provides?

Zanata appears to be a web based translation platform that developers 
can use so I would expect the workflow for a package that uses Zanata to 
be that upstream pulls the latest translations from it at regular 
intervals and incorporates them in the releases they make which then get 
packaged in Fedora.

I'm not saying we should package a tool for analysing translation status 
as I'm sure it will be useful to upstream developers but it's not clear 
how you envisage it fitting into the Fedora development process.

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

On 10 January 2017 at 14:39, Tom Hughes <tom(a)compton.nu&gt; wrote:

>  On 10/01/17 08:39, Jan Kurik wrote:
>
>  Often even after 100% translation in Zanata, few packages do not get
> > build with latest translations in Fedora. This result in poor
> > localization experience.
> >
>
>  I don't really understand the logic here...
>
>  I would expect anything in Fedora to be built with whatever translations
>  are provided in the upstream release - are you saying that packagers are
>  routinely removing such translations for some reason? or just forgetting to
>  package the extra translations?
>  
Yes, forgetting to pull translation is one of the problem.
Also other problem is packages updating strings after translation is
completed. This also result in loss of translation.


>  Or are you proposing that packagers should be pulling translations from
>  some additional source above and beyond what upstream provides?
>  
No, not this.


>  Zanata appears to be a web based translation platform that developers can
>  use so I would expect the workflow for a package that uses Zanata to be
>  that upstream pulls the latest translations from it at regular intervals
>  and incorporates them in the releases they make which then get packaged in
>  Fedora.
>
>  I'm not saying we should package a tool for analysing translation status
>  as I'm sure it will be useful to upstream developers but it's not clear how
>  you envisage it fitting into the Fedora development process.
>  
In Fedora development process, i think such a script/project will be
helpful like.
After Beta release, one can simply run this script on fully installed
translation machine and verify whether all latest translation from upstream
are available in Fedora or is there any specific string breakage (i.e. New
English words in package ).

At the present still exploring ideas, once we have basic script ready we
can further think how can we use it for more benefits to Fedora translation
quality.

-
Pravin

[Tom Hughes]

On 12/01/17 06:59, pravin.d.s(a)gmail.com wrote:

>  On 10 January 2017 at 14:39, Tom Hughes <tom(a)compton.nu
>  <mailto:[e-mail]>> wrote:
>
>      I don't really understand the logic here...
>
>      I would expect anything in Fedora to be built with whatever
>      translations are provided in the upstream release - are you saying
>      that packagers are routinely removing such translations for some
>      reason? or just forgetting to package the extra translations?
>
>  Yes, forgetting to pull translation is one of the problem.
>  Also other problem is packages updating strings after translation is
>  completed. This also result in loss of translation. 
Right, but both of those are primarily issues for upstream rather than 
for Fedora packages.

>      Or are you proposing that packagers should be pulling
> translations
>      from some additional source above and beyond what upstream provides?
>
>  No, not this. 
OK.

>      I'm not saying we should package a tool for analysing
> translation
>      status as I'm sure it will be useful to upstream developers but it's
>      not clear how you envisage it fitting into the Fedora development
>      process.
>
>  In Fedora development process, i think such a script/project will be
>  helpful like.
>  After Beta release, one can simply run this script on fully installed
>  translation machine and verify whether all latest translation from
>  upstream are available in Fedora or is there any specific string
>  breakage (i.e. New English words in package ). 
Right, but as a Fedora packager if I have the latest upstream packaged 
and this script says I am missing translations what will I be expected 
to do about it?

You've already said you don't expect packagers to patch in the extra 
strings so we can rule that out.

So presumably I'm just supposed to open a bug with upstream to tell them 
and then hope they do a new release and that I can then get that in 
before GA?

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

On 12 January 2017 at 14:33, Tom Hughes <tom(a)compton.nu&gt; wrote:

>  On 12/01/17 06:59, pravin.d.s(a)gmail.com wrote:
>
>  On 10 January 2017 at 14:39, Tom Hughes <tom(a)compton.nu
> > <mailto:[e-mail]>> wrote:
> >
> >     I don't really understand the logic here...
> >
> >     I would expect anything in Fedora to be built with whatever
> >     translations are provided in the upstream release - are you saying
> >     that packagers are routinely removing such translations for some
> >     reason? or just forgetting to package the extra translations?
> >
> > Yes, forgetting to pull translation is one of the problem.
> > Also other problem is packages updating strings after translation is
> > completed. This also result in loss of translation.
> >
>
>  Right, but both of those are primarily issues for upstream rather than for
>  Fedora packages.
>  
Yes

>      Or are you proposing that packagers should be pulling translations
> >     from some additional source above and beyond what upstream provides?
> >
> > No, not this.
> >
>
>  OK.
>
>      I'm not saying we should package a tool for analysing translation
> >     status as I'm sure it will be useful to upstream developers but it's
> >     not clear how you envisage it fitting into the Fedora development
> >     process.
> >
> > In Fedora development process, i think such a script/project will be
> > helpful like.
> > After Beta release, one can simply run this script on fully installed
> > translation machine and verify whether all latest translation from
> > upstream are available in Fedora or is there any specific string
> > breakage (i.e. New English words in package ).
> >
>
>  Right, but as a Fedora packager if I have the latest upstream packaged and
>  this script says I am missing translations what will I be expected to do
>  about it?
>
>  You've already said you don't expect packagers to patch in the extra
>  strings so we can rule that out.
>
>  So presumably I'm just supposed to open a bug with upstream to tell them
>  and then hope they do a new release and that I can then get that in before
>  GA? 

Perfect. This is more from Translation team side. Since many time even
after 100% translation in Zanata, we do not get those in Fedora release.
Basically translators will report bugs for this.

-
Pravin

[Dominik 'Rathann' Mierzejewski]

On Tuesday, 10 January 2017 at 09:39, Jan Kurik wrote:
>  = Proposed Self Contained Change: Transdiff =
>  https://fedoraproject.org/wiki/Changes/Transdiff
>  
>  Change owner(s):
>  *Sundeep Anand <suanand AT redhat DOT com>
>  
>  Often even after 100% translation in Zanata, few packages do not get
>  build with latest translations in Fedora. This result in poor
>  localization experience. Transdiff is a python program to run on
>  products installations for tracking translations with project upstream
>  and generate diff reports. 
Nice. On the wiki page, you (Sundeep) also mention automation.
Where do you see this in our packaging pipeline?

As a taskotron check? A regular report mailed to the devel list?

>  == Detailed Description ==
>  Often even after 100% translation in Zanata, few packages do not get
>  build with latest translations in Fedora. This result in poor
>  localization experience. Transdiff is a python program to run on
>  products installations for tracking translations with project upstream
>  and generate diff reports. 
Same as summary. Also, why does this warrant a change?

Regards,
Dominik
-- 
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
        -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

[Matthew Miller]

On Tue, Jan 10, 2017 at 12:01:38PM +0100, Dominik 'Rathann' Mierzejewski wrote:
>  Nice. On the wiki page, you (Sundeep) also mention automation.
>  Where do you see this in our packaging pipeline?
>  
>  As a taskotron check? A regular report mailed to the devel list? 
A taskotron check sending mail to the translations list?

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

On 10 January 2017 at 16:31, Dominik 'Rathann' Mierzejewski <
dominik(a)greysector.net&gt; wrote:

>  On Tuesday, 10 January 2017 at 09:39, Jan Kurik wrote:
>  > = Proposed Self Contained Change: Transdiff =
>  > https://fedoraproject.org/wiki/Changes/Transdiff
>  >
>  > Change owner(s):
>  > *Sundeep Anand <suanand AT redhat DOT com>
>  >
>  > Often even after 100% translation in Zanata, few packages do not get
>  > build with latest translations in Fedora. This result in poor
>  > localization experience. Transdiff is a python program to run on
>  > products installations for tracking translations with project upstream
>  > and generate diff reports.
>
>  Nice. On the wiki page, you (Sundeep) also mention automation.
>  Where do you see this in our packaging pipeline?
>
>  As a taskotron check? A regular report mailed to the devel list?
>  
Good point. While discussing on string breakage monitoring we discussed
this idea as well.
Yeah, taskotron is good fit for uses of transdiff, so rather than post
installation of translation, we will have this check in Bodhi itself.

-
Pravin

[Dominik 'Rathann' Mierzejewski]

Hello, Sundeep.

On Thursday, 12 January 2017 at 08:03, pravin.d.s(a)gmail.com wrote:
>  On 10 January 2017 at 16:31, Dominik 'Rathann' Mierzejewski
> <
>  dominik(a)greysector.net&gt; wrote:
>  
>  > On Tuesday, 10 January 2017 at 09:39, Jan Kurik wrote:
>  > > = Proposed Self Contained Change: Transdiff =
>  > > https://fedoraproject.org/wiki/Changes/Transdiff
>  > >
>  > > Change owner(s):
>  > > *Sundeep Anand <suanand AT redhat DOT com>
>  > >
>  > > Often even after 100% translation in Zanata, few packages do not get
>  > > build with latest translations in Fedora. This result in poor
>  > > localization experience. Transdiff is a python program to run on
>  > > products installations for tracking translations with project upstream
>  > > and generate diff reports.
>  >
>  > Nice. On the wiki page, you (Sundeep) also mention automation.
>  > Where do you see this in our packaging pipeline?
>  >
>  > As a taskotron check? A regular report mailed to the devel list?
>  
>  Good point. While discussing on string breakage monitoring we discussed
>  this idea as well.
>  Yeah, taskotron is good fit for uses of transdiff, so rather than post
>  installation of translation, we will have this check in Bodhi itself. 
Could you update the proposal with the conclusions from this thread
and also, more importantly, answer my question? Why does this need
to be a Change?

Regards,
Dominik
-- 
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
        -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

[Sundeep Anand]

Hi Dominik,

Thanks, I have updated the proposal. I see solution to this in two parts:

1) Some service/web-application which do specified tasks based on
milestones in a release cycle (translation-specific) including
notification. This should be talking to Fedora apps/services (koji, fas,
etc.) and translation platforms (zanata, pootle, etc.) where we have
translations of Fedora packages. Plus keep upstream info as well. A project
is already under development on this line[1].

2) A translation check during/post build - as discussed it (transdiff) will
be a taskotron task.

Coming back to - why this need to be a change - sorry, I do not have any
concrete answer at this point of time, perhaps as I need some more time
working on this. I believe aligning this with *Fedora 27* would be more
meaningful and working.

thanks,
sundeep

On Fri, Jan 20, 2017 at 9:54 PM, Dominik 'Rathann' Mierzejewski <
dominik(a)greysector.net&gt; wrote:

>  Hello, Sundeep.
>
>  On Thursday, 12 January 2017 at 08:03, pravin.d.s(a)gmail.com wrote:
>  > On 10 January 2017 at 16:31, Dominik 'Rathann' Mierzejewski <
>  > dominik(a)greysector.net&gt; wrote:
>  >
>  > > On Tuesday, 10 January 2017 at 09:39, Jan Kurik wrote:
>  > > > = Proposed Self Contained Change: Transdiff =
>  > > > https://fedoraproject.org/wiki/Changes/Transdiff
>  > > >
>  > > > Change owner(s):
>  > > > *Sundeep Anand <suanand AT redhat DOT com>
>  > > >
>  > > > Often even after 100% translation in Zanata, few packages do not get
>  > > > build with latest translations in Fedora. This result in poor
>  > > > localization experience. Transdiff is a python program to run on
>  > > > products installations for tracking translations with project
>  upstream
>  > > > and generate diff reports.
>  > >
>  > > Nice. On the wiki page, you (Sundeep) also mention automation.
>  > > Where do you see this in our packaging pipeline?
>  > >
>  > > As a taskotron check? A regular report mailed to the devel list?
>  >
>  > Good point. While discussing on string breakage monitoring we discussed
>  > this idea as well.
>  > Yeah, taskotron is good fit for uses of transdiff, so rather than post
>  > installation of translation, we will have this check in Bodhi itself.
>
>  Could you update the proposal with the conclusions from this thread
>  and also, more importantly, answer my question? Why does this need
>  to be a Change?
>
>  Regards,
>  Dominik
>  --
>  Fedora http://fedoraproject.org/wiki/User:Rathann
>  RPMFusion http://rpmfusion.org
>  "Faith manages."
>          -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>

[Sundeep Anand]

[1] https://github.com/sundeep-co-in/transtats

On Wed, Jan 25, 2017 at 1:38 PM, Sundeep Anand <suanand(a)redhat.com&gt; wrote:

>  Hi Dominik,
>
>  Thanks, I have updated the proposal. I see solution to this in two parts:
>
>  1) Some service/web-application which do specified tasks based on
>  milestones in a release cycle (translation-specific) including
>  notification. This should be talking to Fedora apps/services (koji, fas,
>  etc.) and translation platforms (zanata, pootle, etc.) where we have
>  translations of Fedora packages. Plus keep upstream info as well. A project
>  is already under development on this line[1].
>
>  2) A translation check during/post build - as discussed it (transdiff)
>  will be a taskotron task.
>
>  Coming back to - why this need to be a change - sorry, I do not have any
>  concrete answer at this point of time, perhaps as I need some more time
>  working on this. I believe aligning this with *Fedora 27* would be more
>  meaningful and working.
>
>  thanks,
>  sundeep
>
>  On Fri, Jan 20, 2017 at 9:54 PM, Dominik 'Rathann' Mierzejewski <
>  dominik(a)greysector.net&gt; wrote:
>
> > Hello, Sundeep.
> >
> > On Thursday, 12 January 2017 at 08:03, pravin.d.s(a)gmail.com wrote:
> > > On 10 January 2017 at 16:31, Dominik 'Rathann' Mierzejewski <
> > > dominik(a)greysector.net&gt; wrote:
> > >
> > > > On Tuesday, 10 January 2017 at 09:39, Jan Kurik wrote:
> > > > > = Proposed Self Contained Change: Transdiff =
> > > > > https://fedoraproject.org/wiki/Changes/Transdiff
> > > > >
> > > > > Change owner(s):
> > > > > *Sundeep Anand <suanand AT redhat DOT com>
> > > > >
> > > > > Often even after 100% translation in Zanata, few packages do not get
> > > > > build with latest translations in Fedora. This result in poor
> > > > > localization experience. Transdiff is a python program to run on
> > > > > products installations for tracking translations with project
> > upstream
> > > > > and generate diff reports.
> > > >
> > > > Nice. On the wiki page, you (Sundeep) also mention automation.
> > > > Where do you see this in our packaging pipeline?
> > > >
> > > > As a taskotron check? A regular report mailed to the devel list?
> > >
> > > Good point. While discussing on string breakage monitoring we discussed
> > > this idea as well.
> > > Yeah, taskotron is good fit for uses of transdiff, so rather than post
> > > installation of translation, we will have this check in Bodhi itself.
> >
> > Could you update the proposal with the conclusions from this thread
> > and also, more importantly, answer my question? Why does this need
> > to be a Change?
> >
> > Regards,
> > Dominik
> > --
> > Fedora http://fedoraproject.org/wiki/User:Rathann
> > RPMFusion http://rpmfusion.org
> > "Faith manages."
> >         -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> >
>
>