Intent to orphan dblatex (asciidoc dependency)

[Michael J Gruber]

This is to let you know that I intend to orphan dblatex.

Background: dblatex depends on python 2
(https://bugzilla.redhat.com/show_bug.cgi?id=1737967) which will be retired in Fedora 32
(https://fedoraproject.org/wiki/Changes/RetirePython2).

Some packages depend on dblatex directly, some indirectly via asciidoc which uses dblatex
to transform docbook via LaTeX to pdf. (Yes, one can imagine more direct ways of going
from asciidoc or docbook to pdf.)
In fact, upstream has been dead for quite a long time; its dependencies and the way it is
used (build documentation from shipped sources) justified keeping it for so long.

I have managed to port another package to python 3 and tried with dblatex but failed.
dblatex constructs various decoders and encoders for TeX to preserve (i.e. not quote) TeX
control characters. They read/write "text" (python 2 native strings) and
reencode as soon as they decided on a proper coding. This means that in many places in the
code there is no clear separation between "encoded", "decoded",
"raw", that is to say: between strings and bytes. Even style-wise, you see many
constructs like "line = object.someMethod(line)" where someMethod may or may not
transform line from encoded to decoded or the other way round. This kept at least me from
recognizing a clear byte/string separation.

Therefore, porting to python 3 is much more than simply marking some string literals as
bytes (b"abc") or some reads and writes as "b". I believe the
underlying parser classes need to be rewritten and its users adapted.

dblatex has to more maintainers: @alexlan and @jchaloup

[Jerry James]

On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber <mjg(a)fedoraproject.org&gt; wrote:
>  This is to let you know that I intend to orphan dblatex. 
Note that the gcc package uses dblatex to build libstdc++
documentation.  We as a project need to figure out what to do about
dblatex's python 2 dependency.
-- 
Jerry James
http://www.jamezone.org/

[Miro Hrončok]

On 03. 09. 19 17:57, Jerry James wrote:
>  On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber
> <mjg(a)fedoraproject.org&gt; wrote:
> > This is to let you know that I intend to orphan dblatex.
>  
>  Note that the gcc package uses dblatex to build libstdc++
>  documentation.  We as a project need to figure out what to do about
>  dblatex's python 2 dependency. 
I suppose it can get an exception to continue using Python 2, but since it 
appears dead upstream, maybe we should switch to a different tool instead?

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Jerry James]

On Tue, Sep 3, 2019 at 10:27 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  I suppose it can get an exception to continue using Python 2, but
> since it
>  appears dead upstream, maybe we should switch to a different tool instead? 
I'm interested to hear from the gcc developers just how ingrained the
dependency on dblatex is.  I maintain the gappa package, which
definitely depends on dblatex and nothing else.  I could just stop
building documentation for gappa, but that doesn't seem like a great
idea for libstdc++.  A lot of packages depend on asciidoc, too, so
something has to be done there.
-- 
Jerry James
http://www.jamezone.org/

[Jonathan Wakely]

On 03/09/19 09:57 -0600, Jerry James wrote:
> On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber
> <mjg(a)fedoraproject.org&gt; wrote:
> > This is to let you know that I intend to orphan dblatex.
>
> Note that the gcc package uses dblatex to build libstdc++
> documentation.  We as a project need to figure out what to do about
> dblatex's python 2 dependency. 
I don't think it's used in Fedora actually. I'm pretty sure it's only
used to generate the PDF version of the libstdc++ manual hosted on the
gcc.gnu.org website. Those PDF files aren't packaged for Fedora. I
think we can try just dropping the BuildRequires and see if it still
builds. And if not, what needs to be changed to get it building again.

For the upstream docs I'll look into using fop instead of dblatex.

[Jonathan Wakely]

On 03/09/19 18:34 +0100, Jonathan Wakely wrote:
> On 03/09/19 09:57 -0600, Jerry James wrote:
> >On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber <mjg(a)fedoraproject.org&gt; wrote:
> >>This is to let you know that I intend to orphan dblatex.
> >
> >Note that the gcc package uses dblatex to build libstdc++
> >documentation.  We as a project need to figure out what to do about
> >dblatex's python 2 dependency.
>
> I don't think it's used in Fedora actually. I'm pretty sure it's only
> used to generate the PDF version of the libstdc++ manual hosted on the
> gcc.gnu.org website. Those PDF files aren't packaged for Fedora. I
> think we can try just dropping the BuildRequires and see if it still
> builds. And if not, what needs to be changed to get it building again.
>
> For the upstream docs I'll look into using fop instead of dblatex. 
An x86_64 scratch build with this change completed successfully:

--- a/gcc.spec
+++ b/gcc.spec
@@ -190,7 +190,7 @@ Requires: libisl.so.15
 %endif
 %if %{build_libstdcxx_docs}
 BuildRequires: doxygen >= 1.7.1
-BuildRequires: graphviz, dblatex, texlive-collection-latex, docbook5-style-xsl
+BuildRequires: graphviz, texlive-collection-latex, docbook5-style-xsl
 %endif
 Requires: cpp = %{version}-%{release}
 # Need .eh_frame ld optimizations

So the dblatex dependency can just be dropped from gcc.spec, which
Jakub said he'll take care of when he updates it next.

[Josef Ridky]

From asciidoc perspective, dblatex is quite important for generating PDF files (even
it's called just via shell method).

I have contacted asciidoc upstream (asciidoc-py3 to be precised) with question about
dblatex replacement. 
I'll let you know once they reply.

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Jonathan Wakely" <jwakely(a)fedoraproject.org&gt;
| To: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
| Sent: Wednesday, September 4, 2019 10:42:01 AM
| Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| 
| On 03/09/19 18:34 +0100, Jonathan Wakely wrote:
| >On 03/09/19 09:57 -0600, Jerry James wrote:
| >>On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber <mjg(a)fedoraproject.org&gt;
| >>wrote:
| >>>This is to let you know that I intend to orphan dblatex.
| >>
| >>Note that the gcc package uses dblatex to build libstdc++
| >>documentation.  We as a project need to figure out what to do about
| >>dblatex's python 2 dependency.
| >
| >I don't think it's used in Fedora actually. I'm pretty sure it's
only
| >used to generate the PDF version of the libstdc++ manual hosted on the
| >gcc.gnu.org website. Those PDF files aren't packaged for Fedora. I
| >think we can try just dropping the BuildRequires and see if it still
| >builds. And if not, what needs to be changed to get it building again.
| >
| >For the upstream docs I'll look into using fop instead of dblatex.
| 
| An x86_64 scratch build with this change completed successfully:
| 
| --- a/gcc.spec
| +++ b/gcc.spec
| @@ -190,7 +190,7 @@ Requires: libisl.so.15
|  %endif
|  %if %{build_libstdcxx_docs}
|  BuildRequires: doxygen >= 1.7.1
| -BuildRequires: graphviz, dblatex, texlive-collection-latex,
| docbook5-style-xsl
| +BuildRequires: graphviz, texlive-collection-latex, docbook5-style-xsl
|  %endif
|  Requires: cpp = %{version}-%{release}
|  # Need .eh_frame ld optimizations
| 
| So the dblatex dependency can just be dropped from gcc.spec, which
| Jakub said he'll take care of when he updates it next.
| _______________________________________________
| devel mailing list -- devel(a)lists.fedoraproject.org
| To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| Fedora Code of Conduct:
| https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| List Archives:
| https://lists.fedoraproject.org/archives/list/[e-mail]
|

[Josef Ridky]

So good news, asciidoc(-py3) is already using FOP as second option for generating PDF
files.

So it just need to be set as default option instead of dblatex and all should be fine.

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Josef Ridky" <jridky(a)redhat.com&gt;
| To: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
| Sent: Wednesday, September 4, 2019 11:38:04 AM
| Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| 
| From asciidoc perspective, dblatex is quite important for generating PDF
| files (even it's called just via shell method).
| 
| I have contacted asciidoc upstream (asciidoc-py3 to be precised) with
| question about dblatex replacement.
| I'll let you know once they reply.
| 
| Josef Ridky
| Software Engineer
| Core Services Team
| Red Hat Czech, s.r.o.
| 
| ----- Original Message -----
| | From: "Jonathan Wakely" <jwakely(a)fedoraproject.org&gt;
| | To: "Development discussions related to Fedora"
| | <devel(a)lists.fedoraproject.org&gt;
| | Sent: Wednesday, September 4, 2019 10:42:01 AM
| | Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| | 
| | On 03/09/19 18:34 +0100, Jonathan Wakely wrote:
| | >On 03/09/19 09:57 -0600, Jerry James wrote:
| | >>On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber <mjg(a)fedoraproject.org&gt;
| | >>wrote:
| | >>>This is to let you know that I intend to orphan dblatex.
| | >>
| | >>Note that the gcc package uses dblatex to build libstdc++
| | >>documentation.  We as a project need to figure out what to do about
| | >>dblatex's python 2 dependency.
| | >
| | >I don't think it's used in Fedora actually. I'm pretty sure it's
only
| | >used to generate the PDF version of the libstdc++ manual hosted on the
| | >gcc.gnu.org website. Those PDF files aren't packaged for Fedora. I
| | >think we can try just dropping the BuildRequires and see if it still
| | >builds. And if not, what needs to be changed to get it building again.
| | >
| | >For the upstream docs I'll look into using fop instead of dblatex.
| | 
| | An x86_64 scratch build with this change completed successfully:
| | 
| | --- a/gcc.spec
| | +++ b/gcc.spec
| | @@ -190,7 +190,7 @@ Requires: libisl.so.15
| |  %endif
| |  %if %{build_libstdcxx_docs}
| |  BuildRequires: doxygen >= 1.7.1
| | -BuildRequires: graphviz, dblatex, texlive-collection-latex,
| | docbook5-style-xsl
| | +BuildRequires: graphviz, texlive-collection-latex, docbook5-style-xsl
| |  %endif
| |  Requires: cpp = %{version}-%{release}
| |  # Need .eh_frame ld optimizations
| | 
| | So the dblatex dependency can just be dropped from gcc.spec, which
| | Jakub said he'll take care of when he updates it next.
| | _______________________________________________
| | devel mailing list -- devel(a)lists.fedoraproject.org
| | To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| | Fedora Code of Conduct:
| | https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| | List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| | List Archives:
| | https://lists.fedoraproject.org/archives/list/[e-mail]
| | 
| _______________________________________________
| devel mailing list -- devel(a)lists.fedoraproject.org
| To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| Fedora Code of Conduct:
| https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| List Archives:
| https://lists.fedoraproject.org/archives/list/[e-mail]
|

[Josef Ridky]

But due FOP doesn't use Latex, its configuration is different and FOP output looks
very different to the Latex output, it would be probably better to request Python2
exception for dblatex (or rewrite it to Python3).

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Josef Ridky" <jridky(a)redhat.com&gt;
| To: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
| Sent: Wednesday, September 4, 2019 11:47:41 AM
| Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| 
| So good news, asciidoc(-py3) is already using FOP as second option for
| generating PDF files.
| 
| So it just need to be set as default option instead of dblatex and all should
| be fine.
| 
| Josef Ridky
| Software Engineer
| Core Services Team
| Red Hat Czech, s.r.o.
| 
| ----- Original Message -----
| | From: "Josef Ridky" <jridky(a)redhat.com&gt;
| | To: "Development discussions related to Fedora"
| | <devel(a)lists.fedoraproject.org&gt;
| | Sent: Wednesday, September 4, 2019 11:38:04 AM
| | Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| | 
| | From asciidoc perspective, dblatex is quite important for generating PDF
| | files (even it's called just via shell method).
| | 
| | I have contacted asciidoc upstream (asciidoc-py3 to be precised) with
| | question about dblatex replacement.
| | I'll let you know once they reply.
| | 
| | Josef Ridky
| | Software Engineer
| | Core Services Team
| | Red Hat Czech, s.r.o.
| | 
| | ----- Original Message -----
| | | From: "Jonathan Wakely" <jwakely(a)fedoraproject.org&gt;
| | | To: "Development discussions related to Fedora"
| | | <devel(a)lists.fedoraproject.org&gt;
| | | Sent: Wednesday, September 4, 2019 10:42:01 AM
| | | Subject: Re: Intent to orphan dblatex (asciidoc dependency)
| | | 
| | | On 03/09/19 18:34 +0100, Jonathan Wakely wrote:
| | | >On 03/09/19 09:57 -0600, Jerry James wrote:
| | | >>On Tue, Sep 3, 2019 at 7:37 AM Michael J Gruber
<mjg(a)fedoraproject.org&gt;
| | | >>wrote:
| | | >>>This is to let you know that I intend to orphan dblatex.
| | | >>
| | | >>Note that the gcc package uses dblatex to build libstdc++
| | | >>documentation.  We as a project need to figure out what to do about
| | | >>dblatex's python 2 dependency.
| | | >
| | | >I don't think it's used in Fedora actually. I'm pretty sure it's
only
| | | >used to generate the PDF version of the libstdc++ manual hosted on the
| | | >gcc.gnu.org website. Those PDF files aren't packaged for Fedora. I
| | | >think we can try just dropping the BuildRequires and see if it still
| | | >builds. And if not, what needs to be changed to get it building again.
| | | >
| | | >For the upstream docs I'll look into using fop instead of dblatex.
| | | 
| | | An x86_64 scratch build with this change completed successfully:
| | | 
| | | --- a/gcc.spec
| | | +++ b/gcc.spec
| | | @@ -190,7 +190,7 @@ Requires: libisl.so.15
| | |  %endif
| | |  %if %{build_libstdcxx_docs}
| | |  BuildRequires: doxygen >= 1.7.1
| | | -BuildRequires: graphviz, dblatex, texlive-collection-latex,
| | | docbook5-style-xsl
| | | +BuildRequires: graphviz, texlive-collection-latex, docbook5-style-xsl
| | |  %endif
| | |  Requires: cpp = %{version}-%{release}
| | |  # Need .eh_frame ld optimizations
| | | 
| | | So the dblatex dependency can just be dropped from gcc.spec, which
| | | Jakub said he'll take care of when he updates it next.
| | | _______________________________________________
| | | devel mailing list -- devel(a)lists.fedoraproject.org
| | | To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| | | Fedora Code of Conduct:
| | | https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| | | List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| | | List Archives:
| | | https://lists.fedoraproject.org/archives/list/[e-mail]
| | | 
| | _______________________________________________
| | devel mailing list -- devel(a)lists.fedoraproject.org
| | To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| | Fedora Code of Conduct:
| | https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| | List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| | List Archives:
| | https://lists.fedoraproject.org/archives/list/[e-mail]
| | 
| _______________________________________________
| devel mailing list -- devel(a)lists.fedoraproject.org
| To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
| Fedora Code of Conduct:
| https://docs.fedoraproject.org/en-US/project/code-of-conduct/
| List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
| List Archives:
| https://lists.fedoraproject.org/archives/list/[e-mail]
|

[Michael J Gruber]

Okay, just quickly two good news:

- Nikola Forró has a patch that got me much further, there's hope for a working
dblatex on py3!
- Upstream showed life signs, I'll try to coordinate (and get patches upstreamed now
or later once we have them ready).

[Jerry James]

On Mon, Sep 9, 2019 at 7:16 AM Michael J Gruber <mjg(a)fedoraproject.org&gt; wrote:
>  Okay, just quickly two good news:
>
>  - Nikola Forró has a patch that got me much further, there's hope for a working
> dblatex on py3!
>  - Upstream showed life signs, I'll try to coordinate (and get patches upstreamed now
> or later once we have them ready). 
That is good news.  Thanks for sticking with this.
-- 
Jerry James
http://www.jamezone.org/

[Michael J Gruber]

Okay, more news:

There's a repo for the porting efforts, with NF's and my patches combined and
split into logical chunks:
https://github.com/mjg/dblatex-py3
Let's hope upstream can work with that ;)

There's a cor rep where I've built dblatex with py3, and where I am rebuilding all
packages which BR dblatex against that dblatex:
https://copr.fedorainfracloud.org/coprs/mjg/dblatex/monitor/
Things looking OK so far, but I haven't compared the actual build products (docs) for
consistency.
Note that some of these packages (build-) require python2 for other reasons; gimp-help
even requires py2 to build but does not BR it. Will FTBFS on F32 as is.

[Neal Gompa]

On Tue, Sep 3, 2019 at 9:37 AM Michael J Gruber <mjg(a)fedoraproject.org&gt; wrote:
>  This is to let you know that I intend to orphan dblatex.
>
>  Background: dblatex depends on python 2
> (https://bugzilla.redhat.com/show_bug.cgi?id=1737967) which will be retired in Fedora 32
> (https://fedoraproject.org/wiki/Changes/RetirePython2).
>
>  Some packages depend on dblatex directly, some indirectly via asciidoc which uses dblatex
> to transform docbook via LaTeX to pdf. (Yes, one can imagine more direct ways of going
> from asciidoc or docbook to pdf.)
>  In fact, upstream has been dead for quite a long time; its dependencies and the way it is
> used (build documentation from shipped sources) justified keeping it for so long. 
Does this still apply with the new Python 3 port of asciidoc?
https://github.com/asciidoc/asciidoc-py3



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Michael J Gruber]

>  On Tue, Sep 3, 2019 at 9:37 AM Michael J Gruber
> <mjg(a)fedoraproject.org&gt; wrote:
>  
>  Does this still apply with the new Python 3 port of asciidoc?
>  https://github.com/asciidoc/asciidoc-py3 
Yes. asciidoc-py3 still calls dblatex which depends on python 2.

Note that asciidoc-py3 is a port just to make current projects work, not a continuing
project; asciidoc upstream recommends switching to an active alternative like asciidoctor,
which comes with a pdf backend. But this will need adjustment of your config.

That being said, asciidoc(-py3) needs dblatex only for generating pdf output, and  -
apparantly - has an alternative tool for that. So, out of the many Fedora packages which
formally depend on dblatex via asciidoc, only those which generate pdf output really
depend on dblatex.