N00b questions - I need help getting started on packaging.

[Ludovic Hirlimann]

Hi,
I'm ludo - I'd like to package josm - which requires Apcahe derby and apache-common-jcs-*.

I've managed to compile derby on my fedora 32 vagrant host. 
For that I had to force the compiler to not be the 1.8 using the JAVA_HOME variable so ants would use the compiler from the 11-openjdk-devel package.

I'm reading https://docs.fedoraproject.org/en-US/packaging-guidelines/Java/ and https://fedora-java.github.io/howto/latest/. 
I'm far from being a java programmer and this is my first attempt at building a linux package. And that's why I'm writing this email.

I've got some inspiration from https://src.fedoraproject.org/rpms/derby/pull-request/1#request_diff as derby used to be packaged for fedora/RHEL in the past.

Looking at that spec file, I don't see how one can specify which version of javac to use and force while buidling. I also don't understand the need to the patch files nor do I get the fact that this ant build needs maven magic in the packaging.

I'll gladly RTFM anything sent to me, but I'd like to figure out the next steps in order to package derby , now that I have a build that doesn't fail.

Thanks in advance for your patience and time
Ludovic

[Mat Booth]

On Fri, 14 Aug 2020 at 13:36, Ludovic Hirlimann  wrote:
> Hi,
> I'm ludo - I'd like to package josm - which requires Apcahe derby and apache-common-jcs-*.
>
> I've managed to compile derby on my fedora 32 vagrant host.
> For that I had to force the compiler to not be the 1.8 using the JAVA_HOME variable so ants would use the compiler from the 11-openjdk-devel package.
>
> I'm reading https://docs.fedoraproject.org/en-US/packaging-guidelines/Java/ and https://fedora-java.github.io/howto/latest/.
> I'm far from being a java programmer and this is my first attempt at building a linux package. And that's why I'm writing this email.
>
> I've got some inspiration from https://src.fedoraproject.org/rpms/derby/pull-request/1#request_diff as derby used to be packaged for fedora/RHEL in the past.
>
> Looking at that spec file, I don't see how one can specify which version of javac to use and force while buidling. I also don't understand the need to the patch files nor do I get the fact that this ant build needs maven magic in the packaging.
>
> I'll gladly RTFM anything sent to me, but I'd like to figure out the next steps in order to package derby , now that I have a build that doesn't fail.
>
> Thanks in advance for your patience and time
> Ludovic

From F33 the default version of Java is 11 but you can force a
specific version by adding a BR (BuildRequires) on the appropriate
java-*-devel package and setting JAVA_HOME in the environment before
calling ant or maven.

E.g.:

BuildRequires: java-1.8.0-openjdk-devel
...
%build
export JAVA_HOME=%{_jvmdir}/java-1.8.0
ant ...

Patches are probably needed because Lucene and JavaCC dependencies are
newer in Fedora than expected by Derby. Where possible we generally
try to carry only the latest version of libraries in order to avoid a
combinatorial explosion of things we'd otherwise need to package.
Packages should be patched such that they work with the version
available in Fedora. Updating a project to the latest version of one
of its dependencies are patches that are good candidates to be sent
upstream.

Maven "magic" (actually https://fedora-java.github.io/howto/latest/
does a great job at explaining the macros, see also their man pages)
is needed for the same reason Derby is published to maven central even
though it is built with ant: Maven is the de-facto way of managing
one's dependencies in the Java world. The major build tools maven,
gradle and ant/ivy all speak to maven repositories to resolve their
dependencies. Since other Fedora packages might rely on your package
as a dependency, your packages must be installed in such a way that
Fedora's version of maven can understand.

Does this all make sense?

[Ludovic Hirlimann]

On 8/14/20 4:53 PM, Mat Booth wrote:
> On Fri, 14 Aug 2020 at 13:36, Ludovic Hirlimann  wrote:
> > Hi,
> > I'm ludo - I'd like to package josm - which requires Apcahe derby and apache-common-jcs-*.
> >
> > I've managed to compile derby on my fedora 32 vagrant host.
> > For that I had to force the compiler to not be the 1.8 using the JAVA_HOME variable so ants would use the compiler from the 11-openjdk-devel package.
> >
> > I'm reading https://docs.fedoraproject.org/en-US/packaging-guidelines/Java/ and https://fedora-java.github.io/howto/latest/.
> > I'm far from being a java programmer and this is my first attempt at building a linux package. And that's why I'm writing this email.
> >
> > I've got some inspiration from https://src.fedoraproject.org/rpms/derby/pull-request/1#request_diff as derby used to be packaged for fedora/RHEL in the past.
> >
> > Looking at that spec file, I don't see how one can specify which version of javac to use and force while buidling. I also don't understand the need to the patch files nor do I get the fact that this ant build needs maven magic in the packaging.
> >
> > I'll gladly RTFM anything sent to me, but I'd like to figure out the next steps in order to package derby , now that I have a build that doesn't fail.
> >
> > Thanks in advance for your patience and time
> > Ludovic
>
> From F33 the default version of Java is 11 but you can force a
> specific version by adding a BR (BuildRequires) on the appropriate
> java-*-devel package and setting JAVA_HOME in the environment before
> calling ant or maven.
>
> E.g.:
>
> BuildRequires: java-1.8.0-openjdk-devel
> ...
> %build
> export JAVA_HOME=%{_jvmdir}/java-1.8.0
> ant ...

Thank you very much. This is duly noted, derby's requirement is >9, so
11 is fine.
> Patches are probably needed because Lucene and JavaCC dependencies are
> newer in Fedora than expected by Derby. Where possible we generally
> try to carry only the latest version of libraries in order to avoid a
> combinatorial explosion of things we'd otherwise need to package.

Noted and understood. I'll have a closer look at these then.
> Packages should be patched such that they work with the version
> available in Fedora. Updating a project to the latest version of one
> of its dependencies are patches that are good candidates to be sent
> upstream.
>
> Maven "magic" (actually https://fedora-java.github.io/howto/latest/
> does a great job at explaining the macros, see also their man pages)
> is needed for the same reason Derby is published to maven central even
> though it is built with ant: Maven is the de-facto way of managing
> one's dependencies in the Java world. The major build tools maven,
> gradle and ant/ivy all speak to maven repositories to resolve their
> dependencies. Since other Fedora packages might rely on your package
> as a dependency, your packages must be installed in such a way that
> Fedora's version of maven can understand.
>
> Does this all make sense?

It does. I just need to read an learn a *lot* more about the Java ecosystem.

Thank you

Ludo

[Ludovic Hirlimann]

On 8/14/20 4:53 PM, Mat Booth wrote:
> Patches are probably needed because Lucene and JavaCC dependencies are
> newer in Fedora than expected by Derby. Where possible we generally
> try to carry only the latest version of libraries in order to avoid a
> combinatorial explosion of things we'd otherwise need to package.
> Packages should be patched such that they work with the version
> available in Fedora. Updating a project to the latest version of one
> of its dependencies are patches that are good candidates to be sent

So when I do a dnf search Lucene I don't get any version number in the
pckage name, what's the fedora way to know which version of Lucene is
being currently packaged ? (reading the spec file ?) , I now know what
the dependancies for Derby are in terms of version numbers?

Ludo

[Markku Korkeala]

On 8/22/20 1:16 PM, Ludovic Hirlimann wrote:
> So when I do a dnf search Lucene I don't get any version number in the
> pckage name, what's the fedora way to know which version of Lucene is
> being currently packaged ? (reading the spec file ?) , I now know what
> the dependancies for Derby are in terms of version numbers?

Hi,

the dnf has many options, you can try dnf --help to get available
subcommands and options. Running dnf info <package> gives you more
information about the package including the version:
 dnf info lucene
Last metadata expiration check: 0:02:50 ago on Sat 22 Aug 2020 02:57:13
PM EEST.
Available Packages
Name         : lucene
Version      : 8.1.1
Release      : 4.fc32
Architecture : noarch
Size         : 2.8 M
Source       : lucene-8.1.1-4.fc32.src.rpm
Repository   : fedora
Summary      : High-performance, full-featured text search engine
URL          : http://lucene.apache.org/
License      : ASL 2.0
Description  : Apache Lucene is a high-performance, full-featured text
search
             : engine library written entirely in Java. It is a
technology suitable
             : for nearly any application that requires full-text
search, especially
             : cross-platform.

Best wishes,
Markku
> Ludo
>
> java-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...

[Mat Booth]

On Sat, 22 Aug 2020 at 11:17, Ludovic Hirlimann  wrote:
> On 8/14/20 4:53 PM, Mat Booth wrote:
> > Patches are probably needed because Lucene and JavaCC dependencies are
> > newer in Fedora than expected by Derby. Where possible we generally
> > try to carry only the latest version of libraries in order to avoid a
> > combinatorial explosion of things we'd otherwise need to package.
> > Packages should be patched such that they work with the version
> > available in Fedora. Updating a project to the latest version of one
> > of its dependencies are patches that are good candidates to be sent
>
> So when I do a dnf search Lucene I don't get any version number in the
> pckage name, what's the fedora way to know which version of Lucene is
> being currently packaged ? (reading the spec file ?) , I now know what
> the dependancies for Derby are in terms of version numbers?

Using repoquery on the rawhide repo:

$ sudo dnf --repo=rawhide repoquery lucene
lucene-0:8.4.1-9.fc33.noarch

This gives results as: <package>-<epoch>:<version>-<release>.<arch>

So lucene package is currently at version 8.4.1

[Ludovic Hirlimann]

On 8/14/20 4:53 PM, Mat Booth wrote:
> Patches are probably needed because Lucene and JavaCC dependencies are
> newer in Fedora than expected by Derby. Where possible we generally
> try to carry only the latest version of libraries in order to avoid a
> combinatorial explosion of things we'd otherwise need to package.
> Packages should be patched such that they work with the version
> available in Fedora. Updating a project to the latest version of one
> of its dependencies are patches that are good candidates to be sent
> upstream.

Thanks every one for your help so far.

Now that I understand the patches at
https://src.fedoraproject.org/rpms/derby/pull-request/1#request_diff

I have updated these, I wrote a spec file.

I'm getting a compile error :
> optional:
>     [javac] Compiling 15 source files to
> /home/vagrant/rpmbuild/BUILD/derby-10.15.2.0/db-derby-10.15.2.0-src/classes/optional
>     [javac] error: the unnamed module reads package
> org.apache.lucene.analysis.standard from both lucene.analyzers.common
> and lucene.core
>     [javac] error: module lucene.analyzers.common reads package
> org.apache.lucene.analysis.standard from both lucene.analyzers.common
> and lucene.core
>     [javac] error: module lucene.core reads package
> org.apache.lucene.analysis.standard from both lucene.analyzers.common
> and lucene.core
>     [javac] error: module lucene.queryparser reads package
> org.apache.lucene.analysis.standard from both lucene.core and
> lucene.analyzers.common
>     [javac] error: module json.simple reads package
> org.apache.lucene.analysis.standard from both lucene.analyzers.common
> and lucene.core
>     [javac]
> /home/vagrant/rpmbuild/BUILD/derby-10.15.2.0/db-derby-10.15.2.0-src/java/org.apache.derby.optionaltools/module-info.java:60:
> error: module org.apache.derby.optionaltools reads package
> org.apache.lucene.analysis.standard from both lucene.analyzers.common
> and lucene.core
>     [javac] module org.apache.derby.optionaltools
>     [javac] ^
>     [javac] 6 errors
>
> BUILD FAILED
> /home/vagrant/rpmbuild/BUILD/derby-10.15.2.0/db-derby-10.15.2.0-src/build.xml:1082:
> Compile failed; see the compiler error output for details.

Which is probably because I messed up something related to path or
definition of where java should look for dependancies.

Anybody willing to give me a clue or two on how to fix this ?

Tia

Ludovic

[Jeremy Whiting]

You are seeing the Java Module system report a module access issue. This is with 10.15.2.0 version of Derby. This is a module definition mistake and should be fixed by the project developers.
 I've tried a build with the most recent code on the 10.15 branch. This issue you were seeing has been fixed. Now it's 10.15.2.1
 Try building again using the latest code.

[Fabio Valentini]

On Fri, Aug 14, 2020 at 2:37 PM Ludovic Hirlimann  wrote:
> Hi,
> I'm ludo - I'd like to package josm - which requires Apcahe derby and apache-common-jcs-*.

Hi Ludovic,

I just would like to warn you, josm isn't easy to package (again),
since a lot of the dependencies have been removed from fedora.
You might want to look at the previous efforts to introduce it again
(e.g. jjelen tried to keep it working).

Another problem seems to be that some of the josm dependencies
themselves are pretty outdated and not necessarily with either Java 11
or newer versions of their dependencies, which are in fedora now.
Additionally, some of those projects are built with gradle, which is
no longer available for packaging Java projects in fedora. :(

Fabio

[Ludovic Hirlimann]

On 8/16/20 10:09 AM, Fabio Valentini wrote:
> On Fri, Aug 14, 2020 at 2:37 PM Ludovic Hirlimann  wrote:
> > Hi,
> > I'm ludo - I'd like to package josm - which requires Apcahe derby and apache-common-jcs-*.
>
> Hi Ludovic,
>
> I just would like to warn you, josm isn't easy to package (again),
> since a lot of the dependencies have been removed from fedora.
> You might want to look at the previous efforts to introduce it again
> (e.g. jjelen tried to keep it working).

He told me already and pointer out to this list to ask for help.
> Another problem seems to be that some of the josm dependencies
> themselves are pretty outdated and not necessarily with either Java 11
> or newer versions of their dependencies, which are in fedora now.
> Additionally, some of those projects are built with gradle, which is
> no longer available for packaging Java projects in fedora. :(

I have not ran into that. I'll poke the josm people if I need to on that
one. And see if we can simplify things :p

Thought I doubt.

Ludovic