3:19 a.m.
First off I would like to say that I'm very impressed by the effort you
have put into java packaging over the last few years. Getting a usable
environment for offline maven execution is no small feat.
One thing I noted, however, was that the unit tests for the guava library
was not run on package build. Which seems like a reasonable compromise but
not ideal. I decided to look into what would be needed dependency wise for
that to happen, and I came up with the following:
* updates-testing needs to be activated for the auto-value dependency.
* packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from
the Java Packaging HOWTO
<https://fedorahosted.org/released/javapackages/doc/>;, the packages (SRPMS
and mock built noarch.rpms) are available from
https://resare.com/noa-fedora-playground/repo/22 but please beware of
newbie packaging mistakes :)
* With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new
java8 behaviour, and I back ported a fix from guava master. The changes
needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git
at https://github.com/nresare/fedora-guava in the testlib-tests branch. The
actual commit can be viewed here:
https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806...
I would love for my work to be of use in the fedora project and I'm willing
to spend some time to land my contributions at this time. I will now read
up on what I need to to contribute more formally.
/noa
--
Spotify Free Software ombudsman. I/O Tribe.
4:33 a.m.
On 4 June 2015 at 09:19, Noa Resare <noa(a)spotify.com> wrote:
First off I would like to say that I'm very impressed by the
effort you
have put into java packaging over the last few years. Getting a usable
environment for offline maven execution is no small feat.
One thing I noted, however, was that the unit tests for the guava library
was not run on package build. Which seems like a reasonable compromise but
not ideal. I decided to look into what would be needed dependency wise for
that to happen, and I came up with the following:
* updates-testing needs to be activated for the auto-value dependency.
* packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from
the Java Packaging HOWTO
<https://fedorahosted.org/released/javapackages/doc/>;, the packages
(SRPMS and mock built noarch.rpms) are available from
https://resare.com/noa-fedora-playground/repo/22 but please beware of
newbie packaging mistakes :)
* With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new
java8 behaviour, and I back ported a fix from guava master. The changes
needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git
at https://github.com/nresare/fedora-guava in the testlib-tests branch.
The actual commit can be viewed here:
https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806...
I would love for my work to be of use in the fedora project and I'm
willing to spend some time to land my contributions at this time. I will
now read up on what I need to to contribute more formally.
/noa
--
Spotify Free Software ombudsman. I/O Tribe.
--
java-devel mailing list
java-devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/java-devel
This is great to hear -- it sounds like useful work you have done and I
look forward to seeing your new packages appear in Fedora :-)
I assume you have seen this document already:
https://fedoraproject.org/wiki/Join_the_package_collection_maintainers?rd...
--
Mat Booth
http://fedoraproject.org/get-fedora
4:15 p.m.
On Thu, Jun 4, 2015 at 11:33 AM, Mat Booth <fedora(a)matbooth.co.uk> wrote:
On 4 June 2015 at 09:19, Noa Resare <noa(a)spotify.com> wrote:
> First off I would like to say that I'm very impressed by the effort you
> have put into java packaging over the last few years. Getting a usable
> environment for offline maven execution is no small feat.
>
> One thing I noted, however, was that the unit tests for the guava library
> was not run on package build. Which seems like a reasonable compromise but
> not ideal. I decided to look into what would be needed dependency wise for
> that to happen, and I came up with the following:
>
> * updates-testing needs to be activated for the auto-value dependency.
>
> * packages truth, allocation-instrumenter and caliper needs to be
> introduced. I have created those packages and built them with guidance from
> the Java Packaging HOWTO
> <https://fedorahosted.org/released/javapackages/doc/>;, the packages
> (SRPMS and mock built noarch.rpms) are available from
> https://resare.com/noa-fedora-playground/repo/22 but please beware of
> newbie packaging mistakes :)
>
> * With the dependencies handled I updated guava.spec to re-enable
> guava-testlib and guava-tests. One test was consistently failing due to new
> java8 behaviour, and I back ported a fix from guava master. The changes
> needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git
> at https://github.com/nresare/fedora-guava in the testlib-tests branch.
> The actual commit can be viewed here:
>
https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806...
>
> I would love for my work to be of use in the fedora project and I'm
> willing to spend some time to land my contributions at this time. I will
> now read up on what I need to to contribute more formally.
>
> /noa
>
> --
> Spotify Free Software ombudsman. I/O Tribe.
>
> --
> java-devel mailing list
> java-devel(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/java-devel
>
This is great to hear -- it sounds like useful work you have done and I
look forward to seeing your new packages appear in Fedora :-)
I assume you have seen this document already:
https://fedoraproject.org/wiki/Join_the_package_collection_maintainers?rd...
Thanks!
I hit a slight snag in my progression to publish the depenencies. It turns
out that one of the dependencies, caliper doesn't build in rawhide because
of a dependency on jersey-client:1.11. There is a compatibility package in
rawhide named jersey1 which provides jersey-client:1.19 but it seems like
the "fuzzy artefact resolution" going on works differently when depending
on compat packages and there needs to be an exact version match (updating
the dependency version to something jersey1 provides, such as 1.19 or 1
makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11 to
1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have
potential to be "interesting" when some other code has it's more specific
dependency on jersey fulfilled transitively.
Another quick question: attempting to use the same copr project for both a
released version of fedora and rawhide doesn't make a lot of sense, in the
case of java with fast moving and complex dependencies, right? I assume
after having run into this that I would want one copr project per target so
that I can let the packages diverge.
/noa
--
Spotify Free Software ombudsman. I/O Tribe.
10:05 p.m.
Il 04/06/2015 23:15, Noa Resare ha scritto:
Thanks!
I hit a slight snag in my progression to publish the depenencies. It
turns out that one of the dependencies, caliper doesn't build in
rawhide because of a dependency on jersey-client:1.11. There is a
compatibility package in rawhide named jersey1 which provides
jersey-client:1.19 but it seems like the "fuzzy artefact resolution"
going on works differently when depending on compat packages and there
needs to be an exact version match (updating the dependency version to
something jersey1 provides, such as 1.19 or 1 makes the package build.)
What is the best cause of action here? A, flip the dependency from
1.11 to 1.19. B, flip the dependency from 1.11 to 1. Any of those
seems to have potential to be "interesting" when some other code has
it's more specific dependency on jersey fulfilled transitively.
Another quick question: attempting to use the same copr project for
both a released version of fedora and rawhide doesn't make a lot of
sense, in the case of java with fast moving and complex dependencies,
right? I assume after having run into this that I would want one copr
project per target so that I can let the packages diverge.
/noa
hi
it should work both ways. but in case of an upgrade of jersey1 package
you must change the version references
an example ho manage a compat package:
%pom_xpath_set "pom:dependency[pom:groupId =
'com.sun.jersey']/pom:version" 1
or
%pom_xpath_set "pom:dependency[pom:groupId =
'com.sun.jersey']/pom:version" 1.19
the same should be done for hibernate-core
regards
gil
1:06 a.m.
On 06/04/2015 10:19 AM, Noa Resare wrote:
First off I would like to say that I'm very impressed by the
effort you
have put into java packaging over the last few years. Getting a usable
environment for offline maven execution is no small feat.
Thank you.
One thing I noted, however, was that the unit tests for the guava
library
was not run on package build. Which seems like a reasonable compromise but
not ideal. I decided to look into what would be needed dependency wise for
that to happen, and I came up with the following:
* updates-testing needs to be activated for the auto-value dependency.
Buildroot overrides are useful in such cases. See:
https://fedoraproject.org/wiki/Bodhi/BuildRootOverrides
* packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from
the Java Packaging HOWTO
<https://fedorahosted.org/released/javapackages/doc/>;, the packages (SRPMS
and mock built noarch.rpms) are available from
https://resare.com/noa-fedora-playground/repo/22 but please beware of
newbie packaging mistakes :)
Great. We will review them once you submit review requests.
* With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new
java8 behaviour, and I back ported a fix from guava master. The changes
needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git
at https://github.com/nresare/fedora-guava in the testlib-tests branch. The
actual commit can be viewed here:
https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806...
Overally it looks good. I've added a few comments at Github. I'll be
happy to merge this into Fedora once all dependencies are available in
rawhide.
I would love for my work to be of use in the fedora project and
I'm willing
to spend some time to land my contributions at this time. I will now read
up on what I need to to contribute more formally.
It's really good to see a new contributor. Don't hesitate to ask if you
have any questions or need help.
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
1:30 a.m.
On 06/04/2015 11:15 PM, Noa Resare wrote:
I hit a slight snag in my progression to publish the depenencies. It
turns
out that one of the dependencies, caliper doesn't build in rawhide because
of a dependency on jersey-client:1.11. There is a compatibility package in
rawhide named jersey1 which provides jersey-client:1.19 but it seems like
the "fuzzy artefact resolution" going on works differently when depending
on compat packages and there needs to be an exact version match (updating
the dependency version to something jersey1 provides, such as 1.19 or 1
makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11 to
1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have
potential to be "interesting" when some other code has it's more specific
dependency on jersey fulfilled transitively.
IMHO compat packages should be used or introduced only as last resort,
mostly because they prevent innovation and force us to waste our
precious time on maintaining legacy stuff. See also [1].
In general, the workflow I use:
Try to build the package with latest dependency version;
if (there are compilation or other problems) {
Check latest upstream development branch;
if (upstream already migrated to later dep version) {
Backport upstream patch and include in the spec;
} else {
try {
Port package to newer dep version;
Include patch in spec file;
Forward patch upstream;
} catch (porting not feasible) {
Ask upstream about updating to new dep version;
Give them some time to react;
if (upstream not responding || package is needed urgently) {
Use compat package as dependency;
Periodically check upstream status of dependency;
Try to remove dep on compat package as soon as it is possible;
}
}
}
}
Sorry for pseudo-code, I hope that this is readable :)
If after reading the above you really want to use compat package, then
indeed you need to use exact dependency version. Better version matching
for compat packages has been on my todo [2] for long time, but as you
probably noticed, compat packages are not top priority for me :)
Another quick question: attempting to use the same copr project for
both a
released version of fedora and rawhide doesn't make a lot of sense, in the
case of java with fast moving and complex dependencies, right? I assume
after having run into this that I would want one copr project per target so
that I can let the packages diverge.
Assuming that the goal is to eventually contribute these packages to
Fedora, I would advice to focus on rawhide only for development. Once
rawhide package is done you can consider porting it to older Fedora (or
EPEL) versions.
You don't need to install rawhide to target it. If used correctly, mock
tool can be very useful in building rawhide packages on other systems.
[1]
https://lists.fedoraproject.org/pipermail/java-devel/2013-October/005000....
[2] https://github.com/mizdebsk/xmvn/blob/master/TODO#L50-L51
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
8:27 a.m.
Thank you for all the useful feedback. In the latest few days, I have
looked quite closely at the packages I prepared and concluded that the
guava change to enable testlib and unit tests can be made much smaller if
the caliper dependency is simply disabled.
The dependency is not used by the default maven build (caliper is a
benchmarking framework and benchmarking, while useful, is not part of the
code shipping or testing)
This means that we are down to a single extra dependency to be able to
enable guava testing, the truth package.
I have submitted a review request and I'm looking for sponsors:
https://bugzilla.redhat.com/show_bug.cgi?id=1229704
On Fri, Jun 5, 2015 at 8:30 AM, Mikolaj Izdebski <mizdebsk(a)redhat.com>
wrote:
On 06/04/2015 11:15 PM, Noa Resare wrote:
> I hit a slight snag in my progression to publish the depenencies. It
turns
> out that one of the dependencies, caliper doesn't build in rawhide
because
> of a dependency on jersey-client:1.11. There is a compatibility package
in
> rawhide named jersey1 which provides jersey-client:1.19 but it seems like
> the "fuzzy artefact resolution" going on works differently when depending
> on compat packages and there needs to be an exact version match (updating
> the dependency version to something jersey1 provides, such as 1.19 or 1
> makes the package build.)
>
> What is the best cause of action here? A, flip the dependency from 1.11
to
> 1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have
> potential to be "interesting" when some other code has it's more
specific
> dependency on jersey fulfilled transitively.
IMHO compat packages should be used or introduced only as last resort,
mostly because they prevent innovation and force us to waste our
precious time on maintaining legacy stuff. See also [1].
In general, the workflow I use:
Try to build the package with latest dependency version;
if (there are compilation or other problems) {
Check latest upstream development branch;
if (upstream already migrated to later dep version) {
Backport upstream patch and include in the spec;
} else {
try {
Port package to newer dep version;
Include patch in spec file;
Forward patch upstream;
} catch (porting not feasible) {
Ask upstream about updating to new dep version;
Give them some time to react;
if (upstream not responding || package is needed urgently) {
Use compat package as dependency;
Periodically check upstream status of dependency;
Try to remove dep on compat package as soon as it is possible;
}
}
}
}
Sorry for pseudo-code, I hope that this is readable :)
If after reading the above you really want to use compat package, then
indeed you need to use exact dependency version. Better version matching
for compat packages has been on my todo [2] for long time, but as you
probably noticed, compat packages are not top priority for me :)
> Another quick question: attempting to use the same copr project for both
a
> released version of fedora and rawhide doesn't make a lot of sense, in
the
> case of java with fast moving and complex dependencies, right? I assume
> after having run into this that I would want one copr project per target
so
> that I can let the packages diverge.
Assuming that the goal is to eventually contribute these packages to
Fedora, I would advice to focus on rawhide only for development. Once
rawhide package is done you can consider porting it to older Fedora (or
EPEL) versions.
You don't need to install rawhide to target it. If used correctly, mock
tool can be very useful in building rawhide packages on other systems.
[1]
https://lists.fedoraproject.org/pipermail/java-devel/2013-October/005000....
[2] https://github.com/mizdebsk/xmvn/blob/master/TODO#L50-L51
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
--
java-devel mailing list
java-devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/java-devel
--
Spotify Free Software ombudsman. I/O Tribe.
8:40 a.m.
On 06/09/2015 03:27 PM, Noa Resare wrote:
Thank you for all the useful feedback. In the latest few days, I
have
looked quite closely at the packages I prepared and concluded that the
guava change to enable testlib and unit tests can be made much smaller if
the caliper dependency is simply disabled.
The dependency is not used by the default maven build (caliper is a
benchmarking framework and benchmarking, while useful, is not part of the
code shipping or testing)
This means that we are down to a single extra dependency to be able to
enable guava testing, the truth package.
That's nice.
I have submitted a review request and I'm looking for sponsors:
https://bugzilla.redhat.com/show_bug.cgi?id=1229704
I've taken the review.
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
10:29 a.m.
Since the truth package landed in rawhide a few days ago now (and I have
been busy moving cross Sweden) I thought it might be time to revisit the
guava change. I have updated the
https://github.com/nresare/fedora-guava/tree/testlib-tests branch with your
feedback.
Somewhat troubling, trying to create a koji scratch build (
http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was really
slow and now seem to have stalled indefinitely since a thread died
in TreeMultisetTest with OutOfMemoryError. I guess the build server farm is
somewhat overloaded at the moment, but it is unfortunate if unit tests die
when low on memory.
I hope someone else has some insight into this as the whole build
infrastructure is pretty opaque to me at the moment.
n
On Tue, Jun 9, 2015 at 3:40 PM, Mikolaj Izdebski <mizdebsk(a)redhat.com>
wrote:
On 06/09/2015 03:27 PM, Noa Resare wrote:
> Thank you for all the useful feedback. In the latest few days, I have
> looked quite closely at the packages I prepared and concluded that the
> guava change to enable testlib and unit tests can be made much smaller if
> the caliper dependency is simply disabled.
>
> The dependency is not used by the default maven build (caliper is a
> benchmarking framework and benchmarking, while useful, is not part of the
> code shipping or testing)
>
> This means that we are down to a single extra dependency to be able to
> enable guava testing, the truth package.
That's nice.
> I have submitted a review request and I'm looking for sponsors:
> https://bugzilla.redhat.com/show_bug.cgi?id=1229704
I've taken the review.
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
--
java-devel mailing list
java-devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/java-devel
--
Spotify Free Software ombudsman. I/O Tribe.
10:46 a.m.
Il 18/06/2015 17:29, Noa Resare ha scritto:
Since the truth package landed in rawhide a few days ago now (and I
have been busy moving cross Sweden) I thought it might be time to
revisit the guava change. I have updated the
https://github.com/nresare/fedora-guava/tree/testlib-tests branch with
your feedback.
Somewhat troubling, trying to create a koji scratch build
(http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was
really slow and now seem to have stalled indefinitely since a thread
died in TreeMultisetTest with OutOfMemoryError. I guess the build
server farm is somewhat overloaded at the moment, but it is
unfortunate if unit tests die when low on memory.
hi
can you try with other arch builder (maybe is only a problem with ARM
... builder) ?
koji build --scratch --arch-override [preferred arch x86_64 i386]
rawhide [srpm]
regards
- gil
I hope someone else has some insight into this as the whole build
infrastructure is pretty opaque to me at the moment.
n
On Tue, Jun 9, 2015 at 3:40 PM, Mikolaj Izdebski <mizdebsk(a)redhat.com
<mailto:mizdebsk@redhat.com>> wrote:
On 06/09/2015 03:27 PM, Noa Resare wrote:
> Thank you for all the useful feedback. In the latest few days, I
have
> looked quite closely at the packages I prepared and concluded
that the
> guava change to enable testlib and unit tests can be made much
smaller if
> the caliper dependency is simply disabled.
>
> The dependency is not used by the default maven build (caliper is a
> benchmarking framework and benchmarking, while useful, is not
part of the
> code shipping or testing)
>
> This means that we are down to a single extra dependency to be
able to
> enable guava testing, the truth package.
That's nice.
> I have submitted a review request and I'm looking for sponsors:
> https://bugzilla.redhat.com/show_bug.cgi?id=1229704
I've taken the review.
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
--
java-devel mailing list
java-devel(a)lists.fedoraproject.org
<mailto:java-devel@lists.fedoraproject.org>
https://admin.fedoraproject.org/mailman/listinfo/java-devel
--
Spotify Free Software ombudsman. I/O Tribe.
--
java-devel mailing list
java-devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/java-devel
2:14 a.m.
On 06/18/2015 05:29 PM, Noa Resare wrote:
Since the truth package landed in rawhide a few days ago now (and I
have
been busy moving cross Sweden) I thought it might be time to revisit the
guava change. I have updated the
https://github.com/nresare/fedora-guava/tree/testlib-tests branch with your
feedback.
I've merged your commits. Thank you for contribution.
Somewhat troubling, trying to create a koji scratch build (
http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was really
slow and now seem to have stalled indefinitely since a thread died
in TreeMultisetTest with OutOfMemoryError. I guess the build server farm is
somewhat overloaded at the moment, but it is unfortunate if unit tests die
when low on memory.
You've most likely hit this because your build was picked up by armv7hl
builder. These builders are pretty slow compared to x86_64 ones.
Moreover, we don't have JIT enabled on this architecture, so everything
runs in interpreted mode. Package builds can be one to three orders of
magnitude slower on armv7hl compared to x86_64 and often fail due to
timeouts or OOM.
For scratch builds you can workaround this by using "--arch-override
x86_64" option. For "real" builds the only known workaround is
resubmitting the build until you get non-ARM builder.
--
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
3229
days inactive
3247
days old
java-devel@lists.fedoraproject.org
10 comments
4 participants
participants (4)
-
gil -
Mat Booth -
Mikolaj Izdebski -
Noa Resare