The Future of the Java Stack (also regarding ELN and RHEL)

[Fabio Valentini]

Hi everybody,

So, after some recent FESCo decisions (no default module streams in
fedora, new Module Policy for fedora and ELN modules), it's time to
ask this question again:

What's the future of the Java Stack in fedora, and by extension, in
ELN (and possibly RHEL)?

For the past ~18 months, the Stewardship SIG has picked up the pieces
that were left after the failed attempt of "modularizing" the Java
stack, and now the "re-founded" Java SIG has taken over this effort.
We've been maintaining the core part of the Java stack (about 200
packages) - first trying to keep it from imploding, and lately, trying
to keep it working and up-to-date (also, Java 11 by default, yay).

However! This has mostly been a one-man-show, with regular
contributions by Mat Booth (whos thankless task is maintaining the
Eclipse stack) and the Dogtag PKI team (thanks guys!), who have lately
been busy doing other things (fixing blocker bugs for F33 and RHEL
8.3).

Looking at the Java packages I own, most of them also wind up in ELN,
so I assume that they're going to end up in RHEL at some point. And
here I'm asking myself the question: Who's going to maintain those 200
Java packages? Because I have seen *zero* people from Red Hat
interacting with the Java stack in fedora in any substantial way in
the past year - and by that I mean more than one commit to a package
they didn't own or sending one PR on src.fp.org.

While working through the *massive* backlog of issues and neglected
package updates in the Java stack, I also went through with multiple
non-responsive maintainer processes, and almost nobody from the
original people who maintained Java packages in fedora are left. Some
of those were Red Hat employees, but are apparently no longer, or have
moved on to work on different things.

Not even the Java modules (javapackages-tools, maven, ant, ...) seem
to be in better shape - none of them have been touched in the last 10
months, except for automated mass rebuild commits. They are now pretty
out of sync with mainstream fedora, and they still only target the
"oldstable" fedora 31 platform. Since javapackages-tools:20190x and
maven:3.5 are also no longer default streams, they are unused (?), and
seem to have been abandoned.

So. Who is actually maintaining the Java stack in RHEL? I can't tell.
Who is maintaining the core build tools (xmvn, maven, ant, etc.)? Why
are they not - or no longer - contributing to fedora packages or
interacting with fedora maintainers? Does Red Hat no longer care about
all their Java products [1]? Can we please increase the bus / lottery
factor (= 1) for Java package maintenance in fedora?

Fabio / decathorpe

PS: I'm not talking about the JDK packages here. They're in good
hands, as far as I can tell.

[1]: JBoss / WildFly were among the first things to be retired during
the modularity calamity, because nobody cared. But what about the
shiny new thing, Quarkus? Will it ever be packaged for fedora?

[Guido Aulisi]

Hi,

>  Il giorno 9 set 2020, alle ore 00:19, Fabio Valentini
> <decathorpe(a)gmail.com&gt; ha scritto:
>  
>  Hi everybody,
>  
>  So, after some recent FESCo decisions (no default module streams in
>  fedora, new Module Policy for fedora and ELN modules), it's time to
>  ask this question again:
>  
>  What's the future of the Java Stack in fedora, and by extension, in
>  ELN (and possibly RHEL)? 
… snip

IMHO we could package only the JDK and let the user use Java software directly from
upstream.
Usually upstream means Apache, which is a trusted source, and Java users are smart enough
to manage the Java packages.
I usuali do so when using maven, hadoop, tomcat, etc.
I think this solution could be valid for any other language, like php, python: packaging
only the base language and anything that is not available in executable formats.

This is my personal opinion, your effort in supporting such huge Java stack has always
been appreciated, also because I find it difficult packaging Java software.

Ciao

Guido
FAS: tartina

[Peter Boy]

>  Am 09.09.2020 um 11:28 schrieb Guido Aulisi
> <guido.aulisi(a)gmail.com&gt;:
>  IMHO we could package only the JDK and let the user use Java software directly from
> upstream.
>  Usually upstream means Apache, which is a trusted source, and Java users are smart enough
> to manage the Java packages.
>  I usuali do so when using maven, hadoop, tomcat, etc. 
Sorry, I think you negligently underestimate the importance of Java packages. Even for a
'simple' Tomcat you need system management infrastructure, log file management,
consistency with other programs and libraries, update management and much more. Neither a
download of an Apache Binary nor maven can help.

Maven is great if you need libraries for development, and perhaps for updating individual
libraries in a running system. 

We urgently need a solution for the listed problems. So in short Manpower.

[Björn Persson]

Guido Aulisi wrote:
>  IMHO we could package only the JDK and let the user use Java software
> directly from upstream.
>  Usually upstream means Apache, which is a trusted source, and Java users are smart enough
> to manage the Java packages.
>  I usuali do so when using maven, hadoop, tomcat, etc.
>  I think this solution could be valid for any other language, like php, python: packaging
> only the base language and anything that is not available in executable formats. 
And how well does that scale?

As a sysadmin I don't normally need to know what language each program
is written in. I use language-specific tools only on programs I'm
developing, not on programs I merely use. Should I periodically run
cpan to check for Perl program updates, pip to check for Python program
updates, npm to check for Javascript program updates, gem to check for
Ruby program updates, and so on and so forth? And then manually check a
bunch of individual upstream websites for updates to programs that
aren't in those language-specific repositories either? No way! I run
"yum update" and get *all* the updates for my system.

Björn Persson

[Stephen John Smoogen]

On Wed, 9 Sep 2020 at 09:37, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se&gt; wrote:

>  Guido Aulisi wrote:
>  > IMHO we could package only the JDK and let the user use Java software
>  directly from upstream.
>  > Usually upstream means Apache, which is a trusted source, and Java users
>  are smart enough to manage the Java packages.
>  > I usuali do so when using maven, hadoop, tomcat, etc.
>  > I think this solution could be valid for any other language, like php,
>  python: packaging only the base language and anything that is not available
>  in executable formats.
>
>  And how well does that scale?
>
>  
It doesn't scale.. but neither does having all those packages owned and
looked at by 1 person. Either people need to start helping or this is the
future. People have instead spent the last decade usually saying 'I need
this but have no idea how to maintain it so you can't get rid of it.' That
has led to the point where the people who were trying to do this made it
into modules to make their lives easier and when that made other people's
lives harder.. deciding that it was an unwinnable game so why participate
any longer.

The cost of free packages is eternal vigilance on their maintenance.



>  As a sysadmin I don't normally need to know what language each
> program
>  is written in. I use language-specific tools only on programs I'm
>  developing, not on programs I merely use. Should I periodically run
>  cpan to check for Perl program updates, pip to check for Python program
>  updates, npm to check for Javascript program updates, gem to check for
>  Ruby program updates, and so on and so forth? And then manually check a
>  bunch of individual upstream websites for updates to programs that
>  aren't in those language-specific repositories either? No way! I run
>  "yum update" and get *all* the updates for my system.
>
>  Björn Persson
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Stephen J Smoogen.

[Aleksandar Kurtakov]

On Wed, Sep 9, 2020 at 5:33 PM Stephen John Smoogen <smooge(a)gmail.com&gt;
wrote:

>  On Wed, 9 Sep 2020 at 09:37, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se&gt; wrote:
>
> > Guido Aulisi wrote:
> > > IMHO we could package only the JDK and let the user use Java software
> > directly from upstream.
> > > Usually upstream means Apache, which is a trusted source, and Java
> > users are smart enough to manage the Java packages.
> > > I usuali do so when using maven, hadoop, tomcat, etc.
> > > I think this solution could be valid for any other language, like php,
> > python: packaging only the base language and anything that is not available
> > in executable formats.
> >
> > And how well does that scale?
> >
> >
>  It doesn't scale.. but neither does having all those packages owned and
>  looked at by 1 person. Either people need to start helping or this is the
>  future. People have instead spent the last decade usually saying 'I need
>  this but have no idea how to maintain it so you can't get rid of it.' That
>  has led to the point where the people who were trying to do this made it
>  into modules to make their lives easier and when that made other people's
>  lives harder.. deciding that it was an unwinnable game so why participate
>  any longer.
>  
Couldn't agree more!!! From big proponent of RPM packaging I went to the
"meh, who cares" group - because I see more and more things coming our way
for maintenance so the choice in front of us is "work on upstream Eclipse"
or "keep Eclipse RPM packages". One can easily guess what wins.


>  The cost of free packages is eternal vigilance on their maintenance.
>
>
>
> > As a sysadmin I don't normally need to know what language each program
> > is written in. I use language-specific tools only on programs I'm
> > developing, not on programs I merely use. Should I periodically run
> > cpan to check for Perl program updates, pip to check for Python program
> > updates, npm to check for Javascript program updates, gem to check for
> > Ruby program updates, and so on and so forth? And then manually check a
> > bunch of individual upstream websites for updates to programs that
> > aren't in those language-specific repositories either? No way! I run
> > "yum update" and get *all* the updates for my system.
> >
> > Björn Persson
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]
> >
>
>
>  --
>  Stephen J Smoogen.
>
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Alexander Kurtakov
Red Hat Eclipse Team

[Mario Torre]

I think there's a difference between libraries and applications though.

I for one think that not having Eclipse packaged in Fedora/RHEL etc.. would
be a big loss, the same goes for Mission Control which we maintain

On Wed, Sep 9, 2020 at 5:00 PM Aleksandar Kurtakov <akurtako(a)redhat.com&gt;
wrote:
>  On Wed, Sep 9, 2020 at 5:33 PM Stephen John Smoogen <smooge(a)gmail.com&gt;
wrote:
> >
> >
> >
> > On Wed, 9 Sep 2020 at 09:37, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se&gt; wrote:
> >>
> >> Guido Aulisi wrote:
> >> > IMHO we could package only the JDK and let the user use Java software
directly from upstream.
> >> > Usually upstream means Apache, which is a trusted
> source, and Java 
users are smart enough to manage the Java packages.
> >> > I usuali do so when using maven, hadoop, tomcat, etc.
> >> > I think this solution could be valid for any other language, like 
php,
python: packaging only the base language and anything that is not
available in executable formats.
> >>
> >> And how well does that scale?
> >>
> >
> > It doesn't scale.. but neither does having all those packages owned and
looked at by 1 person. Either people need to start helping or this is the
future. People have instead spent the last decade usually saying 'I need
this but have no idea how to maintain it so you can't get rid of it.' That
has led to the point where the people who were trying to do this made it
into modules to make their lives easier and when that made other people's
lives harder.. deciding that it was an unwinnable game so why participate
any longer.
>  Couldn't agree more!!! From big proponent of RPM packaging I went to the
"meh, who cares" group - because I see more and more things coming our
way
for maintenance so the choice in front of us is "work on upstream Eclipse"
or "keep Eclipse RPM packages". One can easily guess what wins.
> >
> >
> > The cost of free packages is eternal vigilance on their maintenance.
> >
> >
> >>
> >> As a sysadmin I don't normally need to know what language each program
> >> is written in. I use language-specific tools only on programs I'm
> >> developing, not on programs I merely use. Should I periodically run
> >> cpan to check for Perl program updates, pip to check for Python program
> >> updates, npm to check for Javascript program updates, gem to check for
> >> Ruby program updates, and so on and so forth? And then manually check a
> >> bunch of individual upstream websites for updates to programs that
> >> aren't in those language-specific repositories either? No way! I run
> >> "yum update" and get *all* the updates for my system.
> >>
> >> Björn Persson
> >> _______________________________________________
> >> devel mailing list -- devel(a)lists.fedoraproject.org
> >> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> >> Fedora Code of Conduct: https://docs.fedoraproject.
org/en-US/project/code-of-conduct/
> >> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives: https://lists.fedoraproject.
org/archives/list/devel(a)lists.fedoraproject.org
> >
> >
> >
> > --
> > Stephen J Smoogen.
> >
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://docs.fedoraproject.
org/en-US/project/code-of-conduct/
> > List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.
fedoraproject.org
>  --
>  Alexander Kurtakov
>  Red Hat Eclipse Team
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.
org/en-US/project/code-of-conduct/
>  List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.
fedoraproject.org



-- 
Mario Torre
Associate Manager, Software Engineering
Red Hat GmbH <https://www.redhat.com>
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898


-- 
Mario Torre
Associate Manager, Software Engineering
Red Hat GmbH &lt;https://www.redhat.com&gt;<br>9704 A60C B4BE A8B8 0F30
9205 5D7E 4952 3F65 7898

[Emmanuel Seyman]

* Fabio Valentini [09/09/2020 00:19] :
>                               Can we please increase the bus / lottery
>  factor (= 1) for Java package maintenance in fedora? 
I know that at least one distribution bases its Java stack on Fedora's.
Perhaps someonce could contact their maintainers and encourage them to
contribute to their upstream?

Emmanuel

[Jerry James]

On Tue, Sep 8, 2020 at 4:19 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  However! This has mostly been a one-man-show, with regular
>  contributions by Mat Booth (whos thankless task is maintaining the
>  Eclipse stack) and the Dogtag PKI team (thanks guys!), who have lately
>  been busy doing other things (fixing blocker bugs for F33 and RHEL
>  8.3). 
For those who are willing to help out, is
https://pagure.io/java-maint-sig/issues the place to go to figure out
what needs to be done?  Is there any kind of TODO list or wishlist
elsewhere?

You've mentioned having some kind of script that rebuilds the Java
world to check for issues relating to package updates.  Is that
available to the public?  How long does it take to run (and on what
hardware do you run it)?

Speaking of helping out, if anybody wants mockito 3.x in Fedora, here
is a pull request to do just that:
https://src.fedoraproject.org/rpms/mockito/pull-request/2
-- 
Jerry James
http://www.jamezone.org/

[Alexander Scheel]

On Wed, Sep 9, 2020 at 6:27 PM Jerry James <loganjerry(a)gmail.com&gt; wrote:
>  On Tue, Sep 8, 2020 at 4:19 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  > However! This has mostly been a one-man-show, with regular
>  > contributions by Mat Booth (whos thankless task is maintaining the
>  > Eclipse stack) and the Dogtag PKI team (thanks guys!), who have lately
>  > been busy doing other things (fixing blocker bugs for F33 and RHEL
>  > 8.3).
>
>  For those who are willing to help out, is
>  https://pagure.io/java-maint-sig/issues the place to go to figure out
>  what needs to be done?  Is there any kind of TODO list or wishlist
>  elsewhere? 
That has a good list for current issues. Many friendly folks also hang
out in #fedora-java, where you can feel free to ask questions or lend
a hand if you are looking for something more immediate.

The old Stewardship SIG pages also had some interesting data:

https://fedora-stewardship.github.io/overview/

I'm not sure if Fabio has migrated the backlog anywhere.

>  You've mentioned having some kind of script that rebuilds the
> Java
>  world to check for issues relating to package updates.  Is that
>  available to the public?  How long does it take to run (and on what
>  hardware do you run it)? 
This isn't a real integration test of any sort. We rebuild PRs
(manually-triggered) and most all dependent packages in COPR and see
if the world builds fine. You can find the script in the old
Stewardship SIG repo:

https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/m...

There is also:


HTH,
Alex

>  Speaking of helping out, if anybody wants mockito 3.x in Fedora,
> here
>  is a pull request to do just that:
>  https://src.fedoraproject.org/rpms/mockito/pull-request/2
>  --
>  Jerry James
>  http://www.jamezone.org/
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Fabio Valentini]

On Thu, Sep 10, 2020 at 12:43 AM Alexander Scheel <ascheel(a)redhat.com&gt; wrote:
>  On Wed, Sep 9, 2020 at 6:27 PM Jerry James <loganjerry(a)gmail.com&gt; wrote: 
(snip)

>  > For those who are willing to help out, is
>  > https://pagure.io/java-maint-sig/issues the place to go to figure out
>  > what needs to be done?  Is there any kind of TODO list or wishlist
>  > elsewhere? 
Yeah, those are the high-level tasks I'm tracking.

There's also a kanban board for individual, smaller tasks:
https://teams.fedoraproject.org/project/java-package-maintainer-sig/kanban

And if you're feeling old-fashioned, this is a custom bugzilla search
for SIG bugs:
https://bugzilla.redhat.com/buglist.cgi?bug_status=__open__&classific...

>  That has a good list for current issues. Many friendly folks also
> hang
>  out in #fedora-java, where you can feel free to ask questions or lend
>  a hand if you are looking for something more immediate.
>
>  The old Stewardship SIG pages also had some interesting data:
>
>  https://fedora-stewardship.github.io/overview/
>
>  I'm not sure if Fabio has migrated the backlog anywhere. 
I did not. The data was largely maintained manually, and it was too
big a time sink to keep around.
I see the kanban board as a replacement for the backlog.

>  > You've mentioned having some kind of script that rebuilds
> the Java
>  > world to check for issues relating to package updates.  Is that
>  > available to the public?  How long does it take to run (and on what
>  > hardware do you run it)?
>
>  This isn't a real integration test of any sort. We rebuild PRs
>  (manually-triggered) and most all dependent packages in COPR and see
>  if the world builds fine. You can find the script in the old
>  Stewardship SIG repo:
>
>
> https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/m...
I intend to also push the scripts to the java-maint-sig pagure repo eventually.
I just haven't gotten around to do that yet.

Fabio

> There is also:
>
>
> HTH,
> Alex
>
> > Speaking of helping out, if anybody wants mockito 3.x in Fedora, here
> > is a pull request to do just that:
> > https://src.fedoraproject.org/rpms/mockito/pull-request/2
> > --
> > Jerry James
> > http://www.jamezone.org/
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Miro Hrončok]

On 10. 09. 20 0:25, Jerry James wrote:
>  For those who are willing to help out, is
>  https://pagure.io/java-maint-sig/issues  the place to go to figure out
>  what needs to be done?  Is there any kind of TODO list or wishlist
>  elsewhere? 
There is also:

https://packager.fedorainfracloud.org/java-maint-sig

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Joe Orton]

Hi all,

I'm writing as the Red Hat engineering manager responsible for Maven and 
Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my 
team.  I want to give a broad response to some of the points here:

1.  The team has two missions in Fedora:

a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is 
to provide developers with the most popular Java build systems which are 
reviewed, tested, and updated through the release lifecycle. 

b) We design, develop and document tooling that enables anyone to 
package Java software with a simple, efficient and scalable process. We 
are also active members of Java SIG, collaborating on complex changes 
and guiding new contributors.

2.  We are committed to maintaining the Ant and Maven modules in 
Fedora.  We have always expected to make them available as default 
streams and in the buildroot so they can be available and consumed by 
non-modular packages, but we completely respect the decisions of FESCo 
to disallow default streams and of other contributors to adopt and 
maintain the non-modular packages.  We are not going to promise to 
commit time and resources to maintain the non-modular packages.

3.  Our efforts are currently directed mainly at minimization of the 
dependency tree which leads to maven and ant, automating the process of 
bootstrapping maven and updating related components, so that new 
versions can be imported and built reproducibly and with consistent 
quality.

4.  The benefit we want to preserve from modules is to maintain packages 
with varying expectation of quality, specifically separating the 
build-time-only vs runtime dependencies.  e.g. in that case that a web 
server like Eclipse Jetty is required as a dep for testing another 
component during the build, we want to be able to use and build that 
component, without being indefinitely on the hook for security errata.  
(The build dependency tree is particularly complex for Maven and 
involves many examples of packages with frequent and high severity 
vulnerabilies)

Regards, Joe

[Dominik 'Rathann' Mierzejewski]

On Thursday, 10 September 2020 at 14:50, Joe Orton wrote:
[...]
>  1.  The team has two missions in Fedora:
>  
>  a) We deliver, maintain and support Ant and Maven in Fedora. Our aim
>  is to provide developers with the most popular Java build systems
>  which are reviewed, tested, and updated through the release
>  lifecycle. 
>  
>  b) We design, develop and document tooling that enables anyone to
>  package Java software with a simple, efficient and scalable process.
>  We are also active members of Java SIG, collaborating on complex
>  changes and guiding new contributors.
>  
>  2.  We are committed to maintaining the Ant and Maven modules in
>  Fedora.  We have always expected to make them available as default
>  streams and in the buildroot so they can be available and consumed by
>  non-modular packages, but we completely respect the decisions of FESCo
>  to disallow default streams and of other contributors to adopt and
>  maintain the non-modular packages.  We are not going to promise to
>  commit time and resources to maintain the non-modular packages. 
Points 1b and 2 mean that nobody can consume your packages for
maintaining non-modular Java packages, which raises the bar for
maintaining Java packages considerably.

For example, I wanted to maintain a couple of Java packages, but I
didn't have time to (learn how to) maintain modules. This effectively
means that certain software won't be packaged for Fedora (by me). I
certainly don't have the time to maintain non-modular Ant and Maven
packages in addition to what I actually wanted to maintain. I'm pretty
sure there are quite a few people in a similar position.

[...]
>  4.  The benefit we want to preserve from modules is to maintain
>  packages with varying expectation of quality, specifically separating
>  the build-time-only vs runtime dependencies.  e.g. in that case that a
>  web server like Eclipse Jetty is required as a dep for testing another
>  component during the build, we want to be able to use and build that
>  component, without being indefinitely on the hook for security
>  errata.  (The build dependency tree is particularly complex for Maven
>  and involves many examples of packages with frequent and high severity
>  vulnerabilies) 
On one hand, that's understandable, but you're effectively maintaining
those packages anyway. And I don't see any reason why you couldn't do
that in the traditional non-modular way, which makes it easier for
others to step in and co-maintain. I'm not sure what you mean by
"indefinitely on the hook for security errata". Can you elaborate?

Regards,
Dominik
-- 
Fedora   https://getfedora.org  |  RPM Fusion  http://rpmfusion.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
        -- from "Collected Sayings of Muad'Dib" by the Princess Irulan

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 3:29 PM Dominik 'Rathann' Mierzejewski
<dominik(a)greysector.net&gt; wrote:
>  On Thursday, 10 September 2020 at 14:50, Joe Orton wrote:
>  [...]
>  > 1.  The team has two missions in Fedora:
>  >
>  > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim
>  > is to provide developers with the most popular Java build systems
>  > which are reviewed, tested, and updated through the release
>  > lifecycle.
>  >
>  > b) We design, develop and document tooling that enables anyone to
>  > package Java software with a simple, efficient and scalable process.
>  > We are also active members of Java SIG, collaborating on complex
>  > changes and guiding new contributors.
>  >
>  > 2.  We are committed to maintaining the Ant and Maven modules in
>  > Fedora.  We have always expected to make them available as default
>  > streams and in the buildroot so they can be available and consumed by
>  > non-modular packages, but we completely respect the decisions of FESCo
>  > to disallow default streams and of other contributors to adopt and
>  > maintain the non-modular packages.  We are not going to promise to
>  > commit time and resources to maintain the non-modular packages.
>
>  Points 1b and 2 mean that nobody can consume your packages for
>  maintaining non-modular Java packages, which raises the bar for
>  maintaining Java packages considerably. 
Javapackages is the main project that provides Java packaging tooling.
I am maintaining it both upstream [1] and in Fedora, also as ursine
package [2].

>  [...]
>  > 4.  The benefit we want to preserve from modules is to maintain
>  > packages with varying expectation of quality, specifically separating
>  > the build-time-only vs runtime dependencies.  e.g. in that case that a
>  > web server like Eclipse Jetty is required as a dep for testing another
>  > component during the build, we want to be able to use and build that
>  > component, without being indefinitely on the hook for security
>  > errata.  (The build dependency tree is particularly complex for Maven
>  > and involves many examples of packages with frequent and high severity
>  > vulnerabilies)
>
>  On one hand, that's understandable, but you're effectively maintaining
>  those packages anyway. And I don't see any reason why you couldn't do
>  that in the traditional non-modular way, which makes it easier for
>  others to step in and co-maintain. I'm not sure what you mean by
>  "indefinitely on the hook for security errata". Can you elaborate? 
All Fedora packages shipped to users are expected to be high-quality
and well maintained, which may impose a very significant burden on
maintainers.
But for build-only packages (packages are used only during build, not
shipped to users), many of package maintainer responsibilities [3] can
be reduced to almost nothing. For example "manage security issues"
means triaging them and fixing only very small subset of them (since
user can't install the packages, most of vulnerabilities are not
exploitable), "deal with reported bugs" means closing them NOTABUG as
users can't possibly install the package in a "supported" way,  "track
dependency issues" duty is reduced as nothing non-modular can depend
on the package, and so on.

[1] https://github.com/fedora-java/javapackages
[2] https://src.fedoraproject.org/rpms/javapackages-tools
[3] https://docs.fedoraproject.org/en-US/fesco/Package_maintainer_responsibil...

>  Regards,
>  Dominik
>  --
>  Fedora   https://getfedora.org  |  RPM Fusion  http://rpmfusion.org
>  There should be a science of discontent. People need hard times and
>  oppression to develop psychic muscles.
>          -- from "Collected Sayings of Muad'Dib" by the Princess Irulan
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Fabio Valentini]

On Thu, Sep 10, 2020 at 4:23 PM Mikolaj Izdebski <mizdebsk(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 3:29 PM Dominik 'Rathann' Mierzejewski
>  <dominik(a)greysector.net&gt; wrote:
>  >
>  > On Thursday, 10 September 2020 at 14:50, Joe Orton wrote:
>  > [...]
>  > > 1.  The team has two missions in Fedora:
>  > >
>  > > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim
>  > > is to provide developers with the most popular Java build systems
>  > > which are reviewed, tested, and updated through the release
>  > > lifecycle. 
You deliver Ant and Maven to nobody. The modules are essentially
unused and / or useless.

>  > > b) We design, develop and document tooling that enables
> anyone to
>  > > package Java software with a simple, efficient and scalable process.
>  > > We are also active members of Java SIG, collaborating on complex
>  > > changes and guiding new contributors. 
I don't know which Java SIG you're talking about, because in the past
two years, I've seen none of this.

>  > > 2.  We are committed to maintaining the Ant and Maven
> modules in
>  > > Fedora.  We have always expected to make them available as default
>  > > streams and in the buildroot so they can be available and consumed by
>  > > non-modular packages, but we completely respect the decisions of FESCo
>  > > to disallow default streams and of other contributors to adopt and
>  > > maintain the non-modular packages.  We are not going to promise to
>  > > commit time and resources to maintain the non-modular packages. 
So, you respect FESCo decisions but you choose to ignore the
consequences? m'kay.

>  > Points 1b and 2 mean that nobody can consume your packages for
>  > maintaining non-modular Java packages, which raises the bar for
>  > maintaining Java packages considerably.
>
>  Javapackages is the main project that provides Java packaging tooling.
>  I am maintaining it both upstream [1] and in Fedora, also as ursine
>  package [2]. 
It's also the *only* non-modular package you still maintain.
However, the only changes you made in the past two years were 1)
enabling xmvn-javadoc upon my request and 2) switching the default
Java to OpenJDK 11 opon Jiri Vanek's request ...

>  > [...]
>  > > 4.  The benefit we want to preserve from modules is to maintain
>  > > packages with varying expectation of quality, specifically separating
>  > > the build-time-only vs runtime dependencies.  e.g. in that case that a
>  > > web server like Eclipse Jetty is required as a dep for testing another
>  > > component during the build, we want to be able to use and build that
>  > > component, without being indefinitely on the hook for security
>  > > errata.  (The build dependency tree is particularly complex for Maven
>  > > and involves many examples of packages with frequent and high severity
>  > > vulnerabilies) 
This is one of the *worst* aspects of Modularity - being able to
"hide" implementation details from re-use by others.
I would not see this as a benefit, but as a big issue when it comes to
the bigger distribution and collaboration context.

>  > On one hand, that's understandable, but you're
> effectively maintaining
>  > those packages anyway. And I don't see any reason why you couldn't do
>  > that in the traditional non-modular way, which makes it easier for
>  > others to step in and co-maintain. I'm not sure what you mean by
>  > "indefinitely on the hook for security errata". Can you elaborate?
>
>  All Fedora packages shipped to users are expected to be high-quality
>  and well maintained, which may impose a very significant burden on
>  maintainers.
>  But for build-only packages (packages are used only during build, not
>  shipped to users), many of package maintainer responsibilities [3] can
>  be reduced to almost nothing. For example "manage security issues"
>  means triaging them and fixing only very small subset of them (since
>  user can't install the packages, most of vulnerabilities are not
>  exploitable), "deal with reported bugs" means closing them NOTABUG as
>  users can't possibly install the package in a "supported" way, 
> "track
>  dependency issues" duty is reduced as nothing non-modular can depend
>  on the package, and so on. 
Well, I can't even tell where (or if) you've been working on this
stuff at all for the past 10 months.
I find no commits in the modules/javapackages-tools module, nor in
javapackages-tools-201902 branches, nor in the maven-3.6 branches more
recently than 10 months ago.
The most recent builds I can find for you and mkoncek are still in 2019 (!).

So please tell me, am I imagining things, or have you actually not
contributed anything to fedora packages in the last 10 months?

Fabio

[Daniel P. Berrangé]

On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:

>  4.  The benefit we want to preserve from modules is to maintain
> packages 
>  with varying expectation of quality, specifically separating the 
>  build-time-only vs runtime dependencies.  e.g. in that case that a web 
>  server like Eclipse Jetty is required as a dep for testing another 
>  component during the build, we want to be able to use and build that 
>  component, without being indefinitely on the hook for security errata.  
>  (The build dependency tree is particularly complex for Maven and 
>  involves many examples of packages with frequent and high severity 
>  vulnerabilies) 
What are you doing different in terms of supporting deps in the module
that reduces the security errata burden, compared to non-modular builds ?

It feels like if we have some policy that is creating unsustainable
maint burden wrt non-modular packaging, we should re-examine this
policy rather than trying to workaround it by going modular, which
creates a different kind of maint burden.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Petr Pisar]

On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  
>  > 4.  The benefit we want to preserve from modules is to maintain packages 
>  > with varying expectation of quality, specifically separating the 
>  > build-time-only vs runtime dependencies.  e.g. in that case that a web 
>  > server like Eclipse Jetty is required as a dep for testing another 
>  > component during the build, we want to be able to use and build that 
>  > component, without being indefinitely on the hook for security errata.  
>  > (The build dependency tree is particularly complex for Maven and 
>  > involves many examples of packages with frequent and high severity 
>  > vulnerabilies)
>  
>  What are you doing different in terms of supporting deps in the module
>  that reduces the security errata burden, compared to non-modular builds ?
>  
>  It feels like if we have some policy that is creating unsustainable
>  maint burden wrt non-modular packaging, we should re-examine this
>  policy rather than trying to workaround it by going modular, which
>  creates a different kind of maint burden.
>   
In non-modular Fedora all packages that we have in Fedora build system (Koji)
are tagged into Fedora repositories and made available to all users on their
computers for any purpose. That implies that all packages in Fedora build system
must be fully supported including addressing all security issues.

In modular Fedora that's (effectively) not true. Packages that only exist
for the sake of building other packages (i.e. build-only dependencies) can be
retained in the Fedora build system and never left it. That means those
packages are never made available to Fedora users and thus a service level for
them is significantly lower. E.g. no security fixes, not bug fixes, no
integration, not tests, no API/ABI stability. The only requirement is that
they can be built and used for building other packages.

I wrote that it was not effectively true. There is probably no such policy
that would de jure allowed lowering the service level for the build-only
packages. But at the same time there is nobody who could enforce it. Users do
not have the packages, security team does know about them, they cannot break
a compose, and they do not intefere with packages from other modules. The only
place where they are visible is dist-git and Koji. Thus they only need to pass
a review (a legal requirement).

-- Petr

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 4:04 PM Petr Pisar <ppisar(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  >
>  > > 4.  The benefit we want to preserve from modules is to maintain packages
>  > > with varying expectation of quality, specifically separating the
>  > > build-time-only vs runtime dependencies.  e.g. in that case that a web
>  > > server like Eclipse Jetty is required as a dep for testing another
>  > > component during the build, we want to be able to use and build that
>  > > component, without being indefinitely on the hook for security errata.
>  > > (The build dependency tree is particularly complex for Maven and
>  > > involves many examples of packages with frequent and high severity
>  > > vulnerabilies)
>  >
>  > What are you doing different in terms of supporting deps in the module
>  > that reduces the security errata burden, compared to non-modular builds ?
>  >
>  > It feels like if we have some policy that is creating unsustainable
>  > maint burden wrt non-modular packaging, we should re-examine this
>  > policy rather than trying to workaround it by going modular, which
>  > creates a different kind of maint burden.
>  >
>  In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  are tagged into Fedora repositories and made available to all users on their
>  computers for any purpose. That implies that all packages in Fedora build system
>  must be fully supported including addressing all security issues.
>
>  In modular Fedora that's (effectively) not true. Packages that only exist
>  for the sake of building other packages (i.e. build-only dependencies) can be
>  retained in the Fedora build system and never left it. That means those
>  packages are never made available to Fedora users and thus a service level for
>  them is significantly lower. E.g. no security fixes, not bug fixes, no
>  integration, not tests, no API/ABI stability. The only requirement is that
>  they can be built and used for building other packages.
>
>  I wrote that it was not effectively true. There is probably no such policy
>  that would de jure allowed lowering the service level for the build-only
>  packages. But at the same time there is nobody who could enforce it. Users do
>  not have the packages, security team does know about them, they cannot break
>  a compose, and they do not intefere with packages from other modules. The only
>  place where they are visible is dist-git and Koji. Thus they only need to pass
>  a review (a legal requirement). 
+1
That is very well put, thanks Petr for explaining it in detail.

>  -- Petr
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Tomasz Torcz]

On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  In non-modular Fedora all packages that we have in Fedora build
> system (Koji)
>  are tagged into Fedora repositories and made available to all users on their
>  computers for any purpose. That implies that all packages in Fedora build system
>  must be fully supported including addressing all security issues.
>  
>  In modular Fedora that's (effectively) not true. Packages that only exist
>  for the sake of building other packages (i.e. build-only dependencies) can be
>  retained in the Fedora build system and never left it. That means those
>  packages are never made available to Fedora users and thus a service level for
>  them is significantly lower. E.g. no security fixes, not bug fixes, no
>  integration, not tests, no API/ABI stability. The only requirement is that
>  they can be built and used for building other packages. 

  So, if user wants to locally rebuild the package, they won't be able to
do it? Because BuildRequired packages won't be available?


-- 
Tomasz Torcz                “Funeral in the morning, IDE hacking
tomek(a)pipebreaker.pl         in the afternoon and evening.” - Alan Cox

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 4:35 PM Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:
>  On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  > are tagged into Fedora repositories and made available to all users on their
>  > computers for any purpose. That implies that all packages in Fedora build system
>  > must be fully supported including addressing all security issues.
>  >
>  > In modular Fedora that's (effectively) not true. Packages that only exist
>  > for the sake of building other packages (i.e. build-only dependencies) can be
>  > retained in the Fedora build system and never left it. That means those
>  > packages are never made available to Fedora users and thus a service level for
>  > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > integration, not tests, no API/ABI stability. The only requirement is that
>  > they can be built and used for building other packages.
>
>
>    So, if user wants to locally rebuild the package, they won't be able to
>  do it? Because BuildRequired packages won't be available? 
Modules can be built locally with "fedpkg module-build-local", which
downloads required dependencies from Koji and installs them in mock
chroot. These packages are not installed directly (outsides of chroot)
on users systems.


>  --
>  Tomasz Torcz                “Funeral in the morning, IDE hacking
>  tomek(a)pipebreaker.pl         in the afternoon and evening.” - Alan Cox
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Tomasz Torcz]

On Thu, Sep 10, 2020 at 04:38:04PM +0200, Mikolaj Izdebski wrote:
>  > >
>  > > In modular Fedora that's (effectively) not true. Packages that only exist
>  > > for the sake of building other packages (i.e. build-only dependencies) can be
>  > > retained in the Fedora build system and never left it. That means those
>  > > packages are never made available to Fedora users and thus a service level for
>  > > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > > integration, not tests, no API/ABI stability. The only requirement is that
>  > > they can be built and used for building other packages.
>  >
>  >
>  >   So, if user wants to locally rebuild the package, they won't be able to
>  > do it? Because BuildRequired packages won't be available?
>  
>  Modules can be built locally with "fedpkg module-build-local", which
>  downloads required dependencies from Koji and installs them in mock
>  chroot. These packages are not installed directly (outsides of chroot)
>  on users systems. 
  OK, cool, thanks for dispeling my doubts.

-- 
Tomasz Torcz                “Funeral in the morning, IDE hacking
tomek(a)pipebreaker.pl         in the afternoon and evening.” - Alan Cox

[Daniel P. Berrangé]

On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > 
>  > > 4.  The benefit we want to preserve from modules is to maintain packages 
>  > > with varying expectation of quality, specifically separating the 
>  > > build-time-only vs runtime dependencies.  e.g. in that case that a web 
>  > > server like Eclipse Jetty is required as a dep for testing another 
>  > > component during the build, we want to be able to use and build that 
>  > > component, without being indefinitely on the hook for security errata.  
>  > > (The build dependency tree is particularly complex for Maven and 
>  > > involves many examples of packages with frequent and high severity 
>  > > vulnerabilies)
>  > 
>  > What are you doing different in terms of supporting deps in the module
>  > that reduces the security errata burden, compared to non-modular builds ?
>  > 
>  > It feels like if we have some policy that is creating unsustainable
>  > maint burden wrt non-modular packaging, we should re-examine this
>  > policy rather than trying to workaround it by going modular, which
>  > creates a different kind of maint burden.
>  > 
>  In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  are tagged into Fedora repositories and made available to all users on their
>  computers for any purpose. That implies that all packages in Fedora build system
>  must be fully supported including addressing all security issues.
>  
>  In modular Fedora that's (effectively) not true. Packages that only exist
>  for the sake of building other packages (i.e. build-only dependencies) can be
>  retained in the Fedora build system and never left it. That means those
>  packages are never made available to Fedora users and thus a service level for
>  them is significantly lower. E.g. no security fixes, not bug fixes, no
>  integration, not tests, no API/ABI stability. The only requirement is that
>  they can be built and used for building other packages. 
So conceptually, one way we can solve this problem by implementing a way
to mark certain non-modular RPMs as "build root only" packages and thus
composing them into a separate "build root" yum repo, that is not enabled
by default except in the build system.

Modularity is being used because it is the only solution that is available
today, not because it is a good/desired solution.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Neal Gompa]

On Thu, Sep 10, 2020 at 1:30 PM Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > >
>  > > > 4.  The benefit we want to preserve from modules is to maintain packages
>  > > > with varying expectation of quality, specifically separating the
>  > > > build-time-only vs runtime dependencies.  e.g. in that case that a web
>  > > > server like Eclipse Jetty is required as a dep for testing another
>  > > > component during the build, we want to be able to use and build that
>  > > > component, without being indefinitely on the hook for security errata.
>  > > > (The build dependency tree is particularly complex for Maven and
>  > > > involves many examples of packages with frequent and high severity
>  > > > vulnerabilies)
>  > >
>  > > What are you doing different in terms of supporting deps in the module
>  > > that reduces the security errata burden, compared to non-modular builds ?
>  > >
>  > > It feels like if we have some policy that is creating unsustainable
>  > > maint burden wrt non-modular packaging, we should re-examine this
>  > > policy rather than trying to workaround it by going modular, which
>  > > creates a different kind of maint burden.
>  > >
>  > In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  > are tagged into Fedora repositories and made available to all users on their
>  > computers for any purpose. That implies that all packages in Fedora build system
>  > must be fully supported including addressing all security issues.
>  >
>  > In modular Fedora that's (effectively) not true. Packages that only exist
>  > for the sake of building other packages (i.e. build-only dependencies) can be
>  > retained in the Fedora build system and never left it. That means those
>  > packages are never made available to Fedora users and thus a service level for
>  > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > integration, not tests, no API/ABI stability. The only requirement is that
>  > they can be built and used for building other packages.
>
>  So conceptually, one way we can solve this problem by implementing a way
>  to mark certain non-modular RPMs as "build root only" packages and thus
>  composing them into a separate "build root" yum repo, that is not enabled
>  by default except in the build system.
>  
This is an anti-feature and I personally will not support any effort
to offer this in Fedora. That is just one step removed from not
shipping it at all to people, and I don't want it to be easy for us to
make that kind of decision.

It *barely* makes sense in RHEL and CentOS and has massively screwed
up things for CentOS SIGs (which are, with the exception of the Virt
SIG, all Red Hat teams) by making it impossible for the *projects* to
build their stuff on CentOS.

>  Modularity is being used because it is the only solution that is
> available
>  today, not because it is a good/desired solution.
>  
Whether modularity is a good or bad solution is beside the point. The
problem here is that there is effectively zero official work in Fedora
by the numerous Java teams at Red Hat. With how much Red Hat is
involved in the Java ecosystem, Fedora should be a freaking paradise
for Java users and developers. It's almost the worst experience by a
fair margin due to lack of development and improvement in the tooling
and infrastructure to support the Java ecosystem.

Even *Rust* is a better experience than Java, and Rust is mostly
managed by Igor (with tiny bits from me, Josh, and a few others).




--
真実はいつも一つ！/ Always, there's only one truth!

[Daniel P. Berrangé]

On Thu, Sep 10, 2020 at 01:37:41PM -0400, Neal Gompa wrote:
>  On Thu, Sep 10, 2020 at 1:30 PM Daniel P. Berrangé
> <berrange(a)redhat.com&gt; wrote:
>  >
>  > On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > > >
>  > > > > 4.  The benefit we want to preserve from modules is to maintain
> packages
>  > > > > with varying expectation of quality, specifically separating the
>  > > > > build-time-only vs runtime dependencies.  e.g. in that case that a
> web
>  > > > > server like Eclipse Jetty is required as a dep for testing another
>  > > > > component during the build, we want to be able to use and build that
>  > > > > component, without being indefinitely on the hook for security
> errata.
>  > > > > (The build dependency tree is particularly complex for Maven and
>  > > > > involves many examples of packages with frequent and high severity
>  > > > > vulnerabilies)
>  > > >
>  > > > What are you doing different in terms of supporting deps in the module
>  > > > that reduces the security errata burden, compared to non-modular builds ?
>  > > >
>  > > > It feels like if we have some policy that is creating unsustainable
>  > > > maint burden wrt non-modular packaging, we should re-examine this
>  > > > policy rather than trying to workaround it by going modular, which
>  > > > creates a different kind of maint burden.
>  > > >
>  > > In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  > > are tagged into Fedora repositories and made available to all users on their
>  > > computers for any purpose. That implies that all packages in Fedora build
> system
>  > > must be fully supported including addressing all security issues.
>  > >
>  > > In modular Fedora that's (effectively) not true. Packages that only exist
>  > > for the sake of building other packages (i.e. build-only dependencies) can be
>  > > retained in the Fedora build system and never left it. That means those
>  > > packages are never made available to Fedora users and thus a service level for
>  > > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > > integration, not tests, no API/ABI stability. The only requirement is that
>  > > they can be built and used for building other packages.
>  >
>  > So conceptually, one way we can solve this problem by implementing a way
>  > to mark certain non-modular RPMs as "build root only" packages and thus
>  > composing them into a separate "build root" yum repo, that is not enabled
>  > by default except in the build system.
>  >
>  
>  This is an anti-feature and I personally will not support any effort
>  to offer this in Fedora. That is just one step removed from not
>  shipping it at all to people, and I don't want it to be easy for us to
>  make that kind of decision. 
We already have this feature in Fedora via Modularity. If it is unacceptable
for Fedora, we shouldn't do it in modules either, while if it is acceptable,
then we should allow it for non-modular content. 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Neal Gompa]

On Thu, Sep 10, 2020 at 2:04 PM Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 01:37:41PM -0400, Neal Gompa wrote:
>  > On Thu, Sep 10, 2020 at 1:30 PM Daniel P. Berrangé <berrange(a)redhat.com&gt;
> wrote:
>  > >
>  > > On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > > > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > > > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > > > >
>  > > > > > 4.  The benefit we want to preserve from modules is to maintain
> packages
>  > > > > > with varying expectation of quality, specifically separating
> the
>  > > > > > build-time-only vs runtime dependencies.  e.g. in that case that
> a web
>  > > > > > server like Eclipse Jetty is required as a dep for testing
> another
>  > > > > > component during the build, we want to be able to use and build
> that
>  > > > > > component, without being indefinitely on the hook for security
> errata.
>  > > > > > (The build dependency tree is particularly complex for Maven
> and
>  > > > > > involves many examples of packages with frequent and high
> severity
>  > > > > > vulnerabilies)
>  > > > >
>  > > > > What are you doing different in terms of supporting deps in the
> module
>  > > > > that reduces the security errata burden, compared to non-modular
> builds ?
>  > > > >
>  > > > > It feels like if we have some policy that is creating unsustainable
>  > > > > maint burden wrt non-modular packaging, we should re-examine this
>  > > > > policy rather than trying to workaround it by going modular, which
>  > > > > creates a different kind of maint burden.
>  > > > >
>  > > > In non-modular Fedora all packages that we have in Fedora build system
> (Koji)
>  > > > are tagged into Fedora repositories and made available to all users on
> their
>  > > > computers for any purpose. That implies that all packages in Fedora build
> system
>  > > > must be fully supported including addressing all security issues.
>  > > >
>  > > > In modular Fedora that's (effectively) not true. Packages that only
> exist
>  > > > for the sake of building other packages (i.e. build-only dependencies) can
> be
>  > > > retained in the Fedora build system and never left it. That means those
>  > > > packages are never made available to Fedora users and thus a service level
> for
>  > > > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > > > integration, not tests, no API/ABI stability. The only requirement is
> that
>  > > > they can be built and used for building other packages.
>  > >
>  > > So conceptually, one way we can solve this problem by implementing a way
>  > > to mark certain non-modular RPMs as "build root only" packages and
> thus
>  > > composing them into a separate "build root" yum repo, that is not
> enabled
>  > > by default except in the build system.
>  > >
>  >
>  > This is an anti-feature and I personally will not support any effort
>  > to offer this in Fedora. That is just one step removed from not
>  > shipping it at all to people, and I don't want it to be easy for us to
>  > make that kind of decision.
>
>  We already have this feature in Fedora via Modularity. If it is unacceptable
>  for Fedora, we shouldn't do it in modules either, while if it is acceptable,
>  then we should allow it for non-modular content. 
It's not allowed there either. You are not supposed to do that, *period*.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 7:31 PM Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > >
>  > > > 4.  The benefit we want to preserve from modules is to maintain packages
>  > > > with varying expectation of quality, specifically separating the
>  > > > build-time-only vs runtime dependencies.  e.g. in that case that a web
>  > > > server like Eclipse Jetty is required as a dep for testing another
>  > > > component during the build, we want to be able to use and build that
>  > > > component, without being indefinitely on the hook for security errata.
>  > > > (The build dependency tree is particularly complex for Maven and
>  > > > involves many examples of packages with frequent and high severity
>  > > > vulnerabilies)
>  > >
>  > > What are you doing different in terms of supporting deps in the module
>  > > that reduces the security errata burden, compared to non-modular builds ?
>  > >
>  > > It feels like if we have some policy that is creating unsustainable
>  > > maint burden wrt non-modular packaging, we should re-examine this
>  > > policy rather than trying to workaround it by going modular, which
>  > > creates a different kind of maint burden.
>  > >
>  > In non-modular Fedora all packages that we have in Fedora build system (Koji)
>  > are tagged into Fedora repositories and made available to all users on their
>  > computers for any purpose. That implies that all packages in Fedora build system
>  > must be fully supported including addressing all security issues.
>  >
>  > In modular Fedora that's (effectively) not true. Packages that only exist
>  > for the sake of building other packages (i.e. build-only dependencies) can be
>  > retained in the Fedora build system and never left it. That means those
>  > packages are never made available to Fedora users and thus a service level for
>  > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > integration, not tests, no API/ABI stability. The only requirement is that
>  > they can be built and used for building other packages.
>
>  So conceptually, one way we can solve this problem by implementing a way
>  to mark certain non-modular RPMs as "build root only" packages and thus
>  composing them into a separate "build root" yum repo, that is not enabled
>  by default except in the build system. 
Yes. This can also be achieved with on-demand side tags that are
already implemented: "build-only" packages are built in a sidetag and
untagged before sidetag is merged. They never appear in release tags
and they are not shipped to users. Builds can be reproduced locally in
mock with configs generated by "koji mock-config" command.

>  Modularity is being used because it is the only solution that is
> available
>  today, not because it is a good/desired solution. 
Right. Modularity is definitely not the best solution, and IMHO it
definitely has worse user experience compared to ursine packages.
Modularity is used because it was the only solution available at the
moment the decision (to modularize Maven and Ant) was made. Since then
an alternative solution was developed, but we haven't decided to
switch back to ursine packages yet, although we are considering that.

>  Regards,
>  Daniel
>  --
>  |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
>  |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
>  |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Adam Williamson]

On Thu, 2020-09-10 at 22:19 +0200, Mikolaj Izdebski wrote:
>  > 
>  > So conceptually, one way we can solve this problem by implementing a way
>  > to mark certain non-modular RPMs as "build root only" packages and thus
>  > composing them into a separate "build root" yum repo, that is not enabled
>  > by default except in the build system.
>  
>  Yes. This can also be achieved with on-demand side tags that are
>  already implemented: "build-only" packages are built in a sidetag and
>  untagged before sidetag is merged. They never appear in release tags
>  and they are not shipped to users. Builds can be reproduced locally in
>  mock with configs generated by "koji mock-config" command. 
I have to say I agree with Neal that for Fedora this is an anti-feature
and should not be done.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Stephen John Smoogen]

On Thu, 10 Sep 2020 at 13:31, Daniel P. Berrangé <berrange(a)redhat.com&gt;
wrote:

>  On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
>  > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
>  > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > >
>  > > > 4.  The benefit we want to preserve from modules is to maintain
>  packages
>  > > > with varying expectation of quality, specifically separating the
>  > > > build-time-only vs runtime dependencies.  e.g. in that case that a
>  web
>  > > > server like Eclipse Jetty is required as a dep for testing another
>  > > > component during the build, we want to be able to use and build that
>  > > > component, without being indefinitely on the hook for security
>  errata.
>  > > > (The build dependency tree is particularly complex for Maven and
>  > > > involves many examples of packages with frequent and high severity
>  > > > vulnerabilies)
>  > >
>  > > What are you doing different in terms of supporting deps in the module
>  > > that reduces the security errata burden, compared to non-modular
>  builds ?
>  > >
>  > > It feels like if we have some policy that is creating unsustainable
>  > > maint burden wrt non-modular packaging, we should re-examine this
>  > > policy rather than trying to workaround it by going modular, which
>  > > creates a different kind of maint burden.
>  > >
>  > In non-modular Fedora all packages that we have in Fedora build system
>  (Koji)
>  > are tagged into Fedora repositories and made available to all users on
>  their
>  > computers for any purpose. That implies that all packages in Fedora
>  build system
>  > must be fully supported including addressing all security issues.
>  >
>  > In modular Fedora that's (effectively) not true. Packages that only exist
>  > for the sake of building other packages (i.e. build-only dependencies)
>  can be
>  > retained in the Fedora build system and never left it. That means those
>  > packages are never made available to Fedora users and thus a service
>  level for
>  > them is significantly lower. E.g. no security fixes, not bug fixes, no
>  > integration, not tests, no API/ABI stability. The only requirement is
>  that
>  > they can be built and used for building other packages.
>
>  So conceptually, one way we can solve this problem by implementing a way
>  to mark certain non-modular RPMs as "build root only" packages and thus
>  composing them into a separate "build root" yum repo, that is not enabled
>  by default except in the build system.
>
>  
The problem is that you may need differing packages for your 'build-root'.
This means that someone has to determine if your buildroot only has
foobar-1.2-3 or foobar-2.3-4 which are neither wanted to be in the
buildroot but unison-2.1 needs one and unison-2.2 needs the other. [Or some
toolkit where to get to the next version you need to compile a version
before the one you have with different yacc/etc and then need the next
version to finish the build.]

The more fundamental problem is that everyone assumes that if it lives in
the buildroot only.. it magically doesn't have any bugs which can cause
problems with a package being compiled by it. Instead it is now a package
which no one can easily inspect to see that XYZ app is dumping core because
your buildroot only yacc is a recycled road apple.



>  Modularity is being used because it is the only solution that is
> available
>  today, not because it is a good/desired solution.
>
>  Regards,
>  Daniel
>  --
>  |: https://berrange.com      -o-
>  https://www.flickr.com/photos/dberrange :|
>  |: https://libvirt.org         -o-
>  https://fstop138.berrange.com :|
>  |: https://entangle-photo.org    -o-
>  https://www.instagram.com/dberrange :|
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Stephen J Smoogen.

[David Cantrell]

On Thu, Sep 10, 2020 at 06:30:05PM +0100, Daniel P. Berrangé wrote:
> On Thu, Sep 10, 2020 at 04:03:46PM +0200, Petr Pisar wrote:
> > On Thu, Sep 10, 2020 at 02:35:13PM +0100, Daniel P. Berrangé wrote:
> > > On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
> > >
> > > > 4.  The benefit we want to preserve from modules is to maintain packages
> > > > with varying expectation of quality, specifically separating the
> > > > build-time-only vs runtime dependencies.  e.g. in that case that a web
> > > > server like Eclipse Jetty is required as a dep for testing another
> > > > component during the build, we want to be able to use and build that
> > > > component, without being indefinitely on the hook for security errata. 
> > > > (The build dependency tree is particularly complex for Maven and
> > > > involves many examples of packages with frequent and high severity
> > > > vulnerabilies)
> > >
> > > What are you doing different in terms of supporting deps in the module
> > > that reduces the security errata burden, compared to non-modular builds ?
> > >
> > > It feels like if we have some policy that is creating unsustainable
> > > maint burden wrt non-modular packaging, we should re-examine this
> > > policy rather than trying to workaround it by going modular, which
> > > creates a different kind of maint burden.
> > >
> > In non-modular Fedora all packages that we have in Fedora build system (Koji)
> > are tagged into Fedora repositories and made available to all users on their
> > computers for any purpose. That implies that all packages in Fedora build system
> > must be fully supported including addressing all security issues.
> >
> > In modular Fedora that's (effectively) not true. Packages that only exist
> > for the sake of building other packages (i.e. build-only dependencies) can be
> > retained in the Fedora build system and never left it. That means those
> > packages are never made available to Fedora users and thus a service level for
> > them is significantly lower. E.g. no security fixes, not bug fixes, no
> > integration, not tests, no API/ABI stability. The only requirement is that
> > they can be built and used for building other packages.
>
> So conceptually, one way we can solve this problem by implementing a way
> to mark certain non-modular RPMs as "build root only" packages and thus
> composing them into a separate "build root" yum repo, that is not enabled
> by default except in the build system.
>
> Modularity is being used because it is the only solution that is available
> today, not because it is a good/desired solution. 
+1

The ability to hide/not-distribute build-only packages is frequently
cited as a benefit of modularity, but it breaks down when you have
multiple packages across modules share build-only packages.  If
moduleA and moduleB both have a build-only package named BuildTool
then those module developers should work together to either jointly
maintain BuildTool -or- duplicate BuildTool across both modules.
Since the advantage stated is being to hide build-only packages, this
makes it impossible to discover what build-only packages might exist
that you could use in your module.  With this example, modularity
leads to the bundling problem we've [Fedora] been very public about
disliking in the past.

The buildroot tries to solve that, but we already had a solution for
that and it was the repo.  Labeling packages as build-only or grouping
them as build tools only or otherwise marking them as packages that
exist to build other packages could be done in numerous ways that
still makes those tools discoverable to other package maintainers and
developers.

-- 
David Cantrell <dcantrell(a)redhat.com&gt;
Red Hat, Inc. | Boston, MA | EST5EDT

[Alexander Scheel]

Hi Joe,

On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
>  Hi all,
>
>  I'm writing as the Red Hat engineering manager responsible for Maven and
>  Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
>  team.  I want to give a broad response to some of the points here:
>
>  1.  The team has two missions in Fedora:
>
>  a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
>  to provide developers with the most popular Java build systems which are
>  reviewed, tested, and updated through the release lifecycle.
>
>  b) We design, develop and document tooling that enables anyone to
>  package Java software with a simple, efficient and scalable process. We
>  are also active members of Java SIG, collaborating on complex changes
>  and guiding new contributors.
>
>  2.  We are committed to maintaining the Ant and Maven modules in
>  Fedora.  We have always expected to make them available as default
>  streams and in the buildroot so they can be available and consumed by
>  non-modular packages, but we completely respect the decisions of FESCo
>  to disallow default streams and of other contributors to adopt and
>  maintain the non-modular packages.  We are not going to promise to
>  commit time and resources to maintain the non-modular packages. 
As a reminder (as in my RHEL devel-list reply): there are no default
module streams in Fedora. There is also no Ursa Major/Prime, so were
they to exist, there would still be no way for non-modular packages to
use them.

This makes the artifacts produced here useful only to other modules.
Non-modular packages maintained by other Red Hatters, like Eclipse and
Dogtag PKI cannot use these artifacts. Both of these stacks have tried
to modularize in Fedora but ultimately remained non-modular.

>  3.  Our efforts are currently directed mainly at minimization of the
>  dependency tree which leads to maven and ant, automating the process of
>  bootstrapping maven and updating related components, so that new
>  versions can be imported and built reproducibly and with consistent
>  quality.
>
>  4.  The benefit we want to preserve from modules is to maintain packages
>  with varying expectation of quality, specifically separating the
>  build-time-only vs runtime dependencies.  e.g. in that case that a web
>  server like Eclipse Jetty is required as a dep for testing another
>  component during the build, we want to be able to use and build that
>  component, without being indefinitely on the hook for security errata.
>  (The build dependency tree is particularly complex for Maven and
>  involves many examples of packages with frequent and high severity
>  vulnerabilies)
>
>  Regards, Joe
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Aleksandar Kurtakov]

On Thu, Sep 10, 2020 at 4:46 PM Alexander Scheel <ascheel(a)redhat.com&gt; wrote:

>  Hi Joe,
>
>  On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
>  >
>  > Hi all,
>  >
>  > I'm writing as the Red Hat engineering manager responsible for Maven and
>  > Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
>  > team.  I want to give a broad response to some of the points here:
>  >
>  > 1.  The team has two missions in Fedora:
>  >
>  > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
>  > to provide developers with the most popular Java build systems which are
>  > reviewed, tested, and updated through the release lifecycle.
>  >
>  > b) We design, develop and document tooling that enables anyone to
>  > package Java software with a simple, efficient and scalable process. We
>  > are also active members of Java SIG, collaborating on complex changes
>  > and guiding new contributors.
>  >
>  > 2.  We are committed to maintaining the Ant and Maven modules in
>  > Fedora.  We have always expected to make them available as default
>  > streams and in the buildroot so they can be available and consumed by
>  > non-modular packages, but we completely respect the decisions of FESCo
>  > to disallow default streams and of other contributors to adopt and
>  > maintain the non-modular packages.  We are not going to promise to
>  > commit time and resources to maintain the non-modular packages.
>
>  As a reminder (as in my RHEL devel-list reply): there are no default
>  module streams in Fedora. There is also no Ursa Major/Prime, so were
>  they to exist, there would still be no way for non-modular packages to
>  use them.
>
>  This makes the artifacts produced here useful only to other modules.
>  Non-modular packages maintained by other Red Hatters, like Eclipse and
>  Dogtag PKI cannot use these artifacts. Both of these stacks have tried
>  to modularize in Fedora but ultimately remained non-modular.
>  
FWIW, I'm eagerly looking to stop packaging Eclipse as RPM entirely.
Flatpak is way better suited for our use case and in addition gives us
access to a way bigger install base. And the involvement on Java packaging
in Fedora is so low that we literally have to maintain whole other stacks
including jetty, lucene and etc. - not feasible work in any way.


>  > 3.  Our efforts are currently directed mainly at minimization of the
>  > dependency tree which leads to maven and ant, automating the process of
>  > bootstrapping maven and updating related components, so that new
>  > versions can be imported and built reproducibly and with consistent
>  > quality.
>  >
>  > 4.  The benefit we want to preserve from modules is to maintain packages
>  > with varying expectation of quality, specifically separating the
>  > build-time-only vs runtime dependencies.  e.g. in that case that a web
>  > server like Eclipse Jetty is required as a dep for testing another
>  > component during the build, we want to be able to use and build that
>  > component, without being indefinitely on the hook for security errata.
>  > (The build dependency tree is particularly complex for Maven and
>  > involves many examples of packages with frequent and high severity
>  > vulnerabilies)
>  >
>  > Regards, Joe
>  > _______________________________________________
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Alexander Kurtakov
Red Hat Eclipse Team

[Alexander Scheel]

On Thu, Sep 10, 2020 at 10:11 AM Aleksandar Kurtakov
<akurtako(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 4:46 PM Alexander Scheel <ascheel(a)redhat.com&gt; wrote:
> >
> > Hi Joe,
> >
> > On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
> > >
> > > Hi all,
> > >
> > > I'm writing as the Red Hat engineering manager responsible for Maven and
> > > Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
> > > team.  I want to give a broad response to some of the points here:
> > >
> > > 1.  The team has two missions in Fedora:
> > >
> > > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
> > > to provide developers with the most popular Java build systems which are
> > > reviewed, tested, and updated through the release lifecycle.
> > >
> > > b) We design, develop and document tooling that enables anyone to
> > > package Java software with a simple, efficient and scalable process. We
> > > are also active members of Java SIG, collaborating on complex changes
> > > and guiding new contributors.
> > >
> > > 2.  We are committed to maintaining the Ant and Maven modules in
> > > Fedora.  We have always expected to make them available as default
> > > streams and in the buildroot so they can be available and consumed by
> > > non-modular packages, but we completely respect the decisions of FESCo
> > > to disallow default streams and of other contributors to adopt and
> > > maintain the non-modular packages.  We are not going to promise to
> > > commit time and resources to maintain the non-modular packages.
> >
> > As a reminder (as in my RHEL devel-list reply): there are no default
> > module streams in Fedora. There is also no Ursa Major/Prime, so were
> > they to exist, there would still be no way for non-modular packages to
> > use them.
> >
> > This makes the artifacts produced here useful only to other modules.
> > Non-modular packages maintained by other Red Hatters, like Eclipse and
> > Dogtag PKI cannot use these artifacts. Both of these stacks have tried
> > to modularize in Fedora but ultimately remained non-modular.
>
>
>  FWIW, I'm eagerly looking to stop packaging Eclipse as RPM entirely. Flatpak is way
> better suited for our use case and in addition gives us access to a way bigger install
> base. And the involvement on Java packaging in Fedora is so low that we literally have to
> maintain whole other stacks including jetty, lucene and etc. - not feasible work in any
> way. 
Mat Booth (also from the Eclipse team!) still actively helps out with
this packaging as RPMs and we thank him for his efforts while they
last. :-)

But Flatpak isn't suitable for everyone--it isn't great for servers
like Dogtag and IdM for instance. So we still need RPM packaging
around for that.

Also, does that mean Eclipse won't be in RHEL-9? So far it looks like
it is not rebuilt for ELN so it seems likely it won't be.

https://koji.fedoraproject.org/koji/packageinfo?packageID=183


- Alex

> >
> >
> > > 3.  Our efforts are currently directed mainly at minimization of the
> > > dependency tree which leads to maven and ant, automating the process of
> > > bootstrapping maven and updating related components, so that new
> > > versions can be imported and built reproducibly and with consistent
> > > quality.
> > >
> > > 4.  The benefit we want to preserve from modules is to maintain packages
> > > with varying expectation of quality, specifically separating the
> > > build-time-only vs runtime dependencies.  e.g. in that case that a web
> > > server like Eclipse Jetty is required as a dep for testing another
> > > component during the build, we want to be able to use and build that
> > > component, without being indefinitely on the hook for security errata.
> > > (The build dependency tree is particularly complex for Maven and
> > > involves many examples of packages with frequent and high severity
> > > vulnerabilies)
> > >
> > > Regards, Joe
> > > _______________________________________________
> > > devel mailing list -- devel(a)lists.fedoraproject.org
> > > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>
>
>
>  --
>  Alexander Kurtakov
>  Red Hat Eclipse Team
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Vitaly Zaitsev]

On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
>  Flatpak is way better suited for our use case and in addition gives
> us
>  access to a way bigger install base. 
Flathub is a third-party repository and not related to Fedora at all.

>  And the involvement on Java packaging in Fedora is so low that we
> literally have to maintain whole other stacks including jetty, lucene and etc. - not
> feasible work in any way. 
Fedora Modularity team destroyed the entire Java stack in Fedora after
moving ant/maven to modules.

I think FESCo should completely forbid modules without packaged
non-modular versions.

-- 
Sincerely,
  Vitaly Zaitsev (vitaly(a)easycoding.org)

[Aleksandar Kurtakov]

On Thu, Sep 10, 2020 at 5:54 PM Vitaly Zaitsev via devel <
devel(a)lists.fedoraproject.org&gt; wrote:

>  On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
>  > Flatpak is way better suited for our use case and in addition gives us
>  > access to a way bigger install base.
>
>  Flathub is a third-party repository and not related to Fedora at all.
>  
>  > And the involvement on Java packaging in Fedora is so low that
> we
>  literally have to maintain whole other stacks including jetty, lucene and
>  etc. - not feasible work in any way.
>
>  Fedora Modularity team destroyed the entire Java stack in Fedora after
>  moving ant/maven to modules.
>  
As I've been involved in ant/maven packaging for a decade or so I would
dare to say that this is not the truth. It just exposed the fact that less
and less people were actively maintaining things as most of the people that
used to do it moved on to other things and the number of new people that
joined is quite low. So the burden on people left is bigger and bigger.


>  I think FESCo should completely forbid modules without packaged
>  non-modular versions.
>  
>  --
>  Sincerely,
>    Vitaly Zaitsev (vitaly(a)easycoding.org)
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Alexander Kurtakov
Red Hat Eclipse Team

[Christopher]

On Thu, Sep 10, 2020 at 11:08 AM Aleksandar Kurtakov
<akurtako(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 5:54 PM Vitaly Zaitsev via devel
> <devel(a)lists.fedoraproject.org&gt; wrote:
> >
> > On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
> > > Flatpak is way better suited for our use case and in addition gives us
> > > access to a way bigger install base.
> >
> > Flathub is a third-party repository and not related to Fedora at all.
> >
> >
> > > And the involvement on Java packaging in Fedora is so low that we literally have
> to maintain whole other stacks including jetty, lucene and etc. - not feasible work in any
> way.
> >
> > Fedora Modularity team destroyed the entire Java stack in Fedora after
> > moving ant/maven to modules.
>
>
>  As I've been involved in ant/maven packaging for a decade or so I would dare to say
> that this is not the truth. It just exposed the fact that less and less people were
> actively maintaining things as most of the people that used to do it moved on to other
> things and the number of new people that joined is quite low. So the burden on people left
> is bigger and bigger. 
I am a relative newcomer to RPM packaging. I became a packager because
I was a long-time Fedora user, and wanted to distribute my Java
packages in Fedora. I began packaging, and slowly began taking over a
few related packages, until the entire stack fell out from under me
because of modularity. Had it been kept alive non-modular, I'd have
been able to encourage others to participate (I had already recruited
one other from $dayjob to help comaintain Java packages and was in the
process of recruiting more when modularity became a thing), and I
would have been able to participate more and more. However, because
everything fell apart, I was not able to do that.

So, from my perspective, it did both: it exposed that less and less
people were actively maintaining things *and* it destroyed the entire
Java stack by making it *harder* for newcomers like me to actively
maintain things that were becoming out of date. As an Apache Software
Foundation member, I really appreciate their "community first"
mindset. One of their driving principles is that having a good
community enables code to get better... prioritizing code or
technologies does not necessary enable community. I feel like Fedora's
modularity efforts, while good intentioned, from a technology
perspectve, were a net negative in terms of community because it
raised the bar to participation and prioritized a design over the
effects on the community.

As mentioned elsewhere in this thread, I also see the modular versions
of Maven/Ant as being worthless (to me). I used to use the non-modular
Maven RPM for my development, but now that it is modular only, I find
that it's actually better to just download the binaries directly from
Apache, because the experience is better than using modules. They are
more up-to-date, break less often, and it requires me to do fewer
steps to keep up-to-date. The convenience of using the stable RPM in
non-modular Fedora is now gone for me, as soon as it became a module.

I wish I could say that modularity didn't have a negative impact...
and that it was a complete success, and that all fault (especially
that pertaining to the Java stack) is elsewhere, but I can't honestly
say I believe that. It would merely be wishful thinking on my part.

>
>>
>>
>> I think FESCo should completely forbid modules without packaged
>> non-modular versions.
>>
>>
>> --
>> Sincerely,
>>   Vitaly Zaitsev (vitaly(a)easycoding.org)
>> _______________________________________________
>> devel mailing list -- devel(a)lists.fedoraproject.org
>> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
>
>
>
> --
> Alexander Kurtakov
> Red Hat Eclipse Team
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Miro Hrončok]

On 10. 09. 20 16:53, Vitaly Zaitsev via devel wrote:
>  I think FESCo should completely forbid modules without packaged
>  non-modular versions. 
It did.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

Modules without packaged non-modular versions (Was Re: The Future of the Java Stack (also regarding ELN and RHEL))

[Jie Kang]

On Thu, Sep 10, 2020 at 11:10 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  On 10. 09. 20 16:53, Vitaly Zaitsev via devel wrote:
>  > I think FESCo should completely forbid modules without packaged
>  > non-modular versions.
>
>  It did. 
Hi,

Can you share the ticket/issue for this restriction with me?


Thank you,
Jie

>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

Modules without packaged non-modular versions (Was Re: The Future of the Java Stack (also regarding ELN and RHEL))

[Fabio Valentini]

On Fri, Sep 11, 2020 at 10:56 PM Jie Kang <jkang(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 11:10 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  >
>  > On 10. 09. 20 16:53, Vitaly Zaitsev via devel wrote:
>  > > I think FESCo should completely forbid modules without packaged
>  > > non-modular versions.
>  >
>  > It did.
>
>  Hi,
>
>  Can you share the ticket/issue for this restriction with me?
>
>
>  Thank you,
>  Jie 
The FESCo ticket is here:
https://pagure.io/fesco/issue/2406

Specifically, the decisions are documented in this comment:
https://pagure.io/fesco/issue/2406#comment-658315

The relevant one being the unanimous decision to ban modular-only packages:

"AGREED: Modular-only packages are not allowed and modular versions
are only [to] be allowed as alternatives to non-modular packages. (+9, 0, -0)"

Hope that helps.
Fabio

Modules without packaged non-modular versions (Was Re: The Future of the Java Stack (also regarding ELN and RHEL))

[Miro Hrončok]

On 11. 09. 20 22:55, Jie Kang wrote:
>  On Thu, Sep 10, 2020 at 11:10 AM Miro Hrončok
> <mhroncok(a)redhat.com&gt; wrote:
> >
> > On 10. 09. 20 16:53, Vitaly Zaitsev via devel wrote:
> >> I think FESCo should completely forbid modules without packaged
> >> non-modular versions.
> >
> > It did.
>  
>  Hi,
>  
>  Can you share the ticket/issue for this restriction with me? 
https://pagure.io/fesco/issue/2406

Modular-only packages are not allowed and modular versions are only allowed as 
alternatives to non-modular packages. There is an exception to this rule: if the 
package does not function in non-modular Fedora (with reasonable amount of 
work), it is permitted to have it in module only. As an example if non-modular 
Fedora has dnf 4, and there is module with dnf 5, a package that only works with 
dnf 5 can remain modular only, until dnf 5 is included in non-modular Fedora. 
For the time being, such exceptions can be granted by FESCo.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Ben Cotton]

On Thu, Sep 10, 2020 at 10:54 AM Vitaly Zaitsev via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
>  > Flatpak is way better suited for our use case and in addition gives us
>  > access to a way bigger install base.
>
>  Flathub is a third-party repository and not related to Fedora at all.
>  
Flat*hub* is third party, but we have first-party flatpaks at
registry.fedoraproject.org. The flatpak package adds it as a remote
via /usr/lib/systemd/system/flatpak-add-fedora-repos.service

-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Alexander Scheel]

Ben,

Can Fedora first-party flatpaks be built from unsigned, untrusted
content outside of the Fedora Repos? Or can they only be built from
content otherwise already present in Fedora? Just curious what
benefits a first-party flatpak has versus an upstream one.

AFAIK, the last Fedora Container docs said that all Container content
must be pulled from Fedora Repos, and therefore was left holding a
bunch of RPMs anyways.


- A

On Thu, Sep 10, 2020 at 11:38 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 10:54 AM Vitaly Zaitsev via devel
>  <devel(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
>  > > Flatpak is way better suited for our use case and in addition gives us
>  > > access to a way bigger install base.
>  >
>  > Flathub is a third-party repository and not related to Fedora at all.
>  >
>  Flat*hub* is third party, but we have first-party flatpaks at
>  registry.fedoraproject.org. The flatpak package adds it as a remote
>  via /usr/lib/systemd/system/flatpak-add-fedora-repos.service
>
>  --
>  Ben Cotton
>  He / Him / His
>  Senior Program Manager, Fedora & CentOS Stream
>  Red Hat
>  TZ=America/Indiana/Indianapolis
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Guido Aulisi]

>  Il giorno 10 set 2020, alle ore 16:53, Vitaly Zaitsev via devel
> <devel(a)lists.fedoraproject.org&gt; ha scritto:
>  
>  On 10.09.2020 16:10, Aleksandar Kurtakov wrote:
> > Flatpak is way better suited for our use case and in addition gives us
> > access to a way bigger install base.
>  
>  Flathub is a third-party repository and not related to Fedora at all.
>  
> > And the involvement on Java packaging in Fedora is so low that we literally have to
> maintain whole other stacks including jetty, lucene and etc. - not feasible work in any
> way.
>  
>  Fedora Modularity team destroyed the entire Java stack in Fedora after
>  moving ant/maven to modules.
>  
>  I think FESCo should completely forbid modules without packaged
>  non-modular versions. 
I totally agree

>  -- 
>  Sincerely,
>   Vitaly Zaitsev (vitaly(a)easycoding.org)
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 

Ciao
Guido

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 3:46 PM Alexander Scheel <ascheel(a)redhat.com&gt; wrote:
>  Hi Joe,
>
>  On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
>  >
>  > Hi all,
>  >
>  > I'm writing as the Red Hat engineering manager responsible for Maven and
>  > Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
>  > team.  I want to give a broad response to some of the points here:
>  >
>  > 1.  The team has two missions in Fedora:
>  >
>  > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
>  > to provide developers with the most popular Java build systems which are
>  > reviewed, tested, and updated through the release lifecycle.
>  >
>  > b) We design, develop and document tooling that enables anyone to
>  > package Java software with a simple, efficient and scalable process. We
>  > are also active members of Java SIG, collaborating on complex changes
>  > and guiding new contributors.
>  >
>  > 2.  We are committed to maintaining the Ant and Maven modules in
>  > Fedora.  We have always expected to make them available as default
>  > streams and in the buildroot so they can be available and consumed by
>  > non-modular packages, but we completely respect the decisions of FESCo
>  > to disallow default streams and of other contributors to adopt and
>  > maintain the non-modular packages.  We are not going to promise to
>  > commit time and resources to maintain the non-modular packages.
>
>  As a reminder (as in my RHEL devel-list reply): there are no default
>  module streams in Fedora. There is also no Ursa Major/Prime, so were
>  they to exist, there would still be no way for non-modular packages to
>  use them. 
There are default module streams in Fedora 31.

>  This makes the artifacts produced here useful only to other modules.
>  Non-modular packages maintained by other Red Hatters, like Eclipse and
>  Dogtag PKI cannot use these artifacts. Both of these stacks have tried
>  to modularize in Fedora but ultimately remained non-modular. 
Our goal is to deliver Maven and Ant to end users, maintaining
dependencies of Dogtag PKI is outside of my area of interest.
End users can consume Maven and Ant from the modules, which aligns
with our mission statement.

>  > 3.  Our efforts are currently directed mainly at minimization of
> the
>  > dependency tree which leads to maven and ant, automating the process of
>  > bootstrapping maven and updating related components, so that new
>  > versions can be imported and built reproducibly and with consistent
>  > quality.
>  >
>  > 4.  The benefit we want to preserve from modules is to maintain packages
>  > with varying expectation of quality, specifically separating the
>  > build-time-only vs runtime dependencies.  e.g. in that case that a web
>  > server like Eclipse Jetty is required as a dep for testing another
>  > component during the build, we want to be able to use and build that
>  > component, without being indefinitely on the hook for security errata.
>  > (The build dependency tree is particularly complex for Maven and
>  > involves many examples of packages with frequent and high severity
>  > vulnerabilies)
>  >
>  > Regards, Joe
>  > _______________________________________________
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Robbie Harwood]

Mikolaj Izdebski <mizdebsk(a)redhat.com&gt; writes:

>  On Thu, Sep 10, 2020 at 3:46 PM Alexander Scheel
> <ascheel(a)redhat.com&gt; wrote:
> >
> > Hi Joe,
> >
> > On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
> > >
> > > Hi all,
> > >
> > > I'm writing as the Red Hat engineering manager responsible for Maven and
> > > Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
> > > team.  I want to give a broad response to some of the points here:
> > >
> > > 1.  The team has two missions in Fedora:
> > >
> > > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
> > > to provide developers with the most popular Java build systems which are
> > > reviewed, tested, and updated through the release lifecycle.
> > >
> > > b) We design, develop and document tooling that enables anyone to
> > > package Java software with a simple, efficient and scalable process. We
> > > are also active members of Java SIG, collaborating on complex changes
> > > and guiding new contributors.
> > >
> > > 2.  We are committed to maintaining the Ant and Maven modules in
> > > Fedora.  We have always expected to make them available as default
> > > streams and in the buildroot so they can be available and consumed by
> > > non-modular packages, but we completely respect the decisions of FESCo
> > > to disallow default streams and of other contributors to adopt and
> > > maintain the non-modular packages.  We are not going to promise to
> > > commit time and resources to maintain the non-modular packages.
> >
> > As a reminder (as in my RHEL devel-list reply): there are no default
> > module streams in Fedora. There is also no Ursa Major/Prime, so were
> > they to exist, there would still be no way for non-modular packages to
> > use them.
>
>  There are default module streams in Fedora 31. 
Fedora 31 is most likely EOL in about two months.  I really hope you're
not targeting your future development against it.

Thanks,
--Robbie

[Zbigniew Jędrzejewski-Szmek]

On Thu, Sep 10, 2020 at 12:12:05PM -0400, Robbie Harwood wrote:
>  Mikolaj Izdebski <mizdebsk(a)redhat.com&gt; writes:
>  > On Thu, Sep 10, 2020 at 3:46 PM Alexander Scheel <ascheel(a)redhat.com&gt; wrote:
>  >> Hi Joe,
>  >> On Thu, Sep 10, 2020 at 8:52 AM Joe Orton <jorton(a)redhat.com&gt; wrote:
>  >> > I'm writing as the Red Hat engineering manager responsible for Maven
> and
>  >> > Ant in RHEL, and on behalf of Mikolaj Izdebski and Marian Koncek from my
>  >> > team.  I want to give a broad response to some of the points here:
>  >> >
>  >> > 1.  The team has two missions in Fedora:
>  >> >
>  >> > a) We deliver, maintain and support Ant and Maven in Fedora. Our aim is
>  >> > to provide developers with the most popular Java build systems which are
>  >> > reviewed, tested, and updated through the release lifecycle.
>  >> >
>  >> > b) We design, develop and document tooling that enables anyone to
>  >> > package Java software with a simple, efficient and scalable process. We
>  >> > are also active members of Java SIG, collaborating on complex changes
>  >> > and guiding new contributors.
>  >> >
>  >> > 2.  We are committed to maintaining the Ant and Maven modules in
>  >> > Fedora.  We have always expected to make them available as default
>  >> > streams and in the buildroot so they can be available and consumed by
>  >> > non-modular packages, but we completely respect the decisions of FESCo
>  >> > to disallow default streams and of other contributors to adopt and
>  >> > maintain the non-modular packages.  We are not going to promise to
>  >> > commit time and resources to maintain the non-modular packages.
>  >>
>  >> As a reminder (as in my RHEL devel-list reply): there are no default
>  >> module streams in Fedora. There is also no Ursa Major/Prime, so were
>  >> they to exist, there would still be no way for non-modular packages to
>  >> use them.
>  >
>  > There are default module streams in Fedora 31.
>  
>  Fedora 31 is most likely EOL in about two months.  I really hope you're
>  not targeting your future development against it. 
This exchange summarizes the situation nicely.

Modularity can be considered an over-complicated hyped-through-the-roof
bundling mechanism.

For a long time Fedora has very strongly discouraged bundling in the sense of
subsumming one package into another to have a custom build of a dependency.

The disapproval of bundling is not because it doesn't work: it does, and in
fact with some crappy libraries it's the least bad solution. The disapproval
is motivated by the fact that bundling doesn't scale and that it subtracts from
the ecosystem. Instead of us all cooperating and each bugfix helping all
users of a given dependency, a maintainer of a bundled library is only helping
themselves and their package.

Bundling is not inherently bad: currently the policy even allows it as
a workaround if using the system-wide package is not feasible. It is a
pragmatic choice to allow it as a last resort. But the effect of bundling
on other packages must be minimized any conflicts or confusion with the
system version avoided.

With Modularity, we threw this accumulated wisdom away.
In two ways: by encouraging private^Wbuild-only dependencies and by
letting bundled^Wmodularized rpms shadow normal rpms.

For Maven packaging the appeal of Modularity is clearly the privatization of
the dependency tree, which obviously undercuts the ecosystem of packages.

The Java ecosystem collapsed so quickly because it was already weak
— hundreds of packages on the shoulders of one person. But even in a stronger
ecosystem, with enough packages modularized, the packaging ecosystem of
mutual cooperation would collapse.

For some other modules, the appeal is the overriding of package deps, which
means that the modular rpms don't have to worry about getting it deps precisely
declared, at the cost of breaking the deps declared in other packages.

If there wasn't Modularity and instead Maven would bundle it's deps into
one huge srpm, the effect on the ecosystem would be the same. As with
bundling, Modularization gives short-term relief at a very high long-term
cost. We should avoid modularization like we avoid bundling.

Zbyszek

[Petr Pisar]

On Thu, Sep 10, 2020 at 09:59:05PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
>  For Maven packaging the appeal of Modularity is clearly the
> privatization of
>  the dependency tree, which obviously undercuts the ecosystem of packages.
>   
You are right that bundling is one of the features of Modularity and that this
freedom undermines an integration effort on the Fedora distribution level.

But bundling is not the only appeal for Maven maintainers. If I can speek for
Mikolaj, then another appeal is sharing a module among multiple Fedora
releases. Because byte-compiled Java code is portable, it is possible to build
a module on Fedora 31 and have the same module build available on Fedora 34.
This saves the module maintainers from the burden of rebuilding the Java
packages for each Fedora and Modularity is the first place that actuallty can
leverage this Java feature.

>  If there wasn't Modularity and instead Maven would bundle
> it's deps into
>  one huge srpm, the effect on the ecosystem would be the same. 
Not really. Modules are collections of packages and various modules can share
the sources of the same packages. That means that Modularity does not prevent
from sharing a maintenance cost of the packages. It only makes it more
difficult, because of worse discoverbility and a wider selection of the
choice.

>  Modularization gives short-term relief at a very high long-term
>  cost. We should avoid modularization like we avoid bundling. 
Sarcasm: I really like how other one people order other ones that they have to
maintain the packages for them.

-- Petr

[Mikolaj Izdebski]

On Fri, Sep 11, 2020 at 8:44 AM Petr Pisar <ppisar(a)redhat.com&gt; wrote:
>  On Thu, Sep 10, 2020 at 09:59:05PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
>  > For Maven packaging the appeal of Modularity is clearly the privatization of
>  > the dependency tree, which obviously undercuts the ecosystem of packages.
>  >
>  You are right that bundling is one of the features of Modularity and that this
>  freedom undermines an integration effort on the Fedora distribution level.
>
>  But bundling is not the only appeal for Maven maintainers. If I can speek for
>  Mikolaj, then another appeal is sharing a module among multiple Fedora
>  releases. Because byte-compiled Java code is portable, it is possible to build
>  a module on Fedora 31 and have the same module build available on Fedora 34.
>  This saves the module maintainers from the burden of rebuilding the Java
>  packages for each Fedora and Modularity is the first place that actuallty can
>  leverage this Java feature. 
Right. Building modules once and shipping the same packages to all
releases used to be important to me, but since then I developed a new
way of bootstrapping Maven (MBI - Maven Bootstrap Initiative) so that
now I can build Maven in a reliable, reproducible way. That modularity
feature is therefore no longer important to me.

--
Mikolaj

[Vít Ondruch]

Dne 11. 09. 20 v 8:43 Petr Pisar napsal(a):
>  On Thu, Sep 10, 2020 at 09:59:05PM +0000, Zbigniew Jędrzejewski-Szmek
> wrote:
> > For Maven packaging the appeal of Modularity is clearly the privatization of
> > the dependency tree, which obviously undercuts the ecosystem of packages.
> >
>  You are right that bundling is one of the features of Modularity and that this
>  freedom undermines an integration effort on the Fedora distribution level.
>
>  But bundling is not the only appeal for Maven maintainers. If I can speek for
>  Mikolaj, then another appeal is sharing a module among multiple Fedora
>  releases. Because byte-compiled Java code is portable, it is possible to build
>  a module on Fedora 31 and have the same module build available on Fedora 34.
>  This saves the module maintainers from the burden of rebuilding the Java
>  packages for each Fedora and Modularity is the first place that actuallty can
>  leverage this Java feature.
>  
While the idea of sharing between Fedora is nice, it won't work in
practice. There is more then just "byte-compiled Java code is portable".
This allows you to ship certain set of the packages, but it won't allow
you to easily update any single package. You have always consider the
API changes, dependency changes and if the update of the package, which
you would do on Rawhide and would be perfectly fine, won't break other
parts of F31 ecosystem. The choice is not any easier.

If on the other hand the F31 was left in the state when it was branched
+ applying fixes, the Rawhide would be free to update the packages.

Not mentioning that keeping the one set of packages is possibly against
the idea of Fedora update policy [1].


Vít



[1] https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/

[Mikolaj Izdebski]

On Fri, Sep 11, 2020 at 12:32 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  This exchange summarizes the situation nicely.
>
>  Modularity can be considered an over-complicated hyped-through-the-roof
>  bundling mechanism.
>
>  For a long time Fedora has very strongly discouraged bundling in the sense of
>  subsumming one package into another to have a custom build of a dependency.
>
>  The disapproval of bundling is not because it doesn't work: it does, and in
>  fact with some crappy libraries it's the least bad solution. The disapproval
>  is motivated by the fact that bundling doesn't scale and that it subtracts from
>  the ecosystem. Instead of us all cooperating and each bugfix helping all
>  users of a given dependency, a maintainer of a bundled library is only helping
>  themselves and their package.
>
>  Bundling is not inherently bad: currently the policy even allows it as
>  a workaround if using the system-wide package is not feasible. It is a
>  pragmatic choice to allow it as a last resort. But the effect of bundling
>  on other packages must be minimized any conflicts or confusion with the
>  system version avoided.
>
>  With Modularity, we threw this accumulated wisdom away.
>  In two ways: by encouraging private^Wbuild-only dependencies and by
>  letting bundled^Wmodularized rpms shadow normal rpms.
>
>  For Maven packaging the appeal of Modularity is clearly the privatization of
>  the dependency tree, which obviously undercuts the ecosystem of packages.
>
>  The Java ecosystem collapsed so quickly because it was already weak
>  — hundreds of packages on the shoulders of one person. But even in a stronger
>  ecosystem, with enough packages modularized, the packaging ecosystem of
>  mutual cooperation would collapse.
>
>  For some other modules, the appeal is the overriding of package deps, which
>  means that the modular rpms don't have to worry about getting it deps precisely
>  declared, at the cost of breaking the deps declared in other packages.
>
>  If there wasn't Modularity and instead Maven would bundle it's deps into
>  one huge srpm, the effect on the ecosystem would be the same. As with
>  bundling, Modularization gives short-term relief at a very high long-term
>  cost. We should avoid modularization like we avoid bundling. 
You summarized it really well, thanks Zbyszek. To me bundling is,
and always was, an important design feature of modularity.
There are other important aspects of it, but I dare to say that from my
PoV its form of bundling is the most important feature.

I wanted to expand on your story, present my version of the same story.

For a long time we (Java maintenance team at Red Hat) were maintaining
Fedora packages without any form of bundling because it was considered
harmful and was not allowed in any way, not without explicit exception.

Over time the number of contributors to Java packaging (number of active
Java SIG members) was decreasing, most of Java maintainers were gone,
or "unresponsive".  Making changes to our packages, like improving
packaging automation, required testing and possibly adjusting
thousands of others' packages, most of which were largely unmaintained.
Unresponsive maintainer processes would not work as it would require us
to take maintenance of orphaned packages, which we could not afford.

In the meantime our Java maintenance team was shrunk from 4 people
down to just one person, me.  I had to severely limit the time I spent
on Fedora due to other commitments - I had to spend more time on RHEL
work and other side projects, like Koschei, that I was responsible
for.  I could not afford to spend my free after-work time on Fedora
due to personal reasons (I got married, started building a house etc.)
Combined with the collapse of Java SIG, the situation was difficult.

Then a new, sophisticated way of bundling, called modularization, was
designed and came to my help. This way of bundling was no only
considered not harmful, but also approved and even made as a Fedora
objective. Since it solved the biggest issues I had with packaging, I
became a very eager early adopter and quickly built everything as
modules.

I was still maintaining non-modular packages, but I hoped that one day
non-modular packages would be completely replaced by modules.  When
modules were added to Fedora proper, I made them default streams and
waited for a solution like Ursa-Major that would allow me to retire
ursine packages, replacing them with modular packages.

RHEL enabled Ursa-Major. My javapackages-tools module was the first
ones to make use of it in RHEL and we succeeded to retire all ursine
packages in RHEL. When Fedora finally rejected Ursa-Major, I was very
frustrated as I perceived it as begin of collapse of modularity, the
great feature that promised to solve the problems I had with
packaging. I could not easily move back to ursine packages and I was
already committed to maintaining modules in RHEL, so I decided to live
with Fedora decision and orphan almost all of my ursine packages.

Since then I've been mostly working in MBI project, which has become a
new upstream for Fedora and RHEL modules. In MBI I made great progress
at improving the packages and reducing their number by more than half
- I didn't have to worry about breaking users or others' packages, and I
was not limited by constraints of Fedora infrastructure.

In the meantime modularity was mostly withdrawn from Fedora - default
streams and then module-only packages were disallowed, which to me
means total failure of modularity as a primary way of delivering
software to users.

Now I can't move on with modules as I don't have any chance against
anti-modularity trends.  I can't convert my modular packages to ursine
packages as this is not allowed by the policy, and would not be
appreciated by current owners of ursine packages.  I can't drop my
packages and move back to co-maintaining ursine packages as it would
mean losing 2 years of my work and the features I developed. The best
I can do is to wait and see what happens, while working on the
packages outside of Fedora, as part of the upstream MBI project.

From a time perspective I do consider modularizing my packages as a
mistake.  But I don't see any sensible way of going back.

--
Mikolaj

[Miro Hrončok]

Thank you for describing the entire story from your pov, I think it's very helpful!

On 11. 09. 20 9:34, Mikolaj Izdebski wrote:
>  I can't drop my
>  packages and move back to co-maintaining ursine packages as it would
>  mean losing 2 years of my work and the features I developed. 
I guess there are two sides of this:

  - the lost features

Can you work together with the new Java maint SIG to have those features 
integrated in the nonmodular packages?

  - the lost years

While sad I believe that at a certain point, we need to be able to admit that 
the years are lost in order to save ourselves from loosing even more years. This 
is a very important aspect in accepting that modularity in Fedora was a failure. 
If we don't do this, we'll keep failing.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Mikolaj Izdebski]

On Fri, Sep 11, 2020 at 11:01 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  Thank you for describing the entire story from your pov, I think it's very helpful!
>
>  On 11. 09. 20 9:34, Mikolaj Izdebski wrote:
>  > I can't drop my
>  > packages and move back to co-maintaining ursine packages as it would
>  > mean losing 2 years of my work and the features I developed.
>
>  I guess there are two sides of this:
>
>    - the lost features
>
>  Can you work together with the new Java maint SIG to have those features
>  integrated in the nonmodular packages? 
Possibly. I will think about this more.

>    - the lost years
>
>  While sad I believe that at a certain point, we need to be able to admit that
>  the years are lost in order to save ourselves from loosing even more years. This
>  is a very important aspect in accepting that modularity in Fedora was a failure.
>  If we don't do this, we'll keep failing. 
I agree and I accept that.

Thanks,
--
Mikolaj

>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>

[Matthew Miller]

On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  maintain the non-modular packages.  We are not going to promise to 
>  commit time and resources to maintain the non-modular packages. 
Joe, here's a part I hope you can help me understand better. Modularity
isn't an entirely new packaging technology. It's simply a layer on top of
RPM. Modules are made out of RPMs, and by design, their specfiles are
exactly the same as RPMs which happen to be grouped into modules.

I understand that it's nice to have exactly one workflow, but it doesn't
seem like it's actually all that much extra effort on top of what you
already have to do to maintain the module to make a non-modular build. Is
this something where triggering the non-modular builds in the same way you
build the module would make a difference?

Or is there something else that I'm not seeing?

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Mikolaj Izdebski]

On Thu, Sep 10, 2020 at 9:59 PM Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Sep 10, 2020 at 01:50:55PM +0100, Joe Orton wrote:
>  > maintain the non-modular packages.  We are not going to promise to
>  > commit time and resources to maintain the non-modular packages.
>
>  Joe, here's a part I hope you can help me understand better. Modularity
>  isn't an entirely new packaging technology. It's simply a layer on top of
>  RPM. Modules are made out of RPMs, and by design, their specfiles are
>  exactly the same as RPMs which happen to be grouped into modules. 
Modularity provides not only a different way of grouping or delivering
RPM packages, but most importantly, a different way of building RPM
packages.

You get a side tag in Koji where you can have private build-only
dependencies that are discarded (filtered) once they are no longer
needed, after module build is done. For build-only packages most of
security vulnerabilities are not exploitable easily, or at all,
therefore are low-severity, which greatly limits maintenance work
required to address them. For example, if upstream tests are ran on an
obsolete, 12-years old version of Tomcat, I don't need to skip tests,
but I can package old Tomcat and run the tests. I don't need to update
that Tomcat or fix security issues as they are not exploitable in
buildroot - Tomcat runs in embedded mode, does not listen on the
network etc. Not something that I could with ursine Tomcat packages -
it would get delivered to users, and we have no control how they use
it - we would need to fix all important user-reported bugs and CVEs as
they are potentially exploitable.

Modularity also introduced the concept of API packages - non-API
packages can have limited usability, with some non-important features
removed. For example if all I need is a small part of Tomcat, I can
reduce tomcat package to build just the tiny parts that I need, which
don't have any dependencies. Again, not something I could do with
ursine Tomcat package.

Another, more concrete example: core Ant doesn't have any dependencies
beyond JDK. It is easy to build and maintain, yet very functional. On
the other hand, full Ant with all the optional tasks depends on more
than 200 Java packages. For the purpose of building all packages that
I am interested in, core Ant with JUnit tasks (total of 3 packages) is
sufficient. Functionalities of Ant like sending emails or image
processing are obviously not needed in this case. If building other
packages is the only use of Ant then it can be maintained much more
easily (3 instead of ~250 packages). But when Ant is ursine package in
Fedora then it should be the full Ant - we can't claim to deliver Ant
to users while it is just part of it. Eclipse in Fedora requires full
Ant too.

>  I understand that it's nice to have exactly one workflow, but it
> doesn't
>  seem like it's actually all that much extra effort on top of what you
>  already have to do to maintain the module to make a non-modular build. Is
>  this something where triggering the non-modular builds in the same way you
>  build the module would make a difference? 
It's not about the workflow, it is about reducing package maintenance
work that is required. Modularity allows us to greatly reduce the
number of packages by patching-out non-API packages to remove unneeded
features and it allows us to spend less time on fixing bugs in
packages that are used merely as build dependencies and which we don't
intend to be used by end users.

>  Or is there something else that I'm not seeing?
>
>  --
>  Matthew Miller
>  <mattdm(a)fedoraproject.org&gt;
>  Fedora Project Leader
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Tomasz Torcz]

On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  You get a side tag in Koji where you can have private build-only
>  dependencies that are discarded (filtered) once they are no longer
>  needed, after module build is done. For build-only packages most of
>  security vulnerabilities are not exploitable easily, or at all,
>  therefore are low-severity, which greatly limits maintenance work
>  required to address them. For example, if upstream tests are ran on an
>  obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  but I can package old Tomcat and run the tests. 
  Whoha! Let's step back for a minute and look at this example.
What are the reasons to run tests?  To make sure the package will run
correctly..
Why run tests on 12-years old version instead of on current one?
Probably because tests fail on current version?

  Will the end user run the package on obsolete Tomcat or on the current one?
Of course on the current one. The one on which tests fail.
Tests in this case are worthless, they are not testing the real
situation. Disabling tests is no worse than testing on obsolete version.
Relying on such tests is a disservice for the end user.

  The point of testing is to validate code in real situation. 
Crafting special, unrealistic environment (12 years old Tomcat) just to have
test pass is missing the point of tests.

-- 
Tomasz Torcz               There exists no separation between gods and men:
tomek(a)pipebreaker.pl       one blends softly casual into the other.  — Frank Herbert

[Mat Booth]

On Fri, 11 Sep 2020 at 09:54, Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:
>  On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  > You get a side tag in Koji where you can have private build-only
>  > dependencies that are discarded (filtered) once they are no longer
>  > needed, after module build is done. For build-only packages most of
>  > security vulnerabilities are not exploitable easily, or at all,
>  > therefore are low-severity, which greatly limits maintenance work
>  > required to address them. For example, if upstream tests are ran on an
>  > obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  > but I can package old Tomcat and run the tests.
>
>    Whoha! Let's step back for a minute and look at this example.
>  What are the reasons to run tests?  To make sure the package will run
>  correctly..
>  Why run tests on 12-years old version instead of on current one?
>  Probably because tests fail on current version? 
No, not at all. It's not that the tests fail on newer versions (it
probably just needs a servlet container environment) but that this is
the version of tomcat that was current when the test fixtures were
written. Tomcats' changing embedding APIs in no way invalidate tests
performed in a standards-compliant servlet container environment.

>    Will the end user run the package on obsolete Tomcat or on the current one?
>  Of course on the current one. The one on which tests fail.
>  Tests in this case are worthless, they are not testing the real
>  situation. Disabling tests is no worse than testing on obsolete version.
>  Relying on such tests is a disservice for the end user.
>
>    The point of testing is to validate code in real situation.
>  Crafting special, unrealistic environment (12 years old Tomcat) just to have
>  test pass is missing the point of tests.
>
>  --
>  Tomasz Torcz               There exists no separation between gods and men:
>  tomek(a)pipebreaker.pl       one blends softly casual into the other.  — Frank Herbert
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 


-- 
Mat Booth
http://fedoraproject.org/get-fedora

[Aleksandar Kurtakov]

On Fri, Sep 11, 2020 at 11:54 AM Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:

>  On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  > You get a side tag in Koji where you can have private build-only
>  > dependencies that are discarded (filtered) once they are no longer
>  > needed, after module build is done. For build-only packages most of
>  > security vulnerabilities are not exploitable easily, or at all,
>  > therefore are low-severity, which greatly limits maintenance work
>  > required to address them. For example, if upstream tests are ran on an
>  > obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  > but I can package old Tomcat and run the tests.
>
>    Whoha! Let's step back for a minute and look at this example.
>  What are the reasons to run tests?  To make sure the package will run
>  correctly..
>  Why run tests on 12-years old version instead of on current one?
>  Probably because tests fail on current version?
>
>    Will the end user run the package on obsolete Tomcat or on the current
>  one?
>  Of course on the current one. The one on which tests fail.
>  Tests in this case are worthless, they are not testing the real
>  situation. Disabling tests is no worse than testing on obsolete version.
>  Relying on such tests is a disservice for the end user.
>
>    The point of testing is to validate code in real situation.
>  Crafting special, unrealistic environment (12 years old Tomcat) just to
>  have
>  test pass is missing the point of tests.
>  
Well, you do realize how much not feasible it is for the Fedora maintainer
of hundred packages to go out and not only fix the builds, bugs, CVEs and
etc. but even the tests for all these packages. The unrealistic
expectations for what package maintainer should do is really not helping
growing that number.


>  --
>  Tomasz Torcz               There exists no separation between gods and men:
>  tomek(a)pipebreaker.pl       one blends softly casual into the other.  —
>  Frank Herbert
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Alexander Kurtakov
Red Hat Eclipse Team

[Adam Williamson]

On Fri, 2020-09-11 at 12:44 +0300, Aleksandar Kurtakov wrote:
>  On Fri, Sep 11, 2020 at 11:54 AM Tomasz Torcz
> <tomek(a)pipebreaker.pl&gt; wrote:
>  
>  > On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  > > You get a side tag in Koji where you can have private build-only
>  > > dependencies that are discarded (filtered) once they are no longer
>  > > needed, after module build is done. For build-only packages most of
>  > > security vulnerabilities are not exploitable easily, or at all,
>  > > therefore are low-severity, which greatly limits maintenance work
>  > > required to address them. For example, if upstream tests are ran on an
>  > > obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  > > but I can package old Tomcat and run the tests.
>  > 
>  >   Whoha! Let's step back for a minute and look at this example.
>  > What are the reasons to run tests?  To make sure the package will run
>  > correctly..
>  > Why run tests on 12-years old version instead of on current one?
>  > Probably because tests fail on current version?
>  > 
>  >   Will the end user run the package on obsolete Tomcat or on the current
>  > one?
>  > Of course on the current one. The one on which tests fail.
>  > Tests in this case are worthless, they are not testing the real
>  > situation. Disabling tests is no worse than testing on obsolete version.
>  > Relying on such tests is a disservice for the end user.
>  > 
>  >   The point of testing is to validate code in real situation.
>  > Crafting special, unrealistic environment (12 years old Tomcat) just to
>  > have
>  > test pass is missing the point of tests.
>  > 
>  
>  Well, you do realize how much not feasible it is for the Fedora maintainer
>  of hundred packages to go out and not only fix the builds, bugs, CVEs and
>  etc. but even the tests for all these packages. The unrealistic
>  expectations for what package maintainer should do is really not helping
>  growing that number. 
I've been struggling with how to make this point without sounding
cynical, but hey, let's just lay it out there:

If one of the issues here can be stated as "we want buildroot-only
packages because we don't want to maintain those packages to a high
standard", it is demonstrably a viable choice within Fedora to just
*not maintain those packages to a high standard*. Maybe we wish it
wasn't the case, but this is a thing that happens all the time. We have
lots of packages that are, by any reasonable standard, rather badly
maintained. It would be lovely if all Fedora packages had all their
bugs aggressively addressed and had great test suites and had CVE
issues jumped on immediately. But, well, the evidence suggests quite
strongly that this is not the case. For e.g., I did a very sloppy
search yesterday for Fedora bugs with "CVE" in the topic which were
opened before June 2 and are still open:

https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASS...

it is capped at 1000 results. Which, you know, doesn't *scream* that
all our packages are being diligently maintained from a CVE point of
view, which implies at least some of them aren't being diligently
maintained from any point of view. Even if some or many of those CVEs
happen to have been fixed by version updates and it's just that the
bugs weren't closed - the very fact that the bugs aren't being handled
is a maintenance issue.

So here is my cynical point: if you want to have poorly-maintained
packages, isn't it still better that they be poorly-maintained distro
packages where at least everyone understands how the process works and
proven packagers can fix stuff up if they are inclined to, than that
they be hidden buildroot-only poorly maintained packages?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Christopher]

On Fri, Sep 11, 2020 at 2:02 PM Adam Williamson
<adamwill(a)fedoraproject.org&gt; wrote:
>  On Fri, 2020-09-11 at 12:44 +0300, Aleksandar Kurtakov wrote:
>  > On Fri, Sep 11, 2020 at 11:54 AM Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:
>  >
>  > > On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  > > > You get a side tag in Koji where you can have private build-only
>  > > > dependencies that are discarded (filtered) once they are no longer
>  > > > needed, after module build is done. For build-only packages most of
>  > > > security vulnerabilities are not exploitable easily, or at all,
>  > > > therefore are low-severity, which greatly limits maintenance work
>  > > > required to address them. For example, if upstream tests are ran on an
>  > > > obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  > > > but I can package old Tomcat and run the tests.
>  > >
>  > >   Whoha! Let's step back for a minute and look at this example.
>  > > What are the reasons to run tests?  To make sure the package will run
>  > > correctly..
>  > > Why run tests on 12-years old version instead of on current one?
>  > > Probably because tests fail on current version?
>  > >
>  > >   Will the end user run the package on obsolete Tomcat or on the current
>  > > one?
>  > > Of course on the current one. The one on which tests fail.
>  > > Tests in this case are worthless, they are not testing the real
>  > > situation. Disabling tests is no worse than testing on obsolete version.
>  > > Relying on such tests is a disservice for the end user.
>  > >
>  > >   The point of testing is to validate code in real situation.
>  > > Crafting special, unrealistic environment (12 years old Tomcat) just to
>  > > have
>  > > test pass is missing the point of tests.
>  > >
>  >
>  > Well, you do realize how much not feasible it is for the Fedora maintainer
>  > of hundred packages to go out and not only fix the builds, bugs, CVEs and
>  > etc. but even the tests for all these packages. The unrealistic
>  > expectations for what package maintainer should do is really not helping
>  > growing that number.
>
>  I've been struggling with how to make this point without sounding
>  cynical, but hey, let's just lay it out there:
>
>  If one of the issues here can be stated as "we want buildroot-only
>  packages because we don't want to maintain those packages to a high
>  standard", it is demonstrably a viable choice within Fedora to just
>  *not maintain those packages to a high standard*. Maybe we wish it
>  wasn't the case, but this is a thing that happens all the time. We have
>  lots of packages that are, by any reasonable standard, rather badly
>  maintained. It would be lovely if all Fedora packages had all their
>  bugs aggressively addressed and had great test suites and had CVE
>  issues jumped on immediately. But, well, the evidence suggests quite
>  strongly that this is not the case. For e.g., I did a very sloppy
>  search yesterday for Fedora bugs with "CVE" in the topic which were
>  opened before June 2 and are still open:
>
>
> https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASS...
>
>  it is capped at 1000 results. Which, you know, doesn't *scream* that
>  all our packages are being diligently maintained from a CVE point of
>  view, which implies at least some of them aren't being diligently
>  maintained from any point of view. Even if some or many of those CVEs
>  happen to have been fixed by version updates and it's just that the
>  bugs weren't closed - the very fact that the bugs aren't being handled
>  is a maintenance issue.
>
>  So here is my cynical point: if you want to have poorly-maintained
>  packages, isn't it still better that they be poorly-maintained distro
>  packages where at least everyone understands how the process works and
>  proven packagers can fix stuff up if they are inclined to, than that
>  they be hidden buildroot-only poorly maintained packages? 
Thank you for making these points. These are my thoughts *exactly*. As
a newcomer to packaging, I was willing to start helping out with those
packages that I depended on, to bring them up to speed, and it was
relatively easy to take on a few, because everything was out in the
open. But once things started getting retired (at a pace I couldn't
keep up with), even though they were still technically maintained in
some hidden world that I didn't understand, I was completely turned
off from helping out with more of the dependencies that my Java
projects required and had to retire my Java packages as well.

I would go further to suggest that poorly-maintained libs like these
are perfect opportunities for mentoring new people and community
growth.
FWIW, every year in October, for the past several years, Digital Ocean
and GitHub have teamed up to sponsor "Hacktoberfest" (I've
participated, but am not affiliated with either of them at all).
Projects label their issues with a "Hacktoberfest" or "Good first
issue" or "Help Wanted" label, and people from all over the world
contribute to projects with such labels... projects they've never
contributed to before... it's a great outreach program for open
source. I understand the reasons against using GitHub for our package
sources, but imagine if we took our poorly-maintained projects, and
instead of retiring them, we used them for such opportunities? Imagine
how much more awesome Fedora could be! I'd much rather have a bunch of
outdated Java libs laying around in a world where such opportunities
existed with a low bar to entry, than to have a smaller set of
up-to-date packages with fewer such opportunities, and a higher bar to
entry.

> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Matthew Miller]

On Fri, Sep 11, 2020 at 11:01:00AM -0700, Adam Williamson wrote:
>  If one of the issues here can be stated as "we want
> buildroot-only
>  packages because we don't want to maintain those packages to a high
>  standard", it is demonstrably a viable choice within Fedora to just
>  *not maintain those packages to a high standard*. Maybe we wish it
>  wasn't the case, but this is a thing that happens all the time. We have 
YES. In fact, *labeling* this is explicitly one of the things I wanted from
modularity. I have a slide about this in one of my talks even, although I
can't find it right now. The upshot is: packagers care about the software
they want to run, and package up and maintain deps either because they want
to do the right thing and be helpful -- or because they have to.

I mean, some of y'all like to maintain and keep obscure dependency packages
up to date just for their own sake, and that's *awesome* -- but we just
can't hold everyone to that standard. At least, not if we want more than a
few dozen packagers. So the *idea* was that modularity would let anyone
express "I need these packages as dependencies, but I don't have the cycles
to maintain them" -- not because that's an awesome situation, but because
it's the reality and the status quo for a lot of things.

Sooooo: RH Java packagers, what if you build these packages as non-modular
(maybe using some scripting to make it happen at the same time as modular
builds?) and add a readme explaining their maintenance state? I think that'd
be preferable to where we are now, and it gets us to the next thing:

* you could still help update and maintain these as time and inclination
  allows without feeling pressure, and... 

* rather than needing to do duplicative work all alone, the stewardship SIG
  could focus on serious security issues and high-priority bugs in this
  package set.

That way, the application ecosystem would be available, the build deps would
be there, and, actually, because of the collaboration, you wouldn't need to
feel guilty about package maintenance state.


What am I missing with this?


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Stephen John Smoogen]

On Thu, 17 Sep 2020 at 12:53, Matthew Miller <mattdm(a)fedoraproject.org&gt;
wrote:

>  On Fri, Sep 11, 2020 at 11:01:00AM -0700, Adam Williamson wrote:
>  > If one of the issues here can be stated as "we want buildroot-only
>  > packages because we don't want to maintain those packages to a high
>  > standard", it is demonstrably a viable choice within Fedora to just
>  > *not maintain those packages to a high standard*. Maybe we wish it
>  > wasn't the case, but this is a thing that happens all the time. We have
>
>  YES. In fact, *labeling* this is explicitly one of the things I wanted from
>  modularity. I have a slide about this in one of my talks even, although I
>  can't find it right now. The upshot is: packagers care about the software
>  they want to run, and package up and maintain deps either because they want
>  to do the right thing and be helpful -- or because they have to.
>
>  I mean, some of y'all like to maintain and keep obscure dependency packages
>  up to date just for their own sake, and that's *awesome* -- but we just
>  can't hold everyone to that standard. At least, not if we want more than a
>  few dozen packagers. So the *idea* was that modularity would let anyone
>  express "I need these packages as dependencies, but I don't have the cycles
>  to maintain them" -- not because that's an awesome situation, but because
>  it's the reality and the status quo for a lot of things.
>
>  Sooooo: RH Java packagers, what if you build these packages as non-modular
>  (maybe using some scripting to make it happen at the same time as modular
>  builds?) and add a readme explaining their maintenance state? I think
>  that'd
>  be preferable to where we are now, and it gets us to the next thing:
>
>  * you could still help update and maintain these as time and inclination
>    allows without feeling pressure, and...
>
>  * rather than needing to do duplicative work all alone, the stewardship SIG
>    could focus on serious security issues and high-priority bugs in this
>    package set.
>
>  That way, the application ecosystem would be available, the build deps
>  would
>  be there, and, actually, because of the collaboration, you wouldn't need to
>  feel guilty about package maintenance state.
>
>
>  What am I missing with this?
>
>
>  
Those packages get bugs and bugzilla is a monkey on the back of every
packager.. having a ton of packages which you know you aren't going to fix
but are still going to get bugs on with the conversation going like:
'its broke'
'yes I know its broke.. I just need the header files'
'well I need it to work' 'well fix it yourself'
'no that is your job.. it says you OWN THE PACKAGE'.
'I just own it to build foobar'
'too bad.. i am taking this to lwn/slashdot/twitter/reddit'

I think that if we are going to have 'extra' packages which are needed for
'core' packages, we should be explicit and own the fact that this 'package
everything for everyone' only worked when working on Operating Systems was
'hot' and has been replaced with cloud, containers, and whatever other
sugar coated candy cereal people have to have with a toy inside these days.


-- 
Stephen J Smoogen.

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Matthew Miller]

On Thu, Sep 17, 2020 at 01:30:26PM -0400, Stephen John Smoogen wrote:
>  'its broke'
>  'yes I know its broke.. I just need the header files'
>  'well I need it to work' 'well fix it yourself'
>  'no that is your job.. it says you OWN THE PACKAGE'.
>  'I just own it to build foobar'
>  'too bad.. i am taking this to lwn/slashdot/twitter/reddit' 
Set the owner to an alias with an explanatory name, auto-reply to bugs and
email with a bot message that explains the situation and links to more
information for people who want to help. And then if that gets taken to LWN,
*fine*.

>  I think that if we are going to have 'extra' packages which
> are needed for
>  'core' packages, we should be explicit and own the fact that this 'package
Or some sort of ... modular ... composition system for defining and building
packages in this way? :)

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Jeff Fearn]

On 18/9/20 03:43, Matthew Miller wrote:
>  On Thu, Sep 17, 2020 at 01:30:26PM -0400, Stephen John Smoogen
> wrote:
> > 'its broke'
> > 'yes I know its broke.. I just need the header files'
> > 'well I need it to work' 'well fix it yourself'
> > 'no that is your job.. it says you OWN THE PACKAGE'.
> > 'I just own it to build foobar'
> > 'too bad.. i am taking this to lwn/slashdot/twitter/reddit'
>  
>  Set the owner to an alias with an explanatory name, auto-reply to bugs and
>  email with a bot message that explains the situation and links to more
>  information for people who want to help. And then if that gets taken to LWN,
>  *fine*. 
IMO this should be done before the bug is created. You can set the
default bug description template for the component to a template with
this message in it.

It probably needs to be reflected in other UIs related to Fedora
packaging, so it should be tracked somewhere and when the fedora bot
updates the component details in Bugzilla it could change the default
description template to match the update policy.

There might have more than 2 states for this.

Cheers, Jeff.

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Adam Williamson]

On Thu, 2020-09-17 at 13:30 -0400, Stephen John Smoogen wrote:
>  On Thu, 17 Sep 2020 at 12:53, Matthew Miller
> <mattdm(a)fedoraproject.org&gt;
>  wrote:
>  
>  > On Fri, Sep 11, 2020 at 11:01:00AM -0700, Adam Williamson wrote:
>  > > If one of the issues here can be stated as "we want buildroot-only
>  > > packages because we don't want to maintain those packages to a high
>  > > standard", it is demonstrably a viable choice within Fedora to just
>  > > *not maintain those packages to a high standard*. Maybe we wish it
>  > > wasn't the case, but this is a thing that happens all the time. We have
>  > 
>  > YES. In fact, *labeling* this is explicitly one of the things I wanted from
>  > modularity. I have a slide about this in one of my talks even, although I
>  > can't find it right now. The upshot is: packagers care about the software
>  > they want to run, and package up and maintain deps either because they want
>  > to do the right thing and be helpful -- or because they have to.
>  > 
>  > I mean, some of y'all like to maintain and keep obscure dependency packages
>  > up to date just for their own sake, and that's *awesome* -- but we just
>  > can't hold everyone to that standard. At least, not if we want more than a
>  > few dozen packagers. So the *idea* was that modularity would let anyone
>  > express "I need these packages as dependencies, but I don't have the
> cycles
>  > to maintain them" -- not because that's an awesome situation, but because
>  > it's the reality and the status quo for a lot of things.
>  > 
>  > Sooooo: RH Java packagers, what if you build these packages as non-modular
>  > (maybe using some scripting to make it happen at the same time as modular
>  > builds?) and add a readme explaining their maintenance state? I think
>  > that'd
>  > be preferable to where we are now, and it gets us to the next thing:
>  > 
>  > * you could still help update and maintain these as time and inclination
>  >   allows without feeling pressure, and...
>  > 
>  > * rather than needing to do duplicative work all alone, the stewardship SIG
>  >   could focus on serious security issues and high-priority bugs in this
>  >   package set.
>  > 
>  > That way, the application ecosystem would be available, the build deps
>  > would
>  > be there, and, actually, because of the collaboration, you wouldn't need to
>  > feel guilty about package maintenance state.
>  > 
>  > 
>  > What am I missing with this?
>  > 
>  > 
>  > 
>  Those packages get bugs and bugzilla is a monkey on the back of every
>  packager.. having a ton of packages which you know you aren't going to fix
>  but are still going to get bugs on with the conversation going like:
>  'its broke'
>  'yes I know its broke.. I just need the header files'
>  'well I need it to work' 'well fix it yourself'
>  'no that is your job.. it says you OWN THE PACKAGE'.
>  'I just own it to build foobar'
>  'too bad.. i am taking this to lwn/slashdot/twitter/reddit' 
The alternative to this is:

'its broke'
<silence>

This accounts for about 50% of our bugs at any given time.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Kevin Kofler]

Matthew Miller wrote:
>  I mean, some of y'all like to maintain and keep obscure
> dependency
>  packages up to date just for their own sake, and that's *awesome* -- but
>  we just can't hold everyone to that standard. At least, not if we want
>  more than a few dozen packagers. 
That is a pretty strong assertion, with no facts to back it.

I think that packagers are more likely to be driven away by changes such as 
Modularity that make packaging more and more complex, or even by the very 
fact that the dependencies they need were taken private by their maintainer, 
than by the very reasonable minimum standard we want to hold our packagers 
to.

A private dependency is necessarily a form of technical debt that is 
inevitably going to hurt us as soon as we want to package something else 
that happens to depend on that same dependency. (Unless of course the 
dependency really cannot be used for anything else than building that one 
dependent package, in which case nobody is going to expect the maintainer to 
make it work for anything else even if it is public, so that trivial case 
need not be considered any further.) Hence, this risks driving away the 
people who would be packaging the other dependent packages if the dependency 
was readily available to them.

And that is just the impact on packagers. You also have to consider the 
impact on end users. Do not forget that most packagers were users before 
becoming packagers. Hiding packages from users or labeling them as 
officially unsupported is going to make Fedora less attractive to users, 
which in turn will lead to fewer people potentially becoming packagers.

        Kevin Kofler

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Zbigniew Jędrzejewski-Szmek]

On Fri, Sep 18, 2020 at 02:16:11AM +0200, Kevin Kofler wrote:
>  Matthew Miller wrote:
>  > I mean, some of y'all like to maintain and keep obscure dependency
>  > packages up to date just for their own sake, and that's *awesome* -- but
>  > we just can't hold everyone to that standard. At least, not if we want
>  > more than a few dozen packagers.
>  
>  That is a pretty strong assertion, with no facts to back it.
>  
>  I think that packagers are more likely to be driven away by changes such as 
>  Modularity that make packaging more and more complex, or even by the very 
>  fact that the dependencies they need were taken private by their maintainer, 
>  than by the very reasonable minimum standard we want to hold our packagers 
>  to.
>  
>  A private dependency is necessarily a form of technical debt that is 
>  inevitably going to hurt us as soon as we want to package something else 
>  that happens to depend on that same dependency. (Unless of course the 
>  dependency really cannot be used for anything else than building that one 
>  dependent package, in which case nobody is going to expect the maintainer to 
>  make it work for anything else even if it is public, so that trivial case 
>  need not be considered any further.) Hence, this risks driving away the 
>  people who would be packaging the other dependent packages if the dependency 
>  was readily available to them. 
Fully agreed. The problem with modularized packages is that there is
no smooth transition between the states. A normal package, even if not maintained
fully, can be picked up by anyone at any time. Even by a non-packager: any user
can open a PR in pagure to fix some particularly annoying issue.
But once modularized as build-only, the package is not visible to users, is not
visible to other packagers, possibly loses out on any distro-wide mass changes,
and most importantly, trying to take it back out of the module is not smooth
at all.

>  And that is just the impact on packagers. You also have to consider
> the 
>  impact on end users. Do not forget that most packagers were users before 
>  becoming packagers. Hiding packages from users or labeling them as 
>  officially unsupported is going to make Fedora less attractive to users, 
>  which in turn will lead to fewer people potentially becoming packagers. 
I partially disagree about the "labeling them as officially
unsupported is going to make Fedora less attractive to users". If a package
*is* not fully supported, then I don't see users should not know this. In fact,
if I were to install a package as a user and see a message like
"Sorry, we don't have enough round tuits to handle bugs in this package.
Type «I understand» to proceed." then it could make an easy entry point for new
packagers.

Zbyszek

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Petr Pisar]

On Thu, Sep 17, 2020 at 12:51:46PM -0400, Matthew Miller wrote:
>  Sooooo: RH Java packagers, what if you build these packages as
> non-modular
>  (maybe using some scripting to make it happen at the same time as modular
>  builds?) and add a readme explaining their maintenance state? 
Do you mean literally a README file under /usr/share/doc/${PACAKGE}/
directory? That won't work, because nobody looks there.

Either the package should be moved into a separate repository that an end user
have to explicitly enable and later can inspect an origin of the package with
"dnf info $PACKAGE", or the package should not be distributed at all. Pungi
tool that produces the distribution composes supports it. It's only a matter
of configuration.

(Or we can invent a new RPM tag for a support level of the package.)

-- Petr

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Dominik 'Rathann' Mierzejewski]

On Friday, 18 September 2020 at 11:22, Petr Pisar wrote:
>  On Thu, Sep 17, 2020 at 12:51:46PM -0400, Matthew Miller wrote:
>  > Sooooo: RH Java packagers, what if you build these packages as non-modular
>  > (maybe using some scripting to make it happen at the same time as modular
>  > builds?) and add a readme explaining their maintenance state?
>  
>  Do you mean literally a README file under /usr/share/doc/${PACAKGE}/
>  directory? That won't work, because nobody looks there. 
I do. :)

What about https://src.fedoraproject.org/rpms/<name>/blob/master/f/README.md ?

Regards,
Dominik
-- 
Fedora   https://getfedora.org  |  RPM Fusion  http://rpmfusion.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
        -- from "Collected Sayings of Muad'Dib" by the Princess Irulan

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Petr Pisar]

On Fri, Sep 18, 2020 at 12:21:15PM +0200, Dominik 'Rathann' Mierzejewski wrote:
>  On Friday, 18 September 2020 at 11:22, Petr Pisar wrote:
>  > On Thu, Sep 17, 2020 at 12:51:46PM -0400, Matthew Miller wrote:
>  > > Sooooo: RH Java packagers, what if you build these packages as non-modular
>  > > (maybe using some scripting to make it happen at the same time as modular
>  > > builds?) and add a readme explaining their maintenance state?
>  > 
>  > Do you mean literally a README file under /usr/share/doc/${PACAKGE}/
>  > directory? That won't work, because nobody looks there.
>  
>  I do. :)
>  
>  What about https://src.fedoraproject.org/rpms/<name>/blob/master/f/README.md ?
>   
Do you think that package users consult dist-git before reporting a bug?
I think the idea with a template in Bugzilla was better.

Moreover that file was supposed to display a source package description on all
possible places (and originally was supposed to automatically updated from SRPM
packages, but that was never implemented).

At the end a packager can put the notice into %description of the spec file.
I sometimes document there how the files are distributed among the subpcackages.

-- Petr

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[David Cantrell]

On Thu, Sep 17, 2020 at 12:51:46PM -0400, Matthew Miller wrote:
> On Fri, Sep 11, 2020 at 11:01:00AM -0700, Adam Williamson wrote:
> > If one of the issues here can be stated as "we want buildroot-only
> > packages because we don't want to maintain those packages to a high
> > standard", it is demonstrably a viable choice within Fedora to just
> > *not maintain those packages to a high standard*. Maybe we wish it
> > wasn't the case, but this is a thing that happens all the time. We have
>
> YES. In fact, *labeling* this is explicitly one of the things I wanted from
> modularity. I have a slide about this in one of my talks even, although I
> can't find it right now. The upshot is: packagers care about the software
> they want to run, and package up and maintain deps either because they want
> to do the right thing and be helpful -- or because they have to.
>
> I mean, some of y'all like to maintain and keep obscure dependency packages
> up to date just for their own sake, and that's *awesome* -- but we just
> can't hold everyone to that standard. At least, not if we want more than a
> few dozen packagers. So the *idea* was that modularity would let anyone
> express "I need these packages as dependencies, but I don't have the cycles
> to maintain them" -- not because that's an awesome situation, but because
> it's the reality and the status quo for a lot of things. 
Burying build-only dependencies in a module adds more work in the long
run.  If something-doc-tool is a build dep for a module and is now
part of that module, it's visibility is decreased to other package
maintainers that may want to use it.  Being able to find it now puts
the burden of having detailed knowledge of every module in the
distribution as well as the distribution itself on all package
maintainers.  And then if they find it, how can they use it since it's
part of a module as a build only dep.  Can it become part of another
module?  Say it can, now work has to be coordinated among the package
maintainers using something-doc-tool as a build dep so they don't
duplicate work.  The net gain here is that we have avoiding exposing
something-doc-tool as a package that users can install?  Why?

As stated elsewhere on the thread, to achieve what you're describing
is much easier in other ways.  I don't know why we haven't explored
creating an additional repo of shared build deps or even just "rawer"
packages that maybe don't get the maintenance attention that other
packages do.  We already know that goes on, but the packages are just
part of the main repo.  So instead of classifying them collectively
and keeping them open for shared and collaborative maintenance, we
want to make them hidden build deps in modules?

New community members looking to become package maintainers... if we
had a grouping of packages in need of care that were only build deps
for other packages, the barrier to entry is less daunting if you know
that you won't have a million users.

TL;DR modularity decreases component visibility and works against open
collaboration.

> Sooooo: RH Java packagers, what if you build these packages as
> non-modular
> (maybe using some scripting to make it happen at the same time as modular
> builds?) and add a readme explaining their maintenance state? I think that'd
> be preferable to where we are now, and it gets us to the next thing:
>
> * you could still help update and maintain these as time and inclination
>   allows without feeling pressure, and...
>
> * rather than needing to do duplicative work all alone, the stewardship SIG
>   could focus on serious security issues and high-priority bugs in this
>   package set.
>
> That way, the application ecosystem would be available, the build deps would
> be there, and, actually, because of the collaboration, you wouldn't need to
> feel guilty about package maintenance state.
>
>
> What am I missing with this? 
I think this is on the right path.  Shared ownership and collaboration
is a good way forward for these types of packages.

If I am using a package as a build dep and find it's outdated or
broken in some way, I can fix it and send a PR in Pagure to update it.
The maintainer can review and merge that or at least start a
discussion with me on the why and how of the change.  I haven't had to
take complete ownership of that package, but just spent the time to
fix the problem I saw.  For packages in a less than bleeding edge
ultra stable state, my main concern is that they are functional rather
than latest version.  I suspect many of us also prefer that for build
deps in most cases.

Maybe in addition to a per-package README, a broader communication of
this process to the community as well as providing a link to open bug
queries for packages would be useful for work similar to the "kernel
janitors" type work for the kernel.  Lots of stuff has to be done on
an ongoing basis and everyone can chip in from time to time.  We could
consider a standard doc name like "MAINT" or something in the package
that describes the specifics regarding maintenance in Fedora.

Thanks,

-- 
David Cantrell <dcantrell(a)redhat.com&gt;
Red Hat, Inc. | Boston, MA | EST5EDT

maybe a path forward for java and modules? [was Re: The Future of the Java Stack (also regarding ELN and RHEL)]

[Fabio Valentini]

On Fri, Sep 18, 2020 at 5:30 PM David Cantrell <dcantrell(a)redhat.com&gt; wrote:
>  On Thu, Sep 17, 2020 at 12:51:46PM -0400, Matthew Miller wrote:
>  >On Fri, Sep 11, 2020 at 11:01:00AM -0700, Adam Williamson wrote:
>  >> If one of the issues here can be stated as "we want buildroot-only
>  >> packages because we don't want to maintain those packages to a high
>  >> standard", it is demonstrably a viable choice within Fedora to just
>  >> *not maintain those packages to a high standard*. Maybe we wish it
>  >> wasn't the case, but this is a thing that happens all the time. We have
>  >
>  >YES. In fact, *labeling* this is explicitly one of the things I wanted from
>  >modularity. I have a slide about this in one of my talks even, although I
>  >can't find it right now. The upshot is: packagers care about the software
>  >they want to run, and package up and maintain deps either because they want
>  >to do the right thing and be helpful -- or because they have to.
>  >
>  >I mean, some of y'all like to maintain and keep obscure dependency packages
>  >up to date just for their own sake, and that's *awesome* -- but we just
>  >can't hold everyone to that standard. At least, not if we want more than a
>  >few dozen packagers. So the *idea* was that modularity would let anyone
>  >express "I need these packages as dependencies, but I don't have the cycles
>  >to maintain them" -- not because that's an awesome situation, but because
>  >it's the reality and the status quo for a lot of things.
>
>  Burying build-only dependencies in a module adds more work in the long
>  run.  If something-doc-tool is a build dep for a module and is now
>  part of that module, it's visibility is decreased to other package
>  maintainers that may want to use it.  Being able to find it now puts
>  the burden of having detailed knowledge of every module in the
>  distribution as well as the distribution itself on all package
>  maintainers.  And then if they find it, how can they use it since it's
>  part of a module as a build only dep.  Can it become part of another
>  module?  Say it can, now work has to be coordinated among the package
>  maintainers using something-doc-tool as a build dep so they don't
>  duplicate work.  The net gain here is that we have avoiding exposing
>  something-doc-tool as a package that users can install?  Why?
>
>  As stated elsewhere on the thread, to achieve what you're describing
>  is much easier in other ways.  I don't know why we haven't explored
>  creating an additional repo of shared build deps or even just "rawer"
>  packages that maybe don't get the maintenance attention that other
>  packages do.  We already know that goes on, but the packages are just
>  part of the main repo.  So instead of classifying them collectively
>  and keeping them open for shared and collaborative maintenance, we
>  want to make them hidden build deps in modules? 
(snip)

>  New community members looking to become package maintainers... if we
>  had a grouping of packages in need of care that were only build deps
>  for other packages, the barrier to entry is less daunting if you know
>  that you won't have a million users.
>
>  TL;DR modularity decreases component visibility and works against open
>  collaboration. 
I completely agree. The only way to move forward is to make
maintenance of shared parts of the stack a collaborative effort.

>  >Sooooo: RH Java packagers, what if you build these packages as
> non-modular
>  >(maybe using some scripting to make it happen at the same time as modular
>  >builds?) and add a readme explaining their maintenance state? I think that'd
>  >be preferable to where we are now, and it gets us to the next thing:
>  >
>  >* you could still help update and maintain these as time and inclination
>  >  allows without feeling pressure, and...
>  >
>  >* rather than needing to do duplicative work all alone, the stewardship SIG
>  >  could focus on serious security issues and high-priority bugs in this
>  >  package set. 
Just to clarify, the Stewardship SIG now maintains zero packages.
The handful of NodeJS packages we had were either dropped or adopted
by somebody else ages ago.
All Java packages were moved into the "rebirthed" Java SIG.

>  >That way, the application ecosystem would be available, the build
> deps would
>  >be there, and, actually, because of the collaboration, you wouldn't need to
>  >feel guilty about package maintenance state.
>  >
>  >
>  >What am I missing with this?
>
>  I think this is on the right path.  Shared ownership and collaboration
>  is a good way forward for these types of packages. 
Yes. That's kinda the reason I "restarted" the effort for a Java SIG?

The initial progress (then still under the Stewardship SIG umbrella)
was slow - because almost everything was old and broken - but now, the
Java stack is in *much better shape*. Arguably, it is in better shape
than it had been for years, being ~75% up-to-date with upstreams now,
with *zero* FTBFS issues for f32+, and *zero* open CVEs in the core
stack.

We already did the hard work of keeping everything from exploding. Now
it's mostly back to normal, boring package maintenance. This is also
conducive for getting new people involved with the Java SIG. The
barrier of entry is much lower now, since not everything is broken
left and right, but actually surprisingly pleasant to work with.
Making incremental changes and updates is much easier when the stack
is in good shape ...

I've also been seeing an increasing number of contributions (and
contributors) over the past months. It turns out, some of our packages
are depended on by basically the entire distro (including libreoffice,
FreeIPA, libvirt, qemu, etc.) so packagers are actually interested in
keeping things working, now that the stack is not threatening to
implode any day.

Fabio

> If I am using a package as a build dep and find it's outdated or
> broken in some way, I can fix it and send a PR in Pagure to update it.
> The maintainer can review and merge that or at least start a
> discussion with me on the why and how of the change.  I haven't had to
> take complete ownership of that package, but just spent the time to
> fix the problem I saw.  For packages in a less than bleeding edge
> ultra stable state, my main concern is that they are functional rather
> than latest version.  I suspect many of us also prefer that for build
> deps in most cases.
>
> Maybe in addition to a per-package README, a broader communication of
> this process to the community as well as providing a link to open bug
> queries for packages would be useful for work similar to the "kernel
> janitors" type work for the kernel.  Lots of stuff has to be done on
> an ongoing basis and everyone can chip in from time to time.  We could
> consider a standard doc name like "MAINT" or something in the package
> that describes the specifics regarding maintenance in Fedora.
>
> Thanks,
>
> --
> David Cantrell <dcantrell(a)redhat.com&gt;
> Red Hat, Inc. | Boston, MA | EST5EDT
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Mikolaj Izdebski]

On Fri, Sep 11, 2020 at 10:54 AM Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:
>  On Fri, Sep 11, 2020 at 10:16:02AM +0200, Mikolaj Izdebski wrote:
>  > You get a side tag in Koji where you can have private build-only
>  > dependencies that are discarded (filtered) once they are no longer
>  > needed, after module build is done. For build-only packages most of
>  > security vulnerabilities are not exploitable easily, or at all,
>  > therefore are low-severity, which greatly limits maintenance work
>  > required to address them. For example, if upstream tests are ran on an
>  > obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  > but I can package old Tomcat and run the tests.
>
>    Whoha! Let's step back for a minute and look at this example.
>  What are the reasons to run tests?  To make sure the package will run
>  correctly.. 
To prevent RPM build from succeeding in case bugs are found and
forcing the maintainer to investigate the issue (fix the bug, disable
broken test etc.)

>  Why run tests on 12-years old version instead of on current one?
>  Probably because tests fail on current version? 
Because at the time tests were developed that was the most recent
Tomcat version. Since then upstream did not update the perfectly
working test suite ("If it ain't broke, don't fix it"). They will not
even accept our contribution to update the test suite.

>    Will the end user run the package on obsolete Tomcat or on the
> current one? 
Neither. The package is a HTTP client and tests merely need the server
side (Tomcat).

>  Of course on the current one. The one on which tests fail.
>  Tests in this case are worthless, they are not testing the real
>  situation. Disabling tests is no worse than testing on obsolete version.
>  Relying on such tests is a disservice for the end user.
>
>    The point of testing is to validate code in real situation.
>  Crafting special, unrealistic environment (12 years old Tomcat) just to have
>  test pass is missing the point of tests.
>
>  --
>  Tomasz Torcz               There exists no separation between gods and men:
>  tomek(a)pipebreaker.pl       one blends softly casual into the other.  — Frank Herbert
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Hans de Goede]

Hi,

First of all a big thank you to everyone involved in the
discussion for the constructive discussion.

I agree that the situation around java packaging is quite
worrying and I'm happy to see that people are trying to
come up with a pragmatic solution to the current deadlock
situation.

On 9/11/20 10:16 AM, Mikolaj Izdebski wrote:

<snip>

>  Modularity provides not only a different way of grouping or
> delivering
>  RPM packages, but most importantly, a different way of building RPM
>  packages. 
<snip>

Allow me zoom in on some of the examples you give here. I realize
they are just examples but maybe we can extract some patterns from
them and come up with solutions to allow you to maintain the and
and maven stacks inside the ursine package collection, without
increasing your workload.

>  You get a side tag in Koji where you can have private build-only
>  dependencies that are discarded (filtered) once they are no longer
>  needed, after module build is done. For build-only packages most of
>  security vulnerabilities are not exploitable easily, or at all,
>  therefore are low-severity, which greatly limits maintenance work
>  required to address them. For example, if upstream tests are ran on an
>  obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  but I can package old Tomcat and run the tests. I don't need to update
>  that Tomcat or fix security issues as they are not exploitable in
>  buildroot - Tomcat runs in embedded mode, does not listen on the
>  network etc. Not something that I could with ursine Tomcat packages -
>  it would get delivered to users, and we have no control how they use
>  it - we would need to fix all important user-reported bugs and CVEs as
>  they are potentially exploitable. 
So for this tomcat needed for testing problem, I'm thinking that we
might solve this in a very non Fedora way. Why not bundle the old
tomcat-sources with the sources which need it for their test-suite
(and build it before running the tests).

I realize that this is ugly; and if many packages need this old
tomcat it will not scale (but I hope that is not the case). Despite
those disadvantages I believe that this would be a good solution.

The reason why I'm thinking in this direction is as follows:
Like others, I'm very much against the notion of buildroot only
packages, esp. the side-tag idea is terrible as it makes rebuilding
the package from source very hard for end users. To me Fedora
would not be Fedora if users can easily rebuild their packages.

Bundling the old tomcat sources, so that the test-suite can run,
without that old tomcat ever getting installed on the users
system, to seems like a decent workaround for the issue of the
tests depending on this old tomcat.

>  Another, more concrete example: core Ant doesn't have any
> dependencies
>  beyond JDK. It is easy to build and maintain, yet very functional. On
>  the other hand, full Ant with all the optional tasks depends on more
>  than 200 Java packages. For the purpose of building all packages that
>  I am interested in, core Ant with JUnit tasks (total of 3 packages) is
>  sufficient. Functionalities of Ant like sending emails or image
>  processing are obviously not needed in this case. If building other
>  packages is the only use of Ant then it can be maintained much more
>  easily (3 instead of ~250 packages). But when Ant is ursine package in
>  Fedora then it should be the full Ant - we can't claim to deliver Ant
>  to users while it is just part of it. Eclipse in Fedora requires full
>  Ant too. 
So if you say that you only need the minimal Ant, and I guess many
packages only need the minimal Ant to build, then why not have
an ant-minimal package, like we have a vim-minimal ?

Now I guess that vim-minimal and "proper" vim are both build from the
same source-rpm; and normally we never allow 2 srpms with the same
upstream sources in them. But I think that in this case we could make
an exemption where there is a separate pkg-git and srpm for an ant-minimal
which you would maintain.

And then the community / java-sig can maintain the full Ant as I believe
is already happening since we do have an ant packaged in Fedora 33 atm.

I hope that these 2 examples show that at least in some cases we
may be able to come up with creative solutions for the problems
involved in java packaging.

###

An other more generic approach which has been brought up once or
twice, but which not really has been discussed in much detail yet
I believe is creating a fedora-builddep repository.

ATM a normal user has 3 ursine Fedora repos installed:

fedora
fedora-updates
fedora-updates-testing (disabled by default)

What if we add a 4th repo called fedora-builddep:

fedora
fedora-updates
fedora-updates-testing (disabled by default)
fedora-builddep (rolling (within a release), disabled by default)

So the idea is that all the maven deps which you need, but
do not want to offer any end-user support on would go to
fedora-builddep.

Taking Fedora 33 as an example then:

1) The fedora-33-buildroot pkg-set would consist of fedora-33 + fedora-33-updates +
fedora-33-builddep
2) When you build a newer package in the fedora-33-builddep tag (which uses the
fedora-33-buildroot)
it will automatically get added to the fedora-33-builddep compose right away, that is
what
I mean with fedora-builddep being rolling.

I think that this gives you the builddep only packages which you
want, while still allowing end-users to easily rebuild things
(users only need to enable the -builddep repo).

Then the main thing which we need on top of this is to
very clear communicate what sort of expectations users
can have wrt support and esp. wrt security updates from
packages in the builddep repo and make sure that that
message comes across.

We will of course need to define some procedures
surrounding this:

1. Packages in fedora-x + fedora-x-updates may not have any
    runtime deps on packages in fedora-x-builldep is
    the most obvious ones

2. We will need a small procedure for moving packages
    from fedora-x + fedora-x-updates to fedora-x-builddep,
    so that they get removed by dnf for people who do not
    have fedora-x-builddep enabled. Basically a builddep
    version of the fedora-obosolete-packages metapkg.
    But lets not get too much into details right now.

Mikolaj do think that having such a fedora-builddep repo
(combined with e.g. ant-minimal for the "API packages"
part of things) could work for you ?

Regards,

Hans

[Vít Ondruch]

Dne 11. 09. 20 v 11:03 Hans de Goede napsal(a):
>  On 9/11/20 10:16 AM, Mikolaj Izdebski wrote:
>
> > Another, more concrete example: core Ant doesn't have any dependencies
> > beyond JDK. It is easy to build and maintain, yet very functional. On
> > the other hand, full Ant with all the optional tasks depends on more
> > than 200 Java packages. For the purpose of building all packages that
> > I am interested in, core Ant with JUnit tasks (total of 3 packages) is
> > sufficient. Functionalities of Ant like sending emails or image
> > processing are obviously not needed in this case. If building other
> > packages is the only use of Ant then it can be maintained much more
> > easily (3 instead of ~250 packages). But when Ant is ursine package in
> > Fedora then it should be the full Ant - we can't claim to deliver Ant
> > to users while it is just part of it. Eclipse in Fedora requires full
> > Ant too.
>
>  So if you say that you only need the minimal Ant, and I guess many
>  packages only need the minimal Ant to build, then why not have
>  an ant-minimal package, like we have a vim-minimal ?
>
>  Now I guess that vim-minimal and "proper" vim are both build from the
>  same source-rpm; and normally we never allow 2 srpms with the same
>  upstream sources in them. But I think that in this case we could make
>  an exemption where there is a separate pkg-git and srpm for an
>  ant-minimal
>  which you would maintain.
>
>  And then the community / java-sig can maintain the full Ant as I believe
>  is already happening since we do have an ant packaged in Fedora 33 atm. 

How this reduce the workload? As far as I understand, there is currently
Ant minimal in module and full featured Ant as ursine package and both
companies complain (more or less) that they want to do less.

And of course one of the ideas of modularity were that you have one
repository, actually one branch, and from there you build Ant for module
and the same Ant as a ursine package, but apparently, there are
different needs. Therefore Ant has to be maintained twice.


Vít

[Hans de Goede]

Hi,

On 9/11/20 12:47 PM, Vít Ondruch wrote:
>  
>  Dne 11. 09. 20 v 11:03 Hans de Goede napsal(a):
> >
> >
> > On 9/11/20 10:16 AM, Mikolaj Izdebski wrote:
> >
> >> Another, more concrete example: core Ant doesn't have any dependencies
> >> beyond JDK. It is easy to build and maintain, yet very functional. On
> >> the other hand, full Ant with all the optional tasks depends on more
> >> than 200 Java packages. For the purpose of building all packages that
> >> I am interested in, core Ant with JUnit tasks (total of 3 packages) is
> >> sufficient. Functionalities of Ant like sending emails or image
> >> processing are obviously not needed in this case. If building other
> >> packages is the only use of Ant then it can be maintained much more
> >> easily (3 instead of ~250 packages). But when Ant is ursine package in
> >> Fedora then it should be the full Ant - we can't claim to deliver Ant
> >> to users while it is just part of it. Eclipse in Fedora requires full
> >> Ant too.
> >
> > So if you say that you only need the minimal Ant, and I guess many
> > packages only need the minimal Ant to build, then why not have
> > an ant-minimal package, like we have a vim-minimal ?
> >
> > Now I guess that vim-minimal and "proper" vim are both build from the
> > same source-rpm; and normally we never allow 2 srpms with the same
> > upstream sources in them. But I think that in this case we could make
> > an exemption where there is a separate pkg-git and srpm for an
> > ant-minimal
> > which you would maintain.
> >
> > And then the community / java-sig can maintain the full Ant as I believe
> > is already happening since we do have an ant packaged in Fedora 33 atm.
>  
>  
>  How this reduce the workload? 
It doesn't the purpose of my email was to look for ways to get
the modular ant/maven repos work moved back into ursine without
increasing the workload for the maintainers of those modular
repos.

So the way to look at this, is does it increase workload,
and AFAICT it does not increase workload, while it would allow
us to fold the work being done in the modular repos back
into the ursine repo (for the ant example at least).

Alternatively if we fold that work back into the ursine repos,
then the modular builds could start relying on the full-blown
ant build maintained by the java-SIG in the ursine repos and
drop their own ant-minimal, at which point it would reduce the
workload for the maintainers of the modular repos.

My idea behind having a separate ant-minimal is that they
then can ensure that that is new enough and otherwise matches
their needs without relying on the full-blown version, but
actually somply switching to the full-blown java-SIG maintained
version might be better.

Regards,

Hans


p.s.

I would not mind picking up maintenance of 1 or 2 of the
less frequently changing java packages (*) if that helps.
I wonder how much more people there are like me who care
enough about java to be willing to put some work in
(but not lots of work) ?

Perhaps it would be a good idea for the java-SIG to make
a list of easy to maintain (for an experienced packager)
pkgs which could use a new primary maintainer. I realize
maintaining the entire stack is a team effort, so the rest of
the java-SIG will of course be/stay a co-maintainer of these.
But having things like rebasing to latest upstream, or just
addressing bugzillas taken care of mostly by a new
primary maintainer who is willing to pick up some packages
might help reduce the load ?

The idea here is for someone (e.g.) me to NOT jump in now,
do a bunch of work to help out and then disappear again.
Instead I would invest say 1-2 hours every 2 weeks, which
may not seem much but over time adds up and if we can
get more people to do this, then maybe we can spread the
load of the java packages in the ursine repos better ?



*) But definitely not the entire stack

[Fabio Valentini]

On Fri, Sep 11, 2020 at 1:06 PM Hans de Goede <hdegoede(a)redhat.com&gt; wrote:
>  Hi,
>
>  On 9/11/20 12:47 PM, Vít Ondruch wrote:
>  >
>  > Dne 11. 09. 20 v 11:03 Hans de Goede napsal(a):
>  >>
>  >>
>  >> On 9/11/20 10:16 AM, Mikolaj Izdebski wrote:
>  >>
>  >>> Another, more concrete example: core Ant doesn't have any dependencies
>  >>> beyond JDK. It is easy to build and maintain, yet very functional. On
>  >>> the other hand, full Ant with all the optional tasks depends on more
>  >>> than 200 Java packages. For the purpose of building all packages that
>  >>> I am interested in, core Ant with JUnit tasks (total of 3 packages) is
>  >>> sufficient. Functionalities of Ant like sending emails or image
>  >>> processing are obviously not needed in this case. If building other
>  >>> packages is the only use of Ant then it can be maintained much more
>  >>> easily (3 instead of ~250 packages). But when Ant is ursine package in
>  >>> Fedora then it should be the full Ant - we can't claim to deliver Ant
>  >>> to users while it is just part of it. Eclipse in Fedora requires full
>  >>> Ant too.
>  >>
>  >> So if you say that you only need the minimal Ant, and I guess many
>  >> packages only need the minimal Ant to build, then why not have
>  >> an ant-minimal package, like we have a vim-minimal ?
>  >>
>  >> Now I guess that vim-minimal and "proper" vim are both build from the
>  >> same source-rpm; and normally we never allow 2 srpms with the same
>  >> upstream sources in them. But I think that in this case we could make
>  >> an exemption where there is a separate pkg-git and srpm for an
>  >> ant-minimal
>  >> which you would maintain.
>  >>
>  >> And then the community / java-sig can maintain the full Ant as I believe
>  >> is already happening since we do have an ant packaged in Fedora 33 atm.
>  >
>  >
>  > How this reduce the workload?
>
>  It doesn't the purpose of my email was to look for ways to get
>  the modular ant/maven repos work moved back into ursine without
>  increasing the workload for the maintainers of those modular
>  repos.
>
>  So the way to look at this, is does it increase workload,
>  and AFAICT it does not increase workload, while it would allow
>  us to fold the work being done in the modular repos back
>  into the ursine repo (for the ant example at least).
>
>  Alternatively if we fold that work back into the ursine repos,
>  then the modular builds could start relying on the full-blown
>  ant build maintained by the java-SIG in the ursine repos and
>  drop their own ant-minimal, at which point it would reduce the
>  workload for the maintainers of the modular repos.
>
>  My idea behind having a separate ant-minimal is that they
>  then can ensure that that is new enough and otherwise matches
>  their needs without relying on the full-blown version, but
>  actually somply switching to the full-blown java-SIG maintained
>  version might be better.
>
>  Regards,
>
>  Hans 
(snip)

>  p.s.
>
>  I would not mind picking up maintenance of 1 or 2 of the
>  less frequently changing java packages (*) if that helps.
>  I wonder how much more people there are like me who care
>  enough about java to be willing to put some work in
>  (but not lots of work) ?
>
>  Perhaps it would be a good idea for the java-SIG to make
>  a list of easy to maintain (for an experienced packager)
>  pkgs which could use a new primary maintainer. I realize
>  maintaining the entire stack is a team effort, so the rest of
>  the java-SIG will of course be/stay a co-maintainer of these.
>  But having things like rebasing to latest upstream, or just
>  addressing bugzillas taken care of mostly by a new
>  primary maintainer who is willing to pick up some packages
>  might help reduce the load ?
>
>  The idea here is for someone (e.g.) me to NOT jump in now,
>  do a bunch of work to help out and then disappear again.
>  Instead I would invest say 1-2 hours every 2 weeks, which
>  may not seem much but over time adds up and if we can
>  get more people to do this, then maybe we can spread the
>  load of the java packages in the ursine repos better ? 
That would be great, any help is appreciated.
I can try to come up with a list of often-used-but-simple-to-maintain
Java packages?

Side note: I just checked, and the *whole* Java stack that was
previously maintained by the Stewardship SIG and was now moved into a
dedicated Java SIG group - @java-maint-sig -  has only 2 (in words:
TWO) open CVE bugs associated with it. One was filed against log4j12
(which we're actively trying to get rid of), and the other was filed
against resteasy, where a fix is blocked by getting the package
updated to a newer version (with lots of new dependencies). So I have
no idea where that "big burden of fixing CVE issues" is for Java
packages actually is ...

Fabio

[Hans de Goede]

Hi,

On 9/11/20 6:08 PM, Fabio Valentini wrote:
>  On Fri, Sep 11, 2020 at 1:06 PM Hans de Goede
> <hdegoede(a)redhat.com&gt; wrote:
> >
> > Hi,
> >
> > On 9/11/20 12:47 PM, Vít Ondruch wrote:
> >>
> >> Dne 11. 09. 20 v 11:03 Hans de Goede napsal(a):
> >>>
> >>>
> >>> On 9/11/20 10:16 AM, Mikolaj Izdebski wrote:
> >>>
> >>>> Another, more concrete example: core Ant doesn't have any
> dependencies
> >>>> beyond JDK. It is easy to build and maintain, yet very functional. On
> >>>> the other hand, full Ant with all the optional tasks depends on more
> >>>> than 200 Java packages. For the purpose of building all packages that
> >>>> I am interested in, core Ant with JUnit tasks (total of 3 packages) is
> >>>> sufficient. Functionalities of Ant like sending emails or image
> >>>> processing are obviously not needed in this case. If building other
> >>>> packages is the only use of Ant then it can be maintained much more
> >>>> easily (3 instead of ~250 packages). But when Ant is ursine package in
> >>>> Fedora then it should be the full Ant - we can't claim to deliver
> Ant
> >>>> to users while it is just part of it. Eclipse in Fedora requires full
> >>>> Ant too.
> >>>
> >>> So if you say that you only need the minimal Ant, and I guess many
> >>> packages only need the minimal Ant to build, then why not have
> >>> an ant-minimal package, like we have a vim-minimal ?
> >>>
> >>> Now I guess that vim-minimal and "proper" vim are both build from
> the
> >>> same source-rpm; and normally we never allow 2 srpms with the same
> >>> upstream sources in them. But I think that in this case we could make
> >>> an exemption where there is a separate pkg-git and srpm for an
> >>> ant-minimal
> >>> which you would maintain.
> >>>
> >>> And then the community / java-sig can maintain the full Ant as I believe
> >>> is already happening since we do have an ant packaged in Fedora 33 atm.
> >>
> >>
> >> How this reduce the workload?
> >
> > It doesn't the purpose of my email was to look for ways to get
> > the modular ant/maven repos work moved back into ursine without
> > increasing the workload for the maintainers of those modular
> > repos.
> >
> > So the way to look at this, is does it increase workload,
> > and AFAICT it does not increase workload, while it would allow
> > us to fold the work being done in the modular repos back
> > into the ursine repo (for the ant example at least).
> >
> > Alternatively if we fold that work back into the ursine repos,
> > then the modular builds could start relying on the full-blown
> > ant build maintained by the java-SIG in the ursine repos and
> > drop their own ant-minimal, at which point it would reduce the
> > workload for the maintainers of the modular repos.
> >
> > My idea behind having a separate ant-minimal is that they
> > then can ensure that that is new enough and otherwise matches
> > their needs without relying on the full-blown version, but
> > actually somply switching to the full-blown java-SIG maintained
> > version might be better.
> >
> > Regards,
> >
> > Hans
>  
>  (snip)
>  
> > p.s.
> >
> > I would not mind picking up maintenance of 1 or 2 of the
> > less frequently changing java packages (*) if that helps.
> > I wonder how much more people there are like me who care
> > enough about java to be willing to put some work in
> > (but not lots of work) ?
> >
> > Perhaps it would be a good idea for the java-SIG to make
> > a list of easy to maintain (for an experienced packager)
> > pkgs which could use a new primary maintainer. I realize
> > maintaining the entire stack is a team effort, so the rest of
> > the java-SIG will of course be/stay a co-maintainer of these.
> > But having things like rebasing to latest upstream, or just
> > addressing bugzillas taken care of mostly by a new
> > primary maintainer who is willing to pick up some packages
> > might help reduce the load ?
> >
> > The idea here is for someone (e.g.) me to NOT jump in now,
> > do a bunch of work to help out and then disappear again.
> > Instead I would invest say 1-2 hours every 2 weeks, which
> > may not seem much but over time adds up and if we can
> > get more people to do this, then maybe we can spread the
> > load of the java packages in the ursine repos better ?
>  
>  That would be great, any help is appreciated.
>  I can try to come up with a list of often-used-but-simple-to-maintain
>  Java packages? 
Yes that sounds great. I would be happy to pick up a few of
those to help and if a bunch of us do that hopefully we can
lighten the load a bit.

Regards,

Hans

[Fabio Valentini]

On Fri, Sep 11, 2020 at 10:04 PM Hans de Goede <hdegoede(a)redhat.com&gt; wrote:
>  Hi,
>
>  On 9/11/20 6:08 PM, Fabio Valentini wrote:
>  > I can try to come up with a list of often-used-but-simple-to-maintain
>  > Java packages?
>
>  Yes that sounds great. I would be happy to pick up a few of
>  those to help and if a bunch of us do that hopefully we can
>  lighten the load a bit. 
I looked through the list of packages currently associated with the
Java SIG, and I've put together a list of things that are usually easy
to maintain:
https://pagure.io/java-maint-sig/blob/master/f/EASYFIX.md

The easy-to-maintain packages are almost 100% up-to-date in fedora
right now, but as we all well know, that state isn't usually lasting
for long :-)

Fabio

[Hans de Goede]

Hi,

On 9/14/20 5:05 PM, Fabio Valentini wrote:
>  On Fri, Sep 11, 2020 at 10:04 PM Hans de Goede
> <hdegoede(a)redhat.com&gt; wrote:
> >
> > Hi,
> >
> > On 9/11/20 6:08 PM, Fabio Valentini wrote:
> >> I can try to come up with a list of often-used-but-simple-to-maintain
> >> Java packages?
> >
> > Yes that sounds great. I would be happy to pick up a few of
> > those to help and if a bunch of us do that hopefully we can
> > lighten the load a bit.
>  
>  I looked through the list of packages currently associated with the
>  Java SIG, and I've put together a list of things that are usually easy
>  to maintain:
>  https://pagure.io/java-maint-sig/blob/master/f/EASYFIX.md
>  
>  The easy-to-maintain packages are almost 100% up-to-date in fedora
>  right now, but as we all well know, that state isn't usually lasting
>  for long :-) 
Thank you for making this list. I'm afraid that I don't really
see anything which really appeals to me to pick-up there, sorry.

Still hopefully some others will find this list helpful.

Regards,

Hans

[Mario Torre]

On Fri, Sep 11, 2020 at 11:04 AM Hans de Goede <hdegoede(a)redhat.com&gt; wrote:

>  So for this tomcat needed for testing problem, I'm thinking that
> we
>  might solve this in a very non Fedora way. Why not bundle the old
>  tomcat-sources with the sources which need it for their test-suite
>  (and build it before running the tests). 
To be honest, when I look at this example, I think we shouldn't have
that library/application packaged at all.

If something can't even run the tests without requiring a 12+ year old
obsoleted code/infrastructure, what chances are that it's maintained
correctly?

Cheers,
Mario
-- 
Mario Torre
Associate Manager, Software Engineering
Red Hat GmbH <https://www.redhat.com>
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898

[Stephen John Smoogen]

On Fri, 11 Sep 2020 at 07:27, Mario Torre <neugens(a)redhat.com&gt; wrote:

>  On Fri, Sep 11, 2020 at 11:04 AM Hans de Goede
> <hdegoede(a)redhat.com&gt;
>  wrote:
>
>  > So for this tomcat needed for testing problem, I'm thinking that we
>  > might solve this in a very non Fedora way. Why not bundle the old
>  > tomcat-sources with the sources which need it for their test-suite
>  > (and build it before running the tests).
>
>  To be honest, when I look at this example, I think we shouldn't have
>  that library/application packaged at all.
>
>  If something can't even run the tests without requiring a 12+ year old
>  obsoleted code/infrastructure, what chances are that it's maintained
>  correctly?
>
>  
The issue is that like https://xkcd.com/2347/ you need that software for a
couple dozen other things which may have no obvious relation to it. You
start pulling a large number of packages and people start screaming that
you are killing their need for using Fedora. that then puts pressure on
someone else to try a herculian task of keeping that package in. Or you
find out that this java package is getting pulled in for say a test suite
in rust and you have to pull the browser, mail client and dozen other
things out because of that. You might say "well I will maintain it until
that test gets pulled out" and then you find out a dozen other software
tools now need it because they saw Mozilla used it so it must be ok. As the
back of the bottle says "rinse, lather, repeat".

The bigger problem is that the software we ship has gotten more complicated
and interdependent over the last decade while the number of packagers who
can deal with that complexity has not grown. You can't bring in a newbie to
take over part of some of these stacks because it reverberates through a
hundred other packages without knowing it.

Personally I think the real problem are we are constantly trying to square
the circle.
1. we insist on trying to keep everything in one repository and hope that
some patched up depsolver can 'theoretically fix it for us' so we never
have an Extras/Core debacle again.
2. we put incredible pressure on maintainers and the project as a whole to
have ALL the software and not dropping anything even when it is clear it
doesn't work in a bundled with the OS format.
3. Never admitting that we messed up in the past but keep pushing through
assuming that by god our mighty brains can eventually solve these
conundrums.  This may sound like a dig about modularity but it is much more
about admitting that the Java maintainer complaints in 2006-> 2009? about
treating Java like C in packaging was going to lead to no one maintaining
it and it falling apart. The same problem is running through the Node
community, the Erlang community, etc etc. In the end we have a bunch of
rules to try and make a camel like a horse and then wonder why we keep
getting spat on by our 'horses'.



>  Cheers,
>  Mario
>  --
>  Mario Torre
>  Associate Manager, Software Engineering
>  Red Hat GmbH <https://www.redhat.com>
>  9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Stephen J Smoogen.

[Zbigniew Jędrzejewski-Szmek]

On Fri, Sep 11, 2020 at 11:03:39AM +0200, Hans de Goede wrote:
>  An other more generic approach which has been brought up once or
>  twice, but which not really has been discussed in much detail yet
>  I believe is creating a fedora-builddep repository.
>  
>  ATM a normal user has 3 ursine Fedora repos installed:
>  
>  fedora
>  fedora-updates
>  fedora-updates-testing (disabled by default)
>  
>  What if we add a 4th repo called fedora-builddep:
>  
>  fedora
>  fedora-updates
>  fedora-updates-testing (disabled by default)
>  fedora-builddep (rolling (within a release), disabled by default)
>  
>  So the idea is that all the maven deps which you need, but
>  do not want to offer any end-user support on would go to
>  fedora-builddep. 
If we absolutely must have build-only packages, we can do it more simply:
insert
  Requires: fedora-unmaintained-package
or 
  Requires: fedora-buildonly-package
(name TBD), and beef up dnf a bit to explain that "this package cannot
be installed because it's only maintained at the level appropriate for
building packages...". I think there are two advantages: first, no need
for a separate repo, so there'll be less infra change and churn. Second,
this tag can easily set on each subrpm, without any central list to manage.

Zbyszek

[Fabio Valentini]

On Fri, Sep 11, 2020 at 5:52 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Fri, Sep 11, 2020 at 11:03:39AM +0200, Hans de Goede wrote:
>  > An other more generic approach which has been brought up once or
>  > twice, but which not really has been discussed in much detail yet
>  > I believe is creating a fedora-builddep repository.
>  >
>  > ATM a normal user has 3 ursine Fedora repos installed:
>  >
>  > fedora
>  > fedora-updates
>  > fedora-updates-testing (disabled by default)
>  >
>  > What if we add a 4th repo called fedora-builddep:
>  >
>  > fedora
>  > fedora-updates
>  > fedora-updates-testing (disabled by default)
>  > fedora-builddep (rolling (within a release), disabled by default)
>  >
>  > So the idea is that all the maven deps which you need, but
>  > do not want to offer any end-user support on would go to
>  > fedora-builddep.
>
>  If we absolutely must have build-only packages, we can do it more simply:
>  insert
>    Requires: fedora-unmaintained-package
>  or
>    Requires: fedora-buildonly-package
>  (name TBD), and beef up dnf a bit to explain that "this package cannot
>  be installed because it's only maintained at the level appropriate for
>  building packages...". I think there are two advantages: first, no need
>  for a separate repo, so there'll be less infra change and churn. Second,
>  this tag can easily set on each subrpm, without any central list to manage. 
I'm not sure making things available only at build-time is going to
help things. It's just "Modularity under a different name" ...
Most Java packages are either directly or indirectly depended on by
non-build-tool packages as well.
You'd be surprised. Most of the distro directly or indirectly depends
on the Java stack already - including the libvirt stack, libreoffice,
some GNOME components, KDE components, etc. So most of those Java
packages can't be built-time-only packages because they're actually
required at runtime.
The number of packages that are *really only ever needed to build
other RPM packages and for no other reasons* is rather small.

Fabio

[Zbigniew Jędrzejewski-Szmek]

On Fri, Sep 11, 2020 at 06:01:11PM +0200, Fabio Valentini wrote:
>  On Fri, Sep 11, 2020 at 5:52 PM Zbigniew Jędrzejewski-Szmek
>  <zbyszek(a)in.waw.pl&gt; wrote:
>  >
>  > On Fri, Sep 11, 2020 at 11:03:39AM +0200, Hans de Goede wrote:
>  > > An other more generic approach which has been brought up once or
>  > > twice, but which not really has been discussed in much detail yet
>  > > I believe is creating a fedora-builddep repository.
>  > >
>  > > ATM a normal user has 3 ursine Fedora repos installed:
>  > >
>  > > fedora
>  > > fedora-updates
>  > > fedora-updates-testing (disabled by default)
>  > >
>  > > What if we add a 4th repo called fedora-builddep:
>  > >
>  > > fedora
>  > > fedora-updates
>  > > fedora-updates-testing (disabled by default)
>  > > fedora-builddep (rolling (within a release), disabled by default)
>  > >
>  > > So the idea is that all the maven deps which you need, but
>  > > do not want to offer any end-user support on would go to
>  > > fedora-builddep.
>  >
>  > If we absolutely must have build-only packages, we can do it more simply:
>  > insert
>  >   Requires: fedora-unmaintained-package
>  > or
>  >   Requires: fedora-buildonly-package
>  > (name TBD), and beef up dnf a bit to explain that "this package cannot
>  > be installed because it's only maintained at the level appropriate for
>  > building packages...". I think there are two advantages: first, no need
>  > for a separate repo, so there'll be less infra change and churn. Second,
>  > this tag can easily set on each subrpm, without any central list to manage.
>  
>  I'm not sure making things available only at build-time is going to
>  help things. It's just "Modularity under a different name" ...
>  Most Java packages are either directly or indirectly depended on by
>  non-build-tool packages as well.
>  You'd be surprised. Most of the distro directly or indirectly depends
>  on the Java stack already - including the libvirt stack, libreoffice,
>  some GNOME components, KDE components, etc. So most of those Java
>  packages can't be built-time-only packages because they're actually
>  required at runtime.
>  The number of packages that are *really only ever needed to build
>  other RPM packages and for no other reasons* is rather small. 
You are probably right. Still, this would be useful to actually have this
surfaced. If a package marked build-time-only would be needed by anything
else, we would need to either unmark it (and have somebody on the hook
for maintaince) or drop the dependency somehow. And this would be vastly
better than build-time-only packages in modules.

I'm not enthusiastic about build-time-only packages, but if the choice
is between that and retiring the packages (or hiding them in modules
which has the same effect), I'll take it.

Zbyszek

[Alexander Scheel]

What Fabio just mentioned was that the current use case for a
build-time only package is invalid. A lot of packages Mikolaj is
trying to get build-time only (via modules) are still maintained by
the Java Maintenance SIG because they're required by other packages.

Allowing them to be build-time only results in the entire ecosystem
crumbling. If they're retired, the ecosystem crumbles too. Same result
either way.


That's why the Java Maintenance SIG was built: so that it can be more
than one person working on these packages.


- Alex

On Fri, Sep 11, 2020 at 12:44 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Fri, Sep 11, 2020 at 06:01:11PM +0200, Fabio Valentini wrote:
>  > On Fri, Sep 11, 2020 at 5:52 PM Zbigniew Jędrzejewski-Szmek
>  > <zbyszek(a)in.waw.pl&gt; wrote:
>  > >
>  > > On Fri, Sep 11, 2020 at 11:03:39AM +0200, Hans de Goede wrote:
>  > > > An other more generic approach which has been brought up once or
>  > > > twice, but which not really has been discussed in much detail yet
>  > > > I believe is creating a fedora-builddep repository.
>  > > >
>  > > > ATM a normal user has 3 ursine Fedora repos installed:
>  > > >
>  > > > fedora
>  > > > fedora-updates
>  > > > fedora-updates-testing (disabled by default)
>  > > >
>  > > > What if we add a 4th repo called fedora-builddep:
>  > > >
>  > > > fedora
>  > > > fedora-updates
>  > > > fedora-updates-testing (disabled by default)
>  > > > fedora-builddep (rolling (within a release), disabled by default)
>  > > >
>  > > > So the idea is that all the maven deps which you need, but
>  > > > do not want to offer any end-user support on would go to
>  > > > fedora-builddep.
>  > >
>  > > If we absolutely must have build-only packages, we can do it more simply:
>  > > insert
>  > >   Requires: fedora-unmaintained-package
>  > > or
>  > >   Requires: fedora-buildonly-package
>  > > (name TBD), and beef up dnf a bit to explain that "this package cannot
>  > > be installed because it's only maintained at the level appropriate for
>  > > building packages...". I think there are two advantages: first, no need
>  > > for a separate repo, so there'll be less infra change and churn. Second,
>  > > this tag can easily set on each subrpm, without any central list to manage.
>  >
>  > I'm not sure making things available only at build-time is going to
>  > help things. It's just "Modularity under a different name" ...
>  > Most Java packages are either directly or indirectly depended on by
>  > non-build-tool packages as well.
>  > You'd be surprised. Most of the distro directly or indirectly depends
>  > on the Java stack already - including the libvirt stack, libreoffice,
>  > some GNOME components, KDE components, etc. So most of those Java
>  > packages can't be built-time-only packages because they're actually
>  > required at runtime.
>  > The number of packages that are *really only ever needed to build
>  > other RPM packages and for no other reasons* is rather small.
>
>  You are probably right. Still, this would be useful to actually have this
>  surfaced. If a package marked build-time-only would be needed by anything
>  else, we would need to either unmark it (and have somebody on the hook
>  for maintaince) or drop the dependency somehow. And this would be vastly
>  better than build-time-only packages in modules.
>
>  I'm not enthusiastic about build-time-only packages, but if the choice
>  is between that and retiring the packages (or hiding them in modules
>  which has the same effect), I'll take it.
>
>  Zbyszek
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Vitaly Zaitsev]

On 11.09.2020 18:42, Zbigniew Jędrzejewski-Szmek wrote:
>  I'm not enthusiastic about build-time-only packages, but if the
> choice
>  is between that and retiring the packages (or hiding them in modules
>  which has the same effect), I'll take it. 
This will be Modularity 2.0. All packages must be installable if the end
user wants to do so.

Check header-only C++ libraries packaging. They provide only the -devel
and virtual -static subpackages and can be easily installed in Koji/mock
or directly.

Also check Rust packaging. They build crates to a separate packages and
use them only for the static linkage.

-- 
Sincerely,
  Vitaly Zaitsev (vitaly(a)easycoding.org)

[Zbigniew Jędrzejewski-Szmek]

On Sat, Sep 12, 2020 at 10:32:51AM +0200, Vitaly Zaitsev via devel wrote:
>  On 11.09.2020 18:42, Zbigniew Jędrzejewski-Szmek wrote:
>  > I'm not enthusiastic about build-time-only packages, but if the choice
>  > is between that and retiring the packages (or hiding them in modules
>  > which has the same effect), I'll take it.
>  
>  This will be Modularity 2.0. All packages must be installable if the end
>  user wants to do so. 
The packages *would* be trivially installable. I didn't say that explicitly,
but you would always be able to install a package that has
"Provides:fedora-build-time-only"
and then install any such packages on any system. (That is how they would
installed in mock or koji too.)

>  Check header-only C++ libraries packaging. They provide only the
> -devel
>  and virtual -static subpackages and can be easily installed in Koji/mock
>  or directly.
>  
>  Also check Rust packaging. They build crates to a separate packages and
>  use them only for the static linkage. 
Zbyszek

[Fabio Valentini]

On Sat, Sep 12, 2020 at 11:05 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Sat, Sep 12, 2020 at 10:32:51AM +0200, Vitaly Zaitsev via devel wrote:
>  > On 11.09.2020 18:42, Zbigniew Jędrzejewski-Szmek wrote:
>  > > I'm not enthusiastic about build-time-only packages, but if the choice
>  > > is between that and retiring the packages (or hiding them in modules
>  > > which has the same effect), I'll take it.
>  >
>  > This will be Modularity 2.0. All packages must be installable if the end
>  > user wants to do so.
>
>  The packages *would* be trivially installable. I didn't say that explicitly,
>  but you would always be able to install a package that has
> "Provides:fedora-build-time-only"
>  and then install any such packages on any system. (That is how they would
>  installed in mock or koji too.)
>
>  > Check header-only C++ libraries packaging. They provide only the -devel
>  > and virtual -static subpackages and can be easily installed in Koji/mock
>  > or directly.
>  >
>  > Also check Rust packaging. They build crates to a separate packages and
>  > use them only for the static linkage. 
Hm, that comparison doesn't fit that well.
Let me try to clarify some things that have come up in this discussion
a couple of times, probably because the confusing "build-only
dependencies" vs. "source-only dependencies / static linking":

TL;DR: Java packaging in fedora is more like python packaging than
Rust packaging.

- Java code is compiled into architecture-independent bytecode (.class
files) that's usually bundled into Zip files with some metadata (.jar
files).
- That bytecode is - and can be - used at both build-time and at
runtime by dependent packages, so it's basically like Python packages.
On the other hand, Rust packages only contain source code, which is
*freshly compiled every time it's used* in dependent packages and then
statically linked.
- The "build-only dependencies" with respect to Java packages (as
mentioned in this thread) refer to *build tools* (like maven and ant)
and their dependencies, which are only used at build-time, but are
obviously not needed at runtime (just like packages built with
autotools, CMake or meson also don't require them at runtime).

However! Of course the dependency trees of build tools and the built
software itself *do overlap* (this is the fact the Java Modularity
approach is ignoring). A lot of Java packages are used both as runtime
dependencies for build tools *and* as runtime dependencies for
applications (which is apparently outside the scope of Red Hat ???) ,
so those have to be maintained as non-modular packages anyway,
apparently as a community-only effort.

Fabio

[Vitaly Zaitsev]

On 11.09.2020 10:16, Mikolaj Izdebski wrote:
>  You get a side tag in Koji where you can have private build-only
>  dependencies that are discarded (filtered) once they are no longer
>  needed, after module build is done. 
IMO, this is a very bad practice. I maintain a lot of C++ header-only
libraries, needed to build other packages and this is fine. Everyone can
use them without bundling into their own packages.

-- 
Sincerely,
  Vitaly Zaitsev (vitaly(a)easycoding.org)

[Kevin Kofler]

Mikolaj Izdebski wrote:
>  Modularity provides not only a different way of grouping or
> delivering
>  RPM packages, but most importantly, a different way of building RPM
>  packages.
>  
>  You get a side tag in Koji where you can have private build-only
>  dependencies that are discarded (filtered) once they are no longer
>  needed, after module build is done. For build-only packages most of
>  security vulnerabilities are not exploitable easily, or at all,
>  therefore are low-severity, which greatly limits maintenance work
>  required to address them. For example, if upstream tests are ran on an
>  obsolete, 12-years old version of Tomcat, I don't need to skip tests,
>  but I can package old Tomcat and run the tests. I don't need to update
>  that Tomcat or fix security issues as they are not exploitable in
>  buildroot - Tomcat runs in embedded mode, does not listen on the
>  network etc. Not something that I could with ursine Tomcat packages -
>  it would get delivered to users, and we have no control how they use
>  it - we would need to fix all important user-reported bugs and CVEs as
>  they are potentially exploitable.
>  
>  Modularity also introduced the concept of API packages - non-API
>  packages can have limited usability, with some non-important features
>  removed. For example if all I need is a small part of Tomcat, I can
>  reduce tomcat package to build just the tiny parts that I need, which
>  don't have any dependencies. Again, not something I could do with
>  ursine Tomcat package. 
But unfortunately, you unilaterally turning most Java packages private this 
way is exactly what put the Java stack in the current state, where it is 
broken for most other users because the packages they need and were relying 
on are no longer available in the public repository.

Tomcat is used by many users, not just as a build dependency (but that, 
too), but also on development machines to test web applications and on test 
and production servers to deploy them.

Sure, a 12-year-old version is hopefully not what the users will need, but 
there the solution ought to be to fix the offending test suite to work with 
the current version of Tomcat (downstream if upstream refuses the fix).

>  Another, more concrete example: core Ant doesn't have any
> dependencies
>  beyond JDK. It is easy to build and maintain, yet very functional. On
>  the other hand, full Ant with all the optional tasks depends on more
>  than 200 Java packages. For the purpose of building all packages that
>  I am interested in, core Ant with JUnit tasks (total of 3 packages) is
>  sufficient. Functionalities of Ant like sending emails or image
>  processing are obviously not needed in this case. If building other
>  packages is the only use of Ant then it can be maintained much more
>  easily (3 instead of ~250 packages). But when Ant is ursine package in
>  Fedora then it should be the full Ant - we can't claim to deliver Ant
>  to users while it is just part of it. Eclipse in Fedora requires full
>  Ant too. 
Given the last sentence (Eclipse needs full Ant), I see no realistic way 
around packaging the whole thing and not just the minimal subset.

>  It's not about the workflow, it is about reducing package
> maintenance
>  work that is required. Modularity allows us to greatly reduce the
>  number of packages by patching-out non-API packages to remove unneeded
>  features and it allows us to spend less time on fixing bugs in
>  packages that are used merely as build dependencies and which we don't
>  intend to be used by end users. 
But you have to understand that, while all this reduces your workload, it 
makes everyone else's (other packagers', developer users', even end
users') 
life harder.

As far as I can see from this and earlier threads, the packages you maintain 
are, in their current form, entirely useless for several potential users.

Personally, I use Java for my day job, but I'm pretty much left using only 
OpenJDK from packages. The libraries, we have always bundled JARs ourselves. 
NetBeans, I used to use the Fedora package, but it was discontinued years 
ago, so I had to switch to the upstream binaries. And now what you have been 
doing to Tomcat lately means I am going to have to work with the upstream 
JARs there too.

        Kevin Kofler