The great Mailman 3 / Hyperkitty upgrade: bumping flufl-lock and mistune?

[Michel Alexandre Salim]

Hi all,

Neal Gompa and I have been reviving the effort to get our mailing list
server infrastructure (currently running on RHEL 7 with missing packages
provided in an unofficial repo) hostable on RHEL 9 + EPEL.

Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)

We're currently stuck on the following:

https://bugzilla.redhat.com/show_bug.cgi?id=2032607

Your package (python-hyperkitty) Fails To Install in Fedora 36:

can't install hyperkitty:
  - nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch
  - nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch

I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288
and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603

but both have breaking changes I detailed in the above Bz entries; if
you're a maintainer cc:ed on this email please check the relevant bz:

❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-flufl-lock
mailman3-0:3.3.4-5.fc35.noarch
mailman3-0:3.3.4-5.fc35.src
mailman3-0:3.3.4-5.fc36.noarch
mailman3-0:3.3.4-5.fc36.src
odcs-0:0.3.4-6.fc35.noarch
odcs-0:0.3.4-6.fc35.src
odcs-0:0.3.4-6.fc36.noarch
odcs-0:0.3.4-6.fc36.src
python-cartopy-0:0.20.0-1.fc35.src
python-cartopy-0:0.20.1-2.fc36.src

❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune                                
python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src                                                                                     
python-nbconvert-0:6.1.0-2.fc35.src                                                                                                  
python-nbconvert-0:6.1.0-3.fc36.src                               
python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch              
python3-nbconvert-0:6.1.0-2.fc35.noarch                           
python3-nbconvert-0:6.1.0-3.fc36.noarch

In particular, python-nbconvert specifically requires mistune < 2, and
upstream doesn't seem to have a newer release yet. python-cartopy oddly
only requires flufl-lock in its SRPM, not the built RPM.

PRs:
https://src.fedoraproject.org/rpms/python-flufl-lock/pull-request/1
https://src.fedoraproject.org/rpms/python-mistune/pull-request/5

(the packages are not in side tags yet because the PRs are not merged
yet, but if it helps I can build them in a COPR for F35)

We should probably bump the packages in Rawhide anyway, but also to
note:
- both of these packages are not co-maintained by the Python SIG
- most of the recent updates have been done by non-maintainers

Would it make sense to get the following groups officially added to the
package ACLs?
- infra-sig (admin), to ease maintaining the dependencies for Mailman
  and Hyperkitty
- python-sig (commit or admin), for fixing issues e.g. with newer Python
  versions
- epel-packagers-sig (collaborator, epel* branches) for helping to
  bootstrap on new EL releases

Thanks,

[Elliott Sales de Andrade]

On Tue, 14 Dec 2021 at 17:32, Michel Alexandre Salim
 wrote:
> Hi all,
>
> Neal Gompa and I have been reviving the effort to get our mailing list
> server infrastructure (currently running on RHEL 7 with missing packages
> provided in an unofficial repo) hostable on RHEL 9 + EPEL.
>
> Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
> Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
> Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)
>
> We're currently stuck on the following:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=2032607
>
> Your package (python-hyperkitty) Fails To Install in Fedora 36:
>
> can't install hyperkitty:
> nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch
> nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch
>
> I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288
> and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603
>
> but both have breaking changes I detailed in the above Bz entries; if
> you're a maintainer cc:ed on this email please check the relevant bz:
>
> ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-flufl-lock
> mailman3-0:3.3.4-5.fc35.noarch
> mailman3-0:3.3.4-5.fc35.src
> mailman3-0:3.3.4-5.fc36.noarch
> mailman3-0:3.3.4-5.fc36.src
> odcs-0:0.3.4-6.fc35.noarch
> odcs-0:0.3.4-6.fc35.src
> odcs-0:0.3.4-6.fc36.noarch
> odcs-0:0.3.4-6.fc36.src
> python-cartopy-0:0.20.0-1.fc35.src
> python-cartopy-0:0.20.1-2.fc36.src
>
> ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune
> python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src
> python-nbconvert-0:6.1.0-2.fc35.src
> python-nbconvert-0:6.1.0-3.fc36.src
> python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch
> python3-nbconvert-0:6.1.0-2.fc35.noarch
> python3-nbconvert-0:6.1.0-3.fc36.noarch
>
> In particular, python-nbconvert specifically requires mistune < 2, and
> upstream doesn't seem to have a newer release yet. python-cartopy oddly
> only requires flufl-lock in its SRPM, not the built RPM.

Cartopy only needs flufl-lock to run tests. I suppose since those are
installed, it could also have a runtime dependency, but it'd be
largely unused, and anyway Cartopy won't need it at all after the next
minor release.
> PRs:
> https://src.fedoraproject.org/rpms/python-flufl-lock/pull-request/1
> https://src.fedoraproject.org/rpms/python-mistune/pull-request/5
>
> (the packages are not in side tags yet because the PRs are not merged
> yet, but if it helps I can build them in a COPR for F35)
>
> We should probably bump the packages in Rawhide anyway, but also to
> note:
> both of these packages are not co-maintained by the Python SIG
> most of the recent updates have been done by non-maintainers
>
> Would it make sense to get the following groups officially added to the
> package ACLs?
> infra-sig (admin), to ease maintaining the dependencies for Mailman
> and Hyperkitty
> python-sig (commit or admin), for fixing issues e.g. with newer Python
> versions
> epel-packagers-sig (collaborator, epel* branches) for helping to
> bootstrap on new EL releases
>
> Thanks,
>
> --
> Michel Alexandre Salim
> profile: https://keyoxide.org/[e-mail]

[Michel Alexandre Salim]

On Tue, Dec 14, 2021 at 05:50:46PM -0500, Elliott Sales de Andrade wrote:
> On Tue, 14 Dec 2021 at 17:32, Michel Alexandre Salim
>  wrote:
> > Hi all,
> >
> > Neal Gompa and I have been reviving the effort to get our mailing list
> > server infrastructure (currently running on RHEL 7 with missing packages
> > provided in an unofficial repo) hostable on RHEL 9 + EPEL.
> >
> > Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
> > Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
> > Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)
> >
> > We're currently stuck on the following:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=2032607
> >
> > Your package (python-hyperkitty) Fails To Install in Fedora 36:
> >
> > can't install hyperkitty:
> > nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch
> > nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch
> >
> > I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288
> > and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603
> >
> > but both have breaking changes I detailed in the above Bz entries; if
> > you're a maintainer cc:ed on this email please check the relevant bz:
> >
> > ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-flufl-lock
> > mailman3-0:3.3.4-5.fc35.noarch
> > mailman3-0:3.3.4-5.fc35.src
> > mailman3-0:3.3.4-5.fc36.noarch
> > mailman3-0:3.3.4-5.fc36.src
> > odcs-0:0.3.4-6.fc35.noarch
> > odcs-0:0.3.4-6.fc35.src
> > odcs-0:0.3.4-6.fc36.noarch
> > odcs-0:0.3.4-6.fc36.src
> > python-cartopy-0:0.20.0-1.fc35.src
> > python-cartopy-0:0.20.1-2.fc36.src
> >
> > ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune
> > python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src
> > python-nbconvert-0:6.1.0-2.fc35.src
> > python-nbconvert-0:6.1.0-3.fc36.src
> > python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch
> > python3-nbconvert-0:6.1.0-2.fc35.noarch
> > python3-nbconvert-0:6.1.0-3.fc36.noarch
> >
> > In particular, python-nbconvert specifically requires mistune < 2, and
> > upstream doesn't seem to have a newer release yet. python-cartopy oddly
> > only requires flufl-lock in its SRPM, not the built RPM.
>
> Cartopy only needs flufl-lock to run tests. I suppose since those are
> installed, it could also have a runtime dependency, but it'd be
> largely unused, and anyway Cartopy won't need it at all after the next
> minor release.

Thanks, I just checked and all cartopy tests still pass with flufl-lock 6.0.

Best regards,

[Miro Hrončok]

On 14. 12. 21 23:29, Michel Alexandre Salim wrote:
> Hi all,
>
> Neal Gompa and I have been reviving the effort to get our mailing list
> server infrastructure (currently running on RHEL 7 with missing packages
> provided in an unofficial repo) hostable on RHEL 9 + EPEL.
>
> Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
> Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
> Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)
>
> We're currently stuck on the following:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=2032607
>
> Your package (python-hyperkitty) Fails To Install in Fedora 36:
>
> can't install hyperkitty:
> nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch
> nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch
>
> I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288
> and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603
>
> but both have breaking changes I detailed in the above Bz entries; if
> you're a maintainer cc:ed on this email please check the relevant bz:
>
> ...
>
> ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune
> python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src
> python-nbconvert-0:6.1.0-2.fc35.src
> python-nbconvert-0:6.1.0-3.fc36.src
> python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch
> python3-nbconvert-0:6.1.0-2.fc35.noarch
> python3-nbconvert-0:6.1.0-3.fc36.noarch
>
> In particular, python-nbconvert specifically requires mistune < 2, and
> upstream doesn't seem to have a newer release yet.

This is the biggest blocker. If nbconvert figures it out in git, we can 
backport it. Until then, I would not merge the update. If we really need this 
ASAP, we can introduce python-mistune0.8 compat pacakge.
> PRs:
> https://src.fedoraproject.org/rpms/python-flufl-lock/pull-request/1
> https://src.fedoraproject.org/rpms/python-mistune/pull-request/5
>
> ...
>
> We should probably bump the packages in Rawhide anyway

No, please don't yet.
> , but also to
> note:
> both of these packages are not co-maintained by the Python SIG

mistune is.
> most of the recent updates have been done by non-maintainers

For mistune, it was by Python SIG members.
> Would it make sense to get the following groups officially added to the
> package ACLs?
> infra-sig (admin), to ease maintaining the dependencies for Mailman
>  and Hyperkitty

Python SIG is very responsive to pull requests, so I don't think adding another 
SIG makes the "ease of maintaining" significantly better.
> python-sig (commit or admin), for fixing issues e.g. with newer Python
>  versions

Already there.
> epel-packagers-sig (collaborator, epel* branches) for helping to
>  bootstrap on new EL releases

Fine by me, although I prefer actual maintainers to be responsible for EPEL 
branches.

[Miro Hrončok]

On 15. 12. 21 11:51, Miro Hrončok wrote:
> On 14. 12. 21 23:29, Michel Alexandre Salim wrote:
> > In particular, python-nbconvert specifically requires mistune < 2, and
> > upstream doesn't seem to have a newer release yet.
>
> This is the biggest blocker. If nbconvert figures it out in git, we can 
> backport it. Until then, I would not merge the update. If we really need this 
> ASAP, we can introduce python-mistune0.8 compat pacakge.

Upstream issue https://github.com/jupyter/nbconvert/issues/1685

I am quite confident they would accept a PR, but I don't have the capacity to 
contribute one at this moment.

[Michel Alexandre Salim]

On Wed, Dec 15, 2021 at 11:53:02AM +0100, Miro Hrončok wrote:
> On 15. 12. 21 11:51, Miro Hrončok wrote:
> > On 14. 12. 21 23:29, Michel Alexandre Salim wrote:
> > > In particular, python-nbconvert specifically requires mistune < 2, and
> > > upstream doesn't seem to have a newer release yet.
> >
> > This is the biggest blocker. If nbconvert figures it out in git, we can
> > backport it. Until then, I would not merge the update. If we really need
> > this ASAP, we can introduce python-mistune0.8 compat pacakge.
>
> Upstream issue https://github.com/jupyter/nbconvert/issues/1685
>
> I am quite confident they would accept a PR, but I don't have the capacity
> to contribute one at this moment.

I can try looking at this to unblock then.

Thanks,

[Michel Alexandre Salim]

On Tue, Dec 14, 2021 at 02:29:58PM -0800, Michel Alexandre Salim wrote:
> Hi all,
>
> Neal Gompa and I have been reviving the effort to get our mailing list
> server infrastructure (currently running on RHEL 7 with missing packages
> provided in an unofficial repo) hostable on RHEL 9 + EPEL.
>
> Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
> Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
> Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)

Any objection if I modernize python-django's spec file so we get
automatic BRs? I accidentally branched python-mock for EPEL9 because
Django BR-ed it, before realizing that it's an out of date BR and the
unit tests all use `from unittest import mock` already

ref: https://fedoraproject.org/wiki/Changes/DeprecatePythonMock

python-django without python-mock built for Rawhide: https://koji.fedoraproject.org/koji/buildinfo?buildID=1867385

(I haven't built python-mock in EPEL9 yet, and am not going to unless we
really need it for something - but I'll hold off on retiring the branch)

Thanks,

[Miro Hrončok]

On 15. 12. 21 21:49, Michel Alexandre Salim wrote:
> (I haven't built python-mock in EPEL9 yet, and am not going to unless we
> really need it for something - but I'll hold off on retiring the branch)

Please, retire it for EPEL9. It should never be needed.

[Charalampos Stratakis]

On Wed, Dec 15, 2021 at 10:03 PM Miro Hrončok  wrote:
> On 15. 12. 21 21:49, Michel Alexandre Salim wrote:
> > (I haven't built python-mock in EPEL9 yet, and am not going to unless we
> > really need it for something - but I'll hold off on retiring the branch)
>
> Please, retire it for EPEL9. It should never be needed.

Unfortunately it's quite possible something might need it as there are
still a lot of packages relying on the external mock.

I would keep the branch but defer building it for now.
> --
> Miro Hrončok
> --
> Phone: [phone]
> IRC: mhroncok
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure

[Michel Alexandre Salim]

On Wed, Dec 15, 2021 at 10:20:27PM +0100, Charalampos Stratakis wrote:
> On Wed, Dec 15, 2021 at 10:03 PM Miro Hrončok  wrote:
> > On 15. 12. 21 21:49, Michel Alexandre Salim wrote:
> > > (I haven't built python-mock in EPEL9 yet, and am not going to unless we
> > > really need it for something - but I'll hold off on retiring the branch)
> >
> > Please, retire it for EPEL9. It should never be needed.
>
> Unfortunately it's quite possible something might need it as there are
> still a lot of packages relying on the external mock.

True, but we probably should patch these going forward? I'm happy to
help. I wonder how many of those will turn out to be obsolete BRs too.
> I would keep the branch but defer building it for now.

Argh, too late, but unretiring is easy anyway.

Regards,

[Charalampos Stratakis]

On Wed, Dec 15, 2021 at 10:26 PM Michel Alexandre Salim <
[e-mail]> wrote:
> On Wed, Dec 15, 2021 at 10:20:27PM +0100, Charalampos Stratakis wrote:
> > On Wed, Dec 15, 2021 at 10:03 PM Miro Hrončok
>
> wrote:
> > > On 15. 12. 21 21:49, Michel Alexandre Salim wrote:
> > > > (I haven't built python-mock in EPEL9 yet, and am not going to
>
> unless we
> > > > really need it for something - but I'll hold off on retiring the
>
> branch)
> > > Please, retire it for EPEL9. It should never be needed.
> >
> > Unfortunately it's quite possible something might need it as there are
> > still a lot of packages relying on the external mock.
>
> True, but we probably should patch these going forward? I'm happy to
> help. I wonder how many of those will turn out to be obsolete BRs too.
> > I would keep the branch but defer building it for now.
>
> Argh, too late, but unretiring is easy anyway.

No worries, I agree patching the imports out is a better solution IMHO.
> Regards,
>
> --
> Michel Alexandre Salim
> profile: https://keyoxide.org/[e-mail]
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure

[Michel Alexandre Salim]

On Wed, Dec 15, 2021 at 10:03:29PM +0100, Miro Hrončok wrote:
> On 15. 12. 21 21:49, Michel Alexandre Salim wrote:
> > (I haven't built python-mock in EPEL9 yet, and am not going to unless we
> > really need it for something - but I'll hold off on retiring the branch)
>
> Please, retire it for EPEL9. It should never be needed.

Done, sorry for the noise

https://src.fedoraproject.org/rpms/python-mock/blob/9cca66d7e2b5ac0784e160e0...

Thanks,

Adjusting python-django to latest Python Packaging Guidelines

[Michel Alexandre Salim]

On Wed, Dec 15, 2021 at 12:49:51PM -0800, Michel Alexandre Salim wrote:
> On Tue, Dec 14, 2021 at 02:29:58PM -0800, Michel Alexandre Salim wrote:
> > Hi all,
> >
> > Neal Gompa and I have been reviving the effort to get our mailing list
> > server infrastructure (currently running on RHEL 7 with missing packages
> > provided in an unofficial repo) hostable on RHEL 9 + EPEL.
> >
> > Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
> > Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
> > Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)
>
> Any objection if I modernize python-django's spec file so we get
> automatic BRs? I accidentally branched python-mock for EPEL9 because
> Django BR-ed it, before realizing that it's an out of date BR and the
> unit tests all use `from unittest import mock` already
>
> ref: https://fedoraproject.org/wiki/Changes/DeprecatePythonMock
>
> python-django without python-mock built for Rawhide: https://koji.fedoraproject.org/koji/buildinfo?buildID=1867385

PR: https://src.fedoraproject.org/rpms/python-django/pull-request/20

This now adheres closely to the Python Packaging Guidelines
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/

- automatic BRs
- Consistently use pyproject macros

As a bonus, it's so much cleaner and we track the directories for
localization files better. Provides and Requires are sane (I provided
details in the PR comments)

I'll leave this open for a couple of days and then merge it if there's
no objection.

Thanks,