Aggressive updating (Python 3.9): Are we trying to hard?

[Richard Shaw]

So I get the whole Fedora first, but...

Backstory:

FreeCAD has been in a terrible state in Fedora for a couple of years now
and I've nearly given up on trying to maintain the package a few times now.
The previous battle was with the Coin3D stack which finally got updated to
Coin4 in f32 (then Rawhide). I constantly got bugs submitted that freecad
was broken in f31 and my answer was, "Be patient it'll finally be fixed in
Fedora 32 where the Coin3D stack is updated!".

However, that was a lie. Unintentional, but a lie nonetheless.

Fedora 32 releases and I'm enthusiastic that freecad will finally be
*fixed*! And then I get my first BZ, now PySide2 is broken... Why?

Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. But
instead of asking ourselves, "should we push in the VERY latest Python and
hope it's ok?", we just patch the build system to accept it anyway and hope
for the best.

Qt (et all) is a pretty organized upstream, so when asked about Python 3.8
support in 5.13.x, they said, "Nope. Wait for 5.14.x."

What good does that do me? At the time it wasn't released, and once it was
only Rawhide got it. And I get why. Updating the whole Qt stack and
rebuilding all the dependencies is pretty a pretty painful process.

So all that to say, we seem to be taking "Fedora First" to mean, we're
going to update regardless of what it breaks.

This is a little more ranty that I intended it to be, and no, I'm not going
to go research and paste a bunch of BZ urls, this isn't about asking for
help, it's more policy related.

Hell, I still have one project that all but refuses to update to Python 3!

Fortuantly a user found a github project that had the needed patches
backported from 5.14.x! Fingers crossed freecad will work again for more
than 5 seconds.

Now I'm being told PySide2 doesn't build with Python 3.9 pre-release. I'm
not surprised! It's not even released yet. I can (and will) inform
upstream. But what do you think their answer is going to be? Wait for 5.15?

Ok, rant over.

Thanks,
Richard

[Miro Hrončok]

On 19. 05. 20 14:03, Richard Shaw wrote:
>  So I get the whole Fedora first, but...
>  
>  Backstory:
>  
>  FreeCAD has been in a terrible state in Fedora for a couple of years now and 
>  I've nearly given up on trying to maintain the package a few times now. The 
>  previous battle was with the Coin3D stack which finally got updated to Coin4 in 
>  f32 (then Rawhide). I constantly got bugs submitted that freecad was broken in 
>  f31 and my answer was, "Be patient it'll finally be fixed in Fedora 32 where the
>
>  Coin3D stack is updated!".
>  
>  However, that was a lie. Unintentional, but a lie nonetheless.
>  
>  Fedora 32 releases and I'm enthusiastic that freecad will finally be *fixed*! 
>  And then I get my first BZ, now PySide2 is broken... Why?
>  
>  Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. But 
>  instead of asking ourselves, "should we push in the VERY latest Python and hope 
>  it's ok?", we just patch the build system to accept it anyway and hope for the 
>  best.
>  
>  Qt (et all) is a pretty organized upstream, so when asked about Python 3.8 
>  support in 5.13.x, they said, "Nope. Wait for 5.14.x."
>  
>  What good does that do me? At the time it wasn't released, and once it was only 
>  Rawhide got it. And I get why. Updating the whole Qt stack and rebuilding all 
>  the dependencies is pretty a pretty painful process.
>  
>  So all that to say, we seem to be taking "Fedora First" to mean, we're
> going to 
>  update regardless of what it breaks.
>  
>  This is a little more ranty that I intended it to be, and no, I'm not going to 
>  go research and paste a bunch of BZ urls, this isn't about asking for help, it's
>
>  more policy related.
>  
>  Hell, I still have one project that all but refuses to update to Python 3!
>  
>  Fortuantly a user found a github project that had the needed patches backported 
>  from 5.14.x! Fingers crossed freecad will work again for more than 5 seconds.
>  
>  Now I'm being told PySide2 doesn't build with Python 3.9 pre-release. I'm not
>
>  surprised! It's not even released yet. I can (and will) inform upstream. But 
>  what do you think their answer is going to be? Wait for 5.15?
>  
>  Ok, rant over. 
Obviously, updating to latest Python as it is released might not be the only 
available approach. We could stay at 3.7 until it gets security only and then 
update to 3.8 etc. Security support for old Python releases is long enough for 
us to do this. (I am intentionally leaving the EL-support related problem out.)

Question is, whether this model would suit Fedora better. At this point, we are 
following the "(b)leading edge" approach. In my point of view, this means that 
as distributors and integrators, we take the bumpy road for the benefits of the 
projects we package. Often, it's Fedora who drives Python 3.N+1 support in 
various upstream projects. It also means bugs are found in Python itself (and 
significant libraries) before the "stable" version is released. If we integrate

new Python versions later, this "trying hard" will just shift from alphas and 
betas to .1 and .2.

OTOH There are always projects that won't be able to keep up with this and the 
maintainers of such (like you) are caught in the cross fire.

The current approach also means that we can actually drive improvements in 
CPython upstream. If we slow down significantly, it means we can only drive 
changes in downstream patches or wait for our projects to land in years.

I am obviously biased here, so I won't argue that the current approach is better 
-- it has pros and cons. But I'll follow the discussion and try answer questions 
to the best of my abilities.

Thanks for your feedback, even when you call it "rant", it was pretty on topic
;)

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Miro Hrončok]

On 19. 05. 20 14:24, Miro Hrončok wrote:
>  If we integrate new Python versions later, this "trying
> hard" will just shift 
>  from alphas and betas to .1 and .2. 
One more important thing to note is that for example with Python 3.9, it was 
input from Fedora that made a couple long-deprecated-now-removed things 
reverted, because the impact in Fedora was to big. I cannot imagine this 
happening if we integrated Python 3.9.2 instead of 3.9.0a2.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Igor Raits]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi Richard,

On Tue, 2020-05-19 at 07:03 -0500, Richard Shaw wrote:
>  So I get the whole Fedora first, but...
>  
>  Backstory:
>  
>  FreeCAD has been in a terrible state in Fedora for a couple of years
>  now
>  and I've nearly given up on trying to maintain the package a few
>  times now.
>  The previous battle was with the Coin3D stack which finally got
>  updated to
>  Coin4 in f32 (then Rawhide). I constantly got bugs submitted that
>  freecad
>  was broken in f31 and my answer was, "Be patient it'll finally be
>  fixed in
>  Fedora 32 where the Coin3D stack is updated!". 
Was this caused by upgrade of Coin3D stack in Fedora? I did not see any
Change Proposal for this in F32, so probably this should have been
better communicated / tested?

>  However, that was a lie. Unintentional, but a lie nonetheless.
>  
>  Fedora 32 releases and I'm enthusiastic that freecad will finally be
>  *fixed*! And then I get my first BZ, now PySide2 is broken... Why?
>  
>  Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
>  But
>  instead of asking ourselves, "should we push in the VERY latest
>  Python and
>  hope it's ok?", we just patch the build system to accept it anyway
>  and hope
>  for the best. 
I think pushing for latest Python is the best approach for Fedora, but
I agree that this is unfortunate. However, it is not only about Python,
we also have GCC / glibc and others which are built from master branch
in the beginning of Rawhide so that it gets some testing.

>  Qt (et all) is a pretty organized upstream, so when asked about
>  Python 3.8
>  support in 5.13.x, they said, "Nope. Wait for 5.14.x." 
I think what could help here is to make sure that list of specific
applications / framework works with new version of Python when Python
team is upgrading it. If we are not pushing upstream to support new
version of Python, it will happen but much later.

Also we are not pushing for new versions of Python in stable releases,
so at least that is not affected.

>  What good does that do me? At the time it wasn't released, and
> once
>  it was
>  only Rawhide got it. And I get why. Updating the whole Qt stack and
>  rebuilding all the dependencies is pretty a pretty painful process. 
I think we should get people who maintain Qt on board when updating
Python so that they make sure to backport necessary patches from
upstream when we upgrade Python.

>  So all that to say, we seem to be taking "Fedora First" to
> mean,
>  we're
>  going to update regardless of what it breaks. 
All this is about balance, if it breaks only 1% of applications, but
helps 40% of applications (all numbers are random here from my head) it
makes sense to upgrade. Not breaking even 1% of applications is what
RHEL is supposed to solve, not Fedora.

>  This is a little more ranty that I intended it to be, and no, I'm
> not
>  going
>  to go research and paste a bunch of BZ urls, this isn't about asking
>  for
>  help, it's more policy related.
>  
>  Hell, I still have one project that all but refuses to update to
>  Python 3!
>  
>  Fortuantly a user found a github project that had the needed patches
>  backported from 5.14.x! Fingers crossed freecad will work again for
>  more
>  than 5 seconds. 
That would be cool!

>  Now I'm being told PySide2 doesn't build with Python 3.9
> pre-release. 
>  I'm
>  not surprised! It's not even released yet. I can (and will) inform
>  upstream. But what do you think their answer is going to be? Wait for
>  5.15? 
Ask upstream to always test with develop version of Python, I believe
that services like Travis CI have 3.9-dev prepared so that they can
test early and adopt.

Meanwhile, if they can just link all relevant fixes - just backport
them in Rawhide.

>  Ok, rant over. 
Thank you for sharing your experience here.

>  Thanks,
>  Richard
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: 
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: 
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: 
>  https://lists.fedoraproject.org/archives/list/[e-mail] 
- -- 
Igor Raits <ignatenkobrain(a)fedoraproject.org&gt;
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCgAdFiEEcwgJ58gsbV5f5dMcEV1auJxcHh4FAl7D15AACgkQEV1auJxc
> Hh5r3A//VWxp4WlTbr7KmWXJpGoCmbMeEgT2z8VERYpeCMs4AIkalupx5DiyEf9v
> 24kpOqL3mC2djRQ5w0Qg69eF2UjOry+we6b5baXZwcnJmVS8ADB4QBSfMDzwaIw4
> TpC53F/cRmKj3mtCpGED2pbh980yB8xB49gc8xATj3Ip7TrVTwqRB1InUw0uDfU/
> 3Vpbb5QY14YkVR/Y+4f4ZzWHYJRlKiWNjEf4ALVYR3HZ6R+CSiTJbvK1HcCquV2R
> 5YZR+jLetM4odaA09kSIRr6egO9fyZTfAFxl8MBRVVJxoz0FSK03WA0JrEFn4guq
> f6L2b8c2jy7TTk1UniZwCBqc+Y40pFawRqRDjkojcWUPOiP6UOpDcDpKx4EcMtoq
> ZnxMFL8I93wxbuGhrzbNglTcjF2PWQatZeCRktARgCbZyxxX5P6lHXbpRUdVx/0H
> +o+tv3JLIMBGvX7wp9rxA6uHJJtrJfgJpqhR4KTd/DBYOefaKjP+MWdax9xJO4Sa
> OubWohjBtPAzLozEzgupB+NoTox0ljk6MmVLctKb78+5/XYULkQeVMPDYnOJX+aR
> 3TEbGAq1Qus6YHvHuKcsGg0fh6FhYJkHA+An9Hk70n+cD0kkau8WIJk5pdA9lziU
> a5OwRhZYJ5dRDVZNdrKutP7zb9vEgtgBAjDDqFipuZRBVPRODTk=
> =94nS
> -----END PGP SIGNATURE-----

[Richard Shaw]

On Tue, May 19, 2020 at 7:57 AM Igor Raits <ignatenkobrain(a)fedoraproject.org&gt;
wrote:

>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA512
>
>  Hi Richard,
>
>  On Tue, 2020-05-19 at 07:03 -0500, Richard Shaw wrote:
>  > So I get the whole Fedora first, but...
>  >
>  > Backstory:
>  >
>  > FreeCAD has been in a terrible state in Fedora for a couple of years
>  > now
>  > and I've nearly given up on trying to maintain the package a few
>  > times now.
>  > The previous battle was with the Coin3D stack which finally got
>  > updated to
>  > Coin4 in f32 (then Rawhide). I constantly got bugs submitted that
>  > freecad
>  > was broken in f31 and my answer was, "Be patient it'll finally be
>  > fixed in
>  > Fedora 32 where the Coin3D stack is updated!".
>
>  Was this caused by upgrade of Coin3D stack in Fedora? I did not see any
>  Change Proposal for this in F32, so probably this should have been
>  better communicated / tested?
>  
No, this was just backstory. The Coin3D v4 stack was required for FreeCAD
but up until Fedora 32, Only the v3 stack was available., It was only
updated in rawhide (f32 at the time).



>  > However, that was a lie. Unintentional, but a lie nonetheless.
>  >
>  > Fedora 32 releases and I'm enthusiastic that freecad will finally be
>  > *fixed*! And then I get my first BZ, now PySide2 is broken... Why?
>  >
>  > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
>  > But
>  > instead of asking ourselves, "should we push in the VERY latest
>  > Python and
>  > hope it's ok?", we just patch the build system to accept it anyway
>  > and hope
>  > for the best.
>
>  I think pushing for latest Python is the best approach for Fedora, but
>  I agree that this is unfortunate. However, it is not only about Python,
>  we also have GCC / glibc and others which are built from master branch
>  in the beginning of Rawhide so that it gets some testing.
>  
GCC affects a lot more packages and problems tend to get fix pretty quickly.



>  > Qt (et all) is a pretty organized upstream, so when asked about
>  > Python 3.8
>  > support in 5.13.x, they said, "Nope. Wait for 5.14.x."
>
>  I think what could help here is to make sure that list of specific
>  applications / framework works with new version of Python when Python
>  team is upgrading it. If we are not pushing upstream to support new
>  version of Python, it will happen but much later.
>  
This is where I was going even if I didn't mention it explicitly. I think
we need to be more organized in how things are updated and try to keep a
list (somehow) of libraries (more so than end user binaries) that may not
work with the latest and greatest.



>  > What good does that do me? At the time it wasn't released,
> and once
>  > it was
>  > only Rawhide got it. And I get why. Updating the whole Qt stack and
>  > rebuilding all the dependencies is pretty a pretty painful process.
>
>  I think we should get people who maintain Qt on board when updating
>  Python so that they make sure to backport necessary patches from
>  upstream when we upgrade Python.
>  
It didn't affect Qt5 AFAIK, just the python bindings (PySide2), but as far
as people, it's mostly Rex and I help where I can.



>  > So all that to say, we seem to be taking "Fedora
> First" to mean,
>  > we're
>  > going to update regardless of what it breaks.
>
>  All this is about balance, if it breaks only 1% of applications, but
>  helps 40% of applications (all numbers are random here from my head) it
>  makes sense to upgrade. Not breaking even 1% of applications is what
>  RHEL is supposed to solve, not Fedora.
>  
I don't think that's quite a good rationale, breaking 1% of apps is
unfortunate, breaking 1 framework is bad.



>  > Now I'm being told PySide2 doesn't build with Python 3.9
> pre-release.
>  > I'm
>  > not surprised! It's not even released yet. I can (and will) inform
>  > upstream. But what do you think their answer is going to be? Wait for
>  > 5.15?
>
>  Ask upstream to always test with develop version of Python, I believe
>  that services like Travis CI have 3.9-dev prepared so that they can
>  test early and adopt.
>
>  Meanwhile, if they can just link all relevant fixes - just backport
>  them in Rawhide.
>  
Those more familiar with the changes implemented could do a better job at
helping me inform upstream (and maybe help provide solutions?) But I think
that comes with being more organized in how to do major component updates.

Thanks,
Richard

[Miro Hrončok]

On 19. 05. 20 15:19, Richard Shaw wrote:
>      Ask upstream to always test with develop version of Python, I
> believe
>      that services like Travis CI have 3.9-dev prepared so that they can
>      test early and adopt.
>  
>      Meanwhile, if they can just link all relevant fixes - just backport
>      them in Rawhide.
>  
>  
>  Those more familiar with the changes implemented could do a better job at 
>  helping me inform upstream (and maybe help provide solutions?) But I think that 
>  comes with being more organized in how to do major component updates. 
We are doing our best to analyze the failures and provide as much information as 
we can, so informing upstream should be easy. When the build fails with an 
actual error, we are able to do this. In case of bz1837011, this seems like an 
hesienbug and we are not there yet.

I also sincerely believe, that this is organized very well. Where do you think 
this could be organized better? I honestly believe that we go way beyond what 
other stacks do when they are updated to their new major versions (sometimes 
simply updating the compiler/interpreter and than going away).

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Richard Shaw]

On Tue, May 19, 2020 at 8:37 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  On 19. 05. 20 15:19, Richard Shaw wrote:
>  >     Ask upstream to always test with develop version of Python, I believe
>  >     that services like Travis CI have 3.9-dev prepared so that they can
>  >     test early and adopt.
>  >
>  >     Meanwhile, if they can just link all relevant fixes - just backport
>  >     them in Rawhide.
>  >
>  >
>  > Those more familiar with the changes implemented could do a better job
>  at
>  > helping me inform upstream (and maybe help provide solutions?) But I
>  think that
>  > comes with being more organized in how to do major component updates.
>
>  We are doing our best to analyze the failures and provide as much
>  information as
>  we can, so informing upstream should be easy. When the build fails with an
>  actual error, we are able to do this. In case of bz1837011, this seems
>  like an
>  hesienbug and we are not there yet.
>  
Yes, but we still have time on that one as it's COPR only at this point.



>  I also sincerely believe, that this is organized very well. Where do
> you
>  think
>  this could be organized better? I honestly believe that we go way beyond
>  what
>  other stacks do when they are updated to their new major versions
>  (sometimes
>  simply updating the compiler/interpreter and than going away).
>  
I think that's both true, but it says more about some of the other stacks :)

For one, Python is an extremely important part of the distribution so it
warrants the effort.

Secondly, in the specific case of Python 3.8 / PySide 5.13, we blindly
patched the builder to accept Python 3.8 and assumed because it built, that
it was functional. Now it would be fair to say upstream could improve unit
testing and should have caught it, but to be fair to them, they explicitly
said Python 3.8 was not supported.

So perhaps the solution is when we patch a build system, we ask ourselves
(me included), "What can we do to make sure it's functional?"

I think here we need better ways of testing software in Rawhide other than
well, running Rawhide (VM or bare metal), besides the old mock chroot xnest
hack. I'm open to suggestion here.

Random thought... Would it be allowable to abuse the rawhide test instance
to install software and test via SSH/remote X?

Thanks,
Richard

[Adam Williamson]

On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  I think here we need better ways of testing software in Rawhide other
> than
>  well, running Rawhide (VM or bare metal), besides the old mock chroot xnest
>  hack. I'm open to suggestion here. 
https://docs.fedoraproject.org/en-US/ci/ ?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Richard Shaw]

On Tue, May 19, 2020 at 10:31 AM Adam Williamson <adamwill(a)fedoraproject.org&gt;
wrote:

>  On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  > I think here we need better ways of testing software in Rawhide other
>  than
>  > well, running Rawhide (VM or bare metal), besides the old mock chroot
>  xnest
>  > hack. I'm open to suggestion here.
>
>  https://docs.fedoraproject.org/en-US/ci/ ?
>  
Ok, so obviously CI can be very helpful, but is part of the expectation for
all packagers to learn how to set this up? It's non-trivial unless it's
something you already know how to do, and in this case we didn't know there
was a problem until an end user ran FreeCAD with the update and noticed the
crash...

Thanks,
Richard

[Adam Williamson]

On Tue, 2020-05-19 at 10:43 -0500, Richard Shaw wrote:
>  On Tue, May 19, 2020 at 10:31 AM Adam Williamson
> <adamwill(a)fedoraproject.org&gt;
>  wrote:
>  
>  > On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  > > I think here we need better ways of testing software in Rawhide other
>  > than
>  > > well, running Rawhide (VM or bare metal), besides the old mock chroot
>  > xnest
>  > > hack. I'm open to suggestion here.
>  > 
>  > https://docs.fedoraproject.org/en-US/ci/ ?
>  > 
>  
>  Ok, so obviously CI can be very helpful, but is part of the expectation for
>  all packagers to learn how to set this up? 
If by "set this up" you mean "what to put in place to run some specific
test(s) on your package", yes. That's what the docs are for.

We're currently working on implementing what we call "generic" testing
in Fedora CI - that is, tests that automatically run on every package,
the way Taskotron used to do it, as opposed to tests being associated
with packages, the way Fedora CI initially expected - but that doesn't
seem like the desire here.

>   It's non-trivial unless it's
>  something you already know how to do, and in this case we didn't know there
>  was a problem until an end user ran FreeCAD with the update and noticed the
>  crash... 
Well sure, but now you know it's a thing that *can* go wrong, so you
want to test it in future. I'm not saying you should have done it
already, just that now you know it needs testing, that's one way to go
about it...though if you need to do graphical testing Fedora CI is
probably not the way...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Aleksandra Fedorova]

On Tue, 19 May 2020, 18:10 Adam Williamson, <adamwill(a)fedoraproject.org&gt;
wrote:

>  On Tue, 2020-05-19 at 10:43 -0500, Richard Shaw wrote:
>  > On Tue, May 19, 2020 at 10:31 AM Adam Williamson <
>  adamwill(a)fedoraproject.org&gt;
>  > wrote:
>  >
>  > > On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  > > > I think here we need better ways of testing software in Rawhide other
>  > > than
>  > > > well, running Rawhide (VM or bare metal), besides the old mock chroot
>  > > xnest
>  > > > hack. I'm open to suggestion here.
>  > >
>  > > https://docs.fedoraproject.org/en-US/ci/ ?
>  > >
>  >
>  > Ok, so obviously CI can be very helpful, but is part of the expectation
>  for
>  > all packagers to learn how to set this up?
>
>  If by "set this up" you mean "what to put in place to run some specific
>  test(s) on your package", yes. That's what the docs are for.
>  
I would suggest to start with
https://docs.fedoraproject.org/en-US/ci/how-to-add-dist-git-test/

As it describes the most simple way to start testing.

With more complex test setup you can reach out to Fedora CI SIG, for
example, via issue at
https://pagure.io/fedora-ci/general


>  We're currently working on implementing what we call
> "generic" testing
>  in Fedora CI - that is, tests that automatically run on every package,
>  the way Taskotron used to do it, as opposed to tests being associated
>  with packages, the way Fedora CI initially expected - but that doesn't
>  seem like the desire here.
>
>  >  It's non-trivial unless it's
>  > something you already know how to do, and in this case we didn't know
>  there
>  > was a problem until an end user ran FreeCAD with the update and noticed
>  the
>  > crash...
>
>  Well sure, but now you know it's a thing that *can* go wrong, so you
>  want to test it in future. I'm not saying you should have done it
>  already, just that now you know it needs testing, that's one way to go
>  about it...though if you need to do graphical testing Fedora CI is
>  probably not the way...
>  --
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>  http://www.happyassassin.net
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Dan Horák]

On Tue, 19 May 2020 08:29:45 -0700
Adam Williamson <adamwill(a)fedoraproject.org&gt; wrote:

>  On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  > I think here we need better ways of testing software in Rawhide
>  > other than well, running Rawhide (VM or bare metal), besides the
>  > old mock chroot xnest hack. I'm open to suggestion here.
>  
>  https://docs.fedoraproject.org/en-US/ci/ ? 
I think Richard is rather looking for OpenQA for testing the GUI unless
the FreeCAD functionality can be also tested thru some command line /
text mode interface.


		Dan

[Adam Williamson]

On Tue, 2020-05-19 at 17:50 +0200, Dan Horák wrote:
>  On Tue, 19 May 2020 08:29:45 -0700
>  Adam Williamson <adamwill(a)fedoraproject.org&gt; wrote:
>  
>  > On Tue, 2020-05-19 at 08:49 -0500, Richard Shaw wrote:
>  > > I think here we need better ways of testing software in Rawhide
>  > > other than well, running Rawhide (VM or bare metal), besides the
>  > > old mock chroot xnest hack. I'm open to suggestion here.
>  > 
>  > https://docs.fedoraproject.org/en-US/ci/ ?
>  
>  I think Richard is rather looking for OpenQA for testing the GUI unless
>  the FreeCAD functionality can be also tested thru some command line /
>  text mode interface. 
Oh, yeah, I somehow jumped over the references to X in his mail. The
tricky thing there is we can't really run per-package tests for Rawhide
in openQA currently for a couple of reasons:

1) Resources
2) Base stability

the per-package tests (the ones we run on updates) rely on the base
environment - i.e. the underlying distro - being pretty stable, it's no
use to anyone if the tests fail on every update/package not because the
package is broken but because the base image doesn't boot or GNOME is
broken or whatever. This can already be slightly hairy for Branched (I
have to keep a close eye on the tests there and fix or work around
issues quite often) and would be a nightmare for Rawhide.

It's a bit of a chicken-and-egg problem (one way to *make* Rawhide more
stable is to do this sort of testing), but I definitely feel like
whichever one came second in this instance.

We do run *per-compose* tests on Rawhide, of course, but those are kind
of "after the fact": they run on whatever's already gone into Rawhide,
you have to push your change in to find out if it's broken. Which
doesn't sound like what Richard wants in this case.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Przemo Firszt]

W dniu wto, 19.05.2020 o godzinie 17∶50 +0200, użytkownik Dan Horák
napisał:
>  [..]
>  
>  I think Richard is rather looking for OpenQA for testing the GUI
>  unless
>  the FreeCAD functionality can be also tested thru some command line /
>  text mode interface.
>  
>   
"FreeCAD -t 0" performs approx 470 tests. No GUI required. Example
output starts here:
https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103

-- 
Kind regards,
Przemo Firszt

[Richard Shaw]

On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:

>  "FreeCAD -t 0" performs approx 470 tests. No GUI required.
> Example
>  output starts here:
>  https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103 

May need some work to get working from inside mock:

# ./FreeCAD -t 0
FreeCAD 0.18, Libs: 0.18RUnknown
© Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
  #####                 ####  ###   ####
  #                    #      # #   #   #
  #     ##  #### ####  #     #   #  #   #
  ####  # # #  # #  #  #     #####  #   #
  #     #   #### ####  #    #     # #   #
  #     #   #    #     #    #     # #   #  ##  ##  ##
  #     #   #### ####   ### #     # ####   ##  ##  ##

Aborted (core dumped)


Thanks,
Richard

[Aleksandra Fedorova]

On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:
> >
> > "FreeCAD -t 0" performs approx 470 tests. No GUI required. Example
> > output starts here:
> > https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103
>
>
>  May need some work to get working from inside mock:
>
>  # ./FreeCAD -t 0
>  FreeCAD 0.18, Libs: 0.18RUnknown
>  © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
>    #####                 ####  ###   ####
>    #                    #      # #   #   #
>    #     ##  #### ####  #     #   #  #   #
>    ####  # # #  # #  #  #     #####  #   #
>    #     #   #### ####  #    #     # #   #
>    #     #   #    #     #    #     # #   #  ##  ##  ##
>    #     #   #### ####   ### #     # ####   ##  ##  ##
>
>  Aborted (core dumped) 
Please don't run it inside mock.

RPM %check tests are good for unit testing, but they should тще be
used for extensive integration test suites.
The build environment is very different from the target user system.
It has build requirements installed it may have side effects on the
test you run.
Also you would be testing the binary in you working directory, where
you have just executed `make` command.

To run tests in the environment which resembles the actual user
environment we have the STR framework and gating infrastructure [1].

To run the command above as the integration test you need to put
tests/tests.yml file in dist-git repo with the following content:

- hosts: localhost
  roles:
  - role: standard-test-basic
    tags:
    - classic
    tests:
    - simple:
        dir: .
        run: "FreeCAD -t 0"

Note how here you will be using the install FreeCAD binary, not the local one.

[1] https://docs.fedoraproject.org/en-US/ci/how-to-add-dist-git-test/

>  Thanks,
>  Richard
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 
-- 
Aleksandra Fedorova
bookwar

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Richard Shaw]

On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt;
wrote:

>  On Thu, May 21, 2020 at 6:30 PM Richard Shaw
> <hobbes1069(a)gmail.com&gt; wrote:
>  >
>  > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:
>  >>
>  >> "FreeCAD -t 0" performs approx 470 tests. No GUI required. Example
>  >> output starts here:
>  >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103
>  >
>  >
>  > May need some work to get working from inside mock:
>  >
>  > # ./FreeCAD -t 0
>  > FreeCAD 0.18, Libs: 0.18RUnknown
>  > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
>  >   #####                 ####  ###   ####
>  >   #                    #      # #   #   #
>  >   #     ##  #### ####  #     #   #  #   #
>  >   ####  # # #  # #  #  #     #####  #   #
>  >   #     #   #### ####  #    #     # #   #
>  >   #     #   #    #     #    #     # #   #  ##  ##  ##
>  >   #     #   #### ####   ### #     # ####   ##  ##  ##
>  >
>  > Aborted (core dumped)
>
>  Please don't run it inside mock.
>
>  RPM %check tests are good for unit testing, but they should тще be
>  used for extensive integration test suites.
>  The build environment is very different from the target user system.
>  It has build requirements installed it may have side effects on the
>  test you run.
>  Also you would be testing the binary in you working directory, where
>  you have just executed `make` command.
>
>  To run tests in the environment which resembles the actual user
>  environment we have the STR framework and gating infrastructure [1].
>
>  To run the command above as the integration test you need to put
>  tests/tests.yml file in dist-git repo with the following content:
>
>  - hosts: localhost
>    roles:
>    - role: standard-test-basic
>      tags:
>      - classic
>      tests:
>      - simple:
>          dir: .
>          run: "FreeCAD -t 0"
>
>  Note how here you will be using the install FreeCAD binary, not the local
>  one.
>  
Is this only run on "real" builds, or is it possible to run locally or for
scratch builds?

Thanks,
Richard

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Aleksandra Fedorova]

On Thu, May 21, 2020 at 7:04 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt; wrote:
> >
> > On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
> > >
> > > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:
> > >>
> > >> "FreeCAD -t 0" performs approx 470 tests. No GUI required.
> Example
> > >> output starts here:
> > >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103
> > >
> > >
> > > May need some work to get working from inside mock:
> > >
> > > # ./FreeCAD -t 0
> > > FreeCAD 0.18, Libs: 0.18RUnknown
> > > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
> > >   #####                 ####  ###   ####
> > >   #                    #      # #   #   #
> > >   #     ##  #### ####  #     #   #  #   #
> > >   ####  # # #  # #  #  #     #####  #   #
> > >   #     #   #### ####  #    #     # #   #
> > >   #     #   #    #     #    #     # #   #  ##  ##  ##
> > >   #     #   #### ####   ### #     # ####   ##  ##  ##
> > >
> > > Aborted (core dumped)
> >
> > Please don't run it inside mock.
> >
> > RPM %check tests are good for unit testing, but they should тще be
> > used for extensive integration test suites.
> > The build environment is very different from the target user system.
> > It has build requirements installed it may have side effects on the
> > test you run.
> > Also you would be testing the binary in you working directory, where
> > you have just executed `make` command.
> >
> > To run tests in the environment which resembles the actual user
> > environment we have the STR framework and gating infrastructure [1].
> >
> > To run the command above as the integration test you need to put
> > tests/tests.yml file in dist-git repo with the following content:
> >
> > - hosts: localhost
> >   roles:
> >   - role: standard-test-basic
> >     tags:
> >     - classic
> >     tests:
> >     - simple:
> >         dir: .
> >         run: "FreeCAD -t 0"
> >
> > Note how here you will be using the install FreeCAD binary, not the local one.
>
>
>  Is this only run on "real" builds, or is it possible to run locally or for
> scratch builds? 
We run it for pull-requests and for "real builds.

It runs in the following way:

1) we take latest Fedora Rawhide qcow image,
2) we run virtual machine via qemu-kvm from it,
3) inside the vm we install the package,
4) we run the command defined by "run" key in that YAML file.

And it is all orchestrated by Ansible.

It is possible to reproduce the test locally with all the CI wrappers
around it, but I wouldn't recommend it. To debug the test I would just
run the VM with the same Rawhide image [1], install the package and
run the test command manually.
The exact command we use to run qemu can be seen in default_provisioners.log [2]

Now, as soon as you figure out the test itself and you know that it is
working, you can verify that it works when wrapped in CI
orchestration.
You can do that by submitting pull-request with this test
configuration via src.fedoraproject.org.

On pull request we do additional step: we take the patch from the pull
request and submit the scratch build to Koji.
Once scratch build is done, we run the test as described above.

The test result is then published on the pull request page under the
name Fedora CI.
See the right side labels on the example page [3]

[1]
https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-im...
[2]
https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-pr...
[3] https://src.fedoraproject.org/rpms/libdnf-plugin-swidtags/pull-request/5


-- 
Aleksandra Fedorova
bookwar

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Petr Pisar]

On Thu, May 21, 2020 at 07:29:35PM +0200, Aleksandra Fedorova wrote:
>  On Thu, May 21, 2020 at 7:04 PM Richard Shaw
> <hobbes1069(a)gmail.com&gt; wrote:
>  >
>  > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt;
> wrote:
>  >>
>  >> To run the command above as the integration test you need to put
>  >> tests/tests.yml file in dist-git repo with the following content:
>  >>
>  >> - hosts: localhost
>  >>   roles:
>  >>   - role: standard-test-basic
>  >>     tags:
>  >>     - classic
>  >>     tests:
>  >>     - simple:
>  >>         dir: .
>  >>         run: "FreeCAD -t 0"
>  >>
>  >> Note how here you will be using the install FreeCAD binary, not the local one.
>  >
>  >
>  > Is this only run on "real" builds, or is it possible to run locally or for
> scratch builds?
>  
>  We run it for pull-requests and for "real builds.
>  
>  It runs in the following way:
>  
>  1) we take latest Fedora Rawhide qcow image,
>  2) we run virtual machine via qemu-kvm from it,
>  3) inside the vm we install the package,
>  4) we run the command defined by "run" key in that YAML file.
>  
>  And it is all orchestrated by Ansible.
>  
>  It is possible to reproduce the test locally with all the CI wrappers
>  around it, but I wouldn't recommend it. To debug the test I would just
>  run the VM with the same Rawhide image [1], install the package and
>  run the test command manually. 
I'd like to point out that there is a difference in the enviroments between
these two approaches.

While the manual procedure only installs a binary package (and its
dependencies), the OSCI procedure installs all binary packages produced by the
source package (and their depenencies).

In case of freecad, it does not matter, because the freecad component has only
one binary package. But in general it's not true.

And here I'd like to ask Aleksandra whether the same applies to modules or
not. In general case, a module build produces two binary modules - a non-devel
module and a devel module. Does OSCI install components from both of them? And does
it install the components explicitly, or does it rather install a default
profile of the module ("dnf module install" command)?

-- Petr

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Aleksandra Fedorova]

Hi, Petr,

On Fri, May 22, 2020 at 9:29 AM Petr Pisar <ppisar(a)redhat.com&gt; wrote:
>  On Thu, May 21, 2020 at 07:29:35PM +0200, Aleksandra Fedorova wrote:
>  > On Thu, May 21, 2020 at 7:04 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  > >
>  > > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt;
> wrote:
>  > >>
>  > >> To run the command above as the integration test you need to put
>  > >> tests/tests.yml file in dist-git repo with the following content:
>  > >>
>  > >> - hosts: localhost
>  > >>   roles:
>  > >>   - role: standard-test-basic
>  > >>     tags:
>  > >>     - classic
>  > >>     tests:
>  > >>     - simple:
>  > >>         dir: .
>  > >>         run: "FreeCAD -t 0"
>  > >>
>  > >> Note how here you will be using the install FreeCAD binary, not the local
> one.
>  > >
>  > >
>  > > Is this only run on "real" builds, or is it possible to run locally
> or for scratch builds?
>  >
>  > We run it for pull-requests and for "real builds.
>  >
>  > It runs in the following way:
>  >
>  > 1) we take latest Fedora Rawhide qcow image,
>  > 2) we run virtual machine via qemu-kvm from it,
>  > 3) inside the vm we install the package,
>  > 4) we run the command defined by "run" key in that YAML file.
>  >
>  > And it is all orchestrated by Ansible.
>  >
>  > It is possible to reproduce the test locally with all the CI wrappers
>  > around it, but I wouldn't recommend it. To debug the test I would just
>  > run the VM with the same Rawhide image [1], install the package and
>  > run the test command manually.
>
>  I'd like to point out that there is a difference in the enviroments between
>  these two approaches.
>
>  While the manual procedure only installs a binary package (and its
>  dependencies), the OSCI procedure installs all binary packages produced by the
>  source package (and their depenencies). 
You are right here. Jenkins pipeline tries to install all packages
from the koji build before running the test.

And this approach has already hit us many times: there are packages
where binary subpackages conflict with each other (as in curl), or
where different tests require different subpackages to be installed.
There is also a case for reverse dependency check, where we would like
to run test suite from a package A to test the package B, where the
koji build we are installing is different from what is expected by the
test.

In Fedora CI SIG we are discussing if we should just disable this
functionality and make it mandatory to specify all packages explicitly
in the tests.yml under "required_packages" tag, as in [1]

The downside is:
 you would need to maintain list of required packages in the tests.yml

We haven't brought this topic to a wider audience yet, but we should.

>  In case of freecad, it does not matter, because the freecad component
> has only
>  one binary package. But in general it's not true.
>
>  And here I'd like to ask Aleksandra whether the same applies to modules or
>  not. In general case, a module build produces two binary modules - a non-devel
>  module and a devel module. Does OSCI install components from both of them? And does
>  it install the components explicitly, or does it rather install a default
>  profile of the module ("dnf module install" command)? 
We don't have test for modules yet in Fedora CI. So I would say the
question here is:

how do you like it to work?

We can do "dnf module install" but then the same issues would appear:
if you need additional flexibility to test parts of the module, you
would need to revert this default behavior somehow.
Or we can do nothing and only provide environment with repositories
with the updates. So that configuration of the environment is done in
the test metadata.

[1]
https://docs.fedoraproject.org/en-US/ci/how-to-add-dist-git-test/#_tests_...


-- 
Aleksandra Fedorova
bookwar

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Petr Pisar]

On Fri, May 22, 2020 at 01:28:14PM +0200, Aleksandra Fedorova wrote:
>  On Fri, May 22, 2020 at 9:29 AM Petr Pisar <ppisar(a)redhat.com&gt;
> wrote:
>  >
>  > I'd like to point out that there is a difference in the enviroments between
>  > these two approaches.
>  >
>  > While the manual procedure only installs a binary package (and its
>  > dependencies), the OSCI procedure installs all binary packages produced by the
>  > source package (and their depenencies).
>  
>  You are right here. Jenkins pipeline tries to install all packages
>  from the koji build before running the test.
>  
>  And this approach has already hit us many times: there are packages
>  where binary subpackages conflict with each other (as in curl), or
>  where different tests require different subpackages to be installed.
>  There is also a case for reverse dependency check, where we would like
>  to run test suite from a package A to test the package B, where the
>  koji build we are installing is different from what is expected by the
>  test.
>  
>  In Fedora CI SIG we are discussing if we should just disable this
>  functionality and make it mandatory to specify all packages explicitly
>  in the tests.yml under "required_packages" tag, as in [1]
>  
>  The downside is:
>   you would need to maintain list of required packages in the tests.yml
>  
>  We haven't brought this topic to a wider audience yet, but we should.
>   
I don't have the statistics, but I believe that most of Fedora packages only
produce one binary package.

With that assumption, I would recommend defaulting to installing all of the
binary packages of the build. If there were a conflict, the installation would
fail, and a maintainer would fix the test by specifying the required_packages
explicitly. I.e. the required_packages tag should change a semantacs. Instead
of "the additional packages" it would mean "install these package
only". In
other words the required_packages tag would default to the list of binary
packages produced by the build being tested.

That's my ideal approach from point of view of a package maintainer. Of course
I have no idea how difficult the implement is. Maybe it conflicts with the
currecnt OSCI design.

Regarding the case when a test of reverse dependcy fails because the build has
changed, I think it is a standard situation of breaking a compatibility and
the test should fail and the maintainer should decide what to do. E.g. to
move the update into a side tag and adjust the reverse dependency there.

>  > In case of freecad, it does not matter, because the freecad
> component has only
>  > one binary package. But in general it's not true.
>  >
>  > And here I'd like to ask Aleksandra whether the same applies to modules or
>  > not. In general case, a module build produces two binary modules - a non-devel
>  > module and a devel module. Does OSCI install components from both of them? And does
>  > it install the components explicitly, or does it rather install a default
>  > profile of the module ("dnf module install" command)?
>  
>  We don't have test for modules yet in Fedora CI. 
The thing is that producing the devel modules is a matter of MBS
configuration and I feel that Fedora does not produce them in contrast to
CentOS.

>  So I would say the question here is:
>  
>  how do you like it to work?
>  
>  We can do "dnf module install" but then the same issues would appear:
>  if you need additional flexibility to test parts of the module, you
>  would need to revert this default behavior somehow.
>  Or we can do nothing and only provide environment with repositories
>  with the updates. So that configuration of the environment is done in
>  the test metadata.
>   
The devel modules very often contain the pacakges that are only used for
building the non-devel module and building very often includes running unit
tests. That means that running tests for a module, will very probably need
packages from the devel module. Especially when the tests try to
reproduce the unit tests from the build time.

Since the non-devel module is supposed to work without the devel module, it
makes sense to run the tests by default without the devel module. To get to
the production environment as close as possible. On the other hand there will
be a need for having an acccess to the devel module packages.

I would not install the devel module by default, but provide a way of enabling
the devel module when needed.

Your question can also be understand as whether to preinstall all packages
produced by a module. Regardless of the mythical (in Fedora) devel modules.

On the one hand, modules are perceived as a cohesive group of packages. On the
other hand, I cannot preclude that somebody will create a module with curl
and the conflicting subpackages inside. I think I would resort to the same
approach as with the non-modular packages. Default to installing everything
with an option for overriding it to an explicit list.

-- Petr

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Nico Kadel-Garcia]

On Thu, May 21, 2020 at 1:04 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt; wrote:
> >
> > On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
> > >
> > > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:
> > >>
> > >> "FreeCAD -t 0" performs approx 470 tests. No GUI required.
> Example
> > >> output starts here:
> > >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103
> > >
> > >
> > > May need some work to get working from inside mock:
> > >
> > > # ./FreeCAD -t 0
> > > FreeCAD 0.18, Libs: 0.18RUnknown
> > > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
> > >   #####                 ####  ###   ####
> > >   #                    #      # #   #   #
> > >   #     ##  #### ####  #     #   #  #   #
> > >   ####  # # #  # #  #  #     #####  #   #
> > >   #     #   #### ####  #    #     # #   #
> > >   #     #   #    #     #    #     # #   #  ##  ##  ##
> > >   #     #   #### ####   ### #     # ####   ##  ##  ##
> > >
> > > Aborted (core dumped)
> >
> > Please don't run it inside mock. 
Mock testing is extremely useful for testing in a completely defined
environment, for standalone laptop environments with no active
network, and also for testing docker and small VM compatibility. If
tests need to be segregated for entirely local, or isolated testing,
then perhaps the tests can be segregated with some "local" option?

Since a mistyped "%build" step can do "rm -rf /" as the build user,
it's just a lot safer for everyonr in the build environments to use
the mock or other chroot cages or docker images.

FreeCAD dist-git check, WAS:Re: Aggressive updating (Python 3.9): Are we trying to hard?

[Aleksandra Fedorova]

Hi, Nico,

On Fri, May 22, 2020 at 2:38 PM Nico Kadel-Garcia <nkadel(a)gmail.com&gt; wrote:
>  On Thu, May 21, 2020 at 1:04 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  >
>  > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha(a)bookwar.info&gt;
> wrote:
>  >>
>  >> On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1069(a)gmail.com&gt;
> wrote:
>  >> >
>  >> > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo(a)firszt.eu&gt;
> wrote:
>  >> >>
>  >> >> "FreeCAD -t 0" performs approx 470 tests. No GUI required.
> Example
>  >> >> output starts here:
>  >> >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103
>  >> >
>  >> >
>  >> > May need some work to get working from inside mock:
>  >> >
>  >> > # ./FreeCAD -t 0
>  >> > FreeCAD 0.18, Libs: 0.18RUnknown
>  >> > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019
>  >> >   #####                 ####  ###   ####
>  >> >   #                    #      # #   #   #
>  >> >   #     ##  #### ####  #     #   #  #   #
>  >> >   ####  # # #  # #  #  #     #####  #   #
>  >> >   #     #   #### ####  #    #     # #   #
>  >> >   #     #   #    #     #    #     # #   #  ##  ##  ##
>  >> >   #     #   #### ####   ### #     # ####   ##  ##  ##
>  >> >
>  >> > Aborted (core dumped)
>  >>
>  >> Please don't run it inside mock.
>
>  Mock testing is extremely useful for testing in a completely defined
>  environment, for standalone laptop environments with no active
>  network, and also for testing docker and small VM compatibility. If
>  tests need to be segregated for entirely local, or isolated testing,
>  then perhaps the tests can be segregated with some "local" option?
>
>  Since a mistyped "%build" step can do "rm -rf /" as the build user,
>  it's just a lot safer for everyonr in the build environments to use
>  the mock or other chroot cages or docker images. 
I used the wrong term here. Running tests in a mock environment is ok,
if it fits the use case.

What I tried to highlight was running tests as a part of the %check
section of the RPM spec file. This means running tests in a very
specific mock environment of the koji builder during the build of the
binary rpm.

The downsides of this approach are:

1) you don't test the installed package, you test a binary you have just built
You can not check this way that configuration files for that binary
are put into the right places

2) this environment is "dirty" as it has all build files in the
working directory and environment has all build requirements installed
These build files are not going to be packaged and won't appear on the
target environment, but they could cause side effects on the test run.

3) it couples testing with builds.
You can not test package you already have. You can not retrigger test
with the same package if the test failure was caused by infra issue or
the issue with the dependency. You can not reuse test from one package
to test the compose or another package.

4) it creates load on koji builders
When we run integration tests via Fedora CI we use cloud
infrastructure outside of Koji. We can run heavier tests and retrigger
them as much as needed.

-- 
Aleksandra Fedorova
bookwar

[Przemek Klosowski]

On 5/21/20 11:36 AM, Przemo Firszt wrote:
>  "FreeCAD -t 0" performs approx 470 tests. No GUI required.
> Example
>  output starts here: 
That is not the case for me (freecad-0.18.4-5.fc31.x86_64):

    ...

    test60 (PathTests.TestPathLog.TestPathLog)
    Verify track handles no argument. ... TestPathLog(182).test60()
    ok
    test61 (PathTests.TestPathLog.TestPathLog)
    Verify track handles arbitrary argument types correctly. ...
    TestPathLog(189).test61(this, None, 1, 18.25)
    ok
    testzz (PathTests.TestPathLog.TestPathLog)
    Restoring environment after tests. ... ok
    FreeCAD: gl.cpp:2344: const cc_glglue* cc_glglue_instance(int):
    Assertion `current_ctx && "Must have a current GL context when
    instantiating cc_glglue!! (Note: if you are using an old Mesa GL
    version, set the environment variable
    COIN_GL_NO_CURRENT_CONTEXT_CHECK to get around what may be a Mesa
    bug.)"' failed.
    Aborted (core dumped)

I am running  into the problem with Wayland when FreeCAD starts running 
the GUI tests. When I do a workaround for this known issue

https://bugzilla.redhat.com/show_bug.cgi?id=1775688#c2

it continues with the tests including the GUI tests, and finishes , but 
requires IIRC one screen interaction with a popup window.

Ran 370 tests in 74.909s

FAILED (failures=2)
System exit

[Felix Schwarz]

Am 19.05.20 um 14:56 schrieb Igor Raits:
>  I think we should get people who maintain Qt on board when updating
>  Python so that they make sure to backport necessary patches from
>  upstream when we upgrade Python. 
Yeah, I think Richard got pretty unlucky when it comes to PySide2 (though
congrats for his efforts - not a small feat).

Historically PySide is lagging quite a bit behind in terms of Python 3 support
and iirc most (all?) development is done by the QT Company so it works pretty
much like other closed-source company projects (commercial priorities, wait
for stuff to stabilize, only adopt new versions when necessary).

On top PySide2 (and QT) are really a massive amount of complicated code so
just adding a quick Python 3.9 fix is not going to work.

So PySide2 is likely a package which gets the heat from updating QT AND Python...

However: Thank you Richard for taking care of PySide2. I contributed (tiny
bits) to Fedora's PySide/Shiboken packages in the past and I can only guess
how many hours you spent to get the new versions into Fedora.

Felix

PS: And to make things worse I'm not sure how this could work in the future if
the QT Company actually restricts releasing the QT source code by 12 months
(AFAIK this is a rumor and not confirmed).

[Richard Shaw]

On Tue, May 19, 2020 at 8:44 AM Felix Schwarz <fschwarz(a)fedoraproject.org&gt;
wrote:

>  However: Thank you Richard for taking care of PySide2. I contributed (tiny
>  bits) to Fedora's PySide/Shiboken packages in the past and I can only guess
>  how many hours you spent to get the new versions into Fedora.
>  
Thanks! I do overall enjoy contributing to Fedora but like a lot of us 10
year plus packagers, I'm accumulated many packages (some a lot more trouble
than others!) and while I have no intention of taking a hiatus or anything
I'm trying to find a packager life balance :)

I hate having things broken (OCD) which has caused me to work on things
while at $DAYJOB which is ok every now and then but it started happening
too much, but that's another story that I bet a lot us of are
familair with.

Thanks,
Richard

[Felix Schwarz]

Am 19.05.20 um 15:55 schrieb Richard Shaw:
>  Thanks! I do overall enjoy contributing to Fedora but like a lot of
> us 10 year
>  plus packagers, I'm accumulated many packages (some a lot more trouble than
>  others!) and while I have no intention of taking a hiatus or anything I'm
>  trying to find a packager life balance :) 
I can totally feel your pain. Just a few days ago Miro promised me some
cookies and added me to the python-sig. Now pagure says I can commit to 435
packages :-/

Felix

[Igor Raits]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Tue, 2020-05-19 at 16:11 +0200, Felix Schwarz wrote:
>  Am 19.05.20 um 15:55 schrieb Richard Shaw:
>  > Thanks! I do overall enjoy contributing to Fedora but like a lot of
>  > us 10 year
>  > plus packagers, I'm accumulated many packages (some a lot more
>  > trouble than
>  > others!) and while I have no intention of taking a hiatus or
>  > anything I'm
>  > trying to find a packager life balance :)
>  
>  I can totally feel your pain. Just a few days ago Miro promised me
>  some
>  cookies and added me to the python-sig. Now pagure says I can commit
>  to 435
>  packages :-/ 
If you want, I can add yout to the rust-sig so that you can commit to
another ~1k packages, just let me know :)

>  Felix
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: 
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: 
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: 
>  https://lists.fedoraproject.org/archives/list/[e-mail] 
- -- 
Igor Raits <ignatenkobrain(a)fedoraproject.org&gt;
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCgAdFiEEcwgJ58gsbV5f5dMcEV1auJxcHh4FAl7D67wACgkQEV1auJxc
> Hh7WNxAAkuPCKB+pg8su3B1poISacrS6hqSYfGyq269pFzAZ1qCi3hVdEJpf6uP0
> 0qzvdQJGGoOFKNgseje8HXePw0C7BHYyb6ai5E2KYu+bsa5ha4GQYfFhHMlCQ/IV
> 2DkiypoUlAt37pRDEOkdBt4kga6X4eqKDMnzEsTObet3Ev2GJD7hQfCfPm95gn2z
> b1dgZhC+r2lTbziUDC688T8ZVLByOfgtTvNJIHf7lAqJYoM2j4wwFDoQCQje5A5x
> 3UKHhoWugY+dlXqysfzFNBc9p+ygM3uqzvG7G2nrX3F7UWWTOjKO8W2a9PSfh6fP
> 4B39lylnmMUuDClXV9japGRaAbq1+Y9PmSytyOLUpOGpn4/0lsZJFnaet/DNQgxs
> cdTsaJjB6aINSSRK40ODIgUrpFTR2n93+h8OSC8Wv87f4tkX8h2zz0oPkAZoKPJF
> BkbMCZrHLpOnj5KPPTGxBAm14n7+kNMSXnN9Tl+N6K7lrCoGy+NAgz1S+kgTyGGc
> bz5kRoJRonZBTcw44dhuOgI3jAcCthhtoUxlfYLzA9KzLFJs8j+5DVACZNgd4xAe
> iWQlwlCCg4TDHfKnm/D6AtsAwpY6qzSA9ccgCWC+OW2QqX0tUn2mNvXp7bbKlXKm
> P5Pp5u4JOcJQcWLRwh87fAipRwe7OayRVHp9arODl2R2ge0jyjc=
> =TIcA
> -----END PGP SIGNATURE-----

[Orion Poplawski]

On 5/19/20 6:03 AM, Richard Shaw wrote:
>  So I get the whole Fedora first, but...
>  
>  Backstory:
>  
>  FreeCAD has been in a terrible state in Fedora for a couple of years now 
>  and I've nearly given up on trying to maintain the package a few times 
>  now. The previous battle was with the Coin3D stack which finally got 
>  updated to Coin4 in f32 (then Rawhide). I constantly got bugs submitted 
>  that freecad was broken in f31 and my answer was, "Be patient it'll 
>  finally be fixed in Fedora 32 where the Coin3D stack is updated!".
>  
>  However, that was a lie. Unintentional, but a lie nonetheless.
>  
>  Fedora 32 releases and I'm enthusiastic that freecad will finally be 
>  *fixed*! And then I get my first BZ, now PySide2 is broken... Why?
>  
>  Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. 
>  But instead of asking ourselves, "should we push in the VERY latest 
>  Python and hope it's ok?", we just patch the build system to accept it 
>  anyway and hope for the best. 
I'm going to jump on this.  I'm not sure this is what happened in this 
case, but I've run into a couple situations recently where the tests in 
a package were failing - but apparently someone decided that it was more 
important to just get the build in and so disabled failing the build. 
Hopefully with the intention to go back and fix it, but never did.  And 
then lo and behold things were broken.

I guess my hope here is that perhaps we allow build failures hold up 
finishing the rest of a stack's build a bit longer.  Let that pressure 
build to hopefully get some more eyes on really fixing the underlying 
issues.

Again, no idea what happened exactly here.

>  Qt (et all) is a pretty organized upstream, so when asked about
> Python 
>  3.8 support in 5.13.x, they said, "Nope. Wait for 5.14.x."
>  
>  What good does that do me? At the time it wasn't released, and once it 
>  was only Rawhide got it. And I get why. Updating the whole Qt stack and 
>  rebuilding all the dependencies is pretty a pretty painful process.
>  
>  So all that to say, we seem to be taking "Fedora First" to mean, we're 
>  going to update regardless of what it breaks.
>  
>  This is a little more ranty that I intended it to be, and no, I'm not 
>  going to go research and paste a bunch of BZ urls, this isn't about 
>  asking for help, it's more policy related.
>  
>  Hell, I still have one project that all but refuses to update to Python 3!
>  
>  Fortuantly a user found a github project that had the needed patches 
>  backported from 5.14.x! Fingers crossed freecad will work again for more 
>  than 5 seconds.
>  
>  Now I'm being told PySide2 doesn't build with Python 3.9 pre-release. 
>  I'm not surprised! It's not even released yet. I can (and will) inform 
>  upstream. But what do you think their answer is going to be? Wait for 5.15?
>  
>  Ok, rant over.
>  
>  Thanks,
>  Richard 



-- 
Orion Poplawski
Manager of NWRA Technical Systems          720-772-5637
NWRA, Boulder/CoRA Office             FAX: [phone]
3380 Mitchell Lane                       orion(a)nwra.com
Boulder, CO 80301                 https://www.nwra.com/

[Kevin Kofler]

Orion Poplawski wrote:
>  I guess my hope here is that perhaps we allow build failures hold up
>  finishing the rest of a stack's build a bit longer.  Let that pressure
>  build to hopefully get some more eyes on really fixing the underlying
>  issues. 
I am of that mindset too.

Unfortunately, these days, the "Rawhide must be usable at all times" faction 
has the say (and the tooling is set up that way, too, e.g., any broken 
dependency in any release-blocking deliverable will fail the entire Rawhide 
compose, even if we are not anywhere near a release). So just leaving a 
package broken for a couple weeks until it can be fixed *properly* is not 
currently doable, unfortunately. I still think it is the right thing to do 
though.

"Temporary" workarounds tend to become permanent, either because people 
simply forget to turn them off or because the proper fix is not ready in 
time for the release.

        Kevin Kofler

[Miroslav Suchý]

Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
>  Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
> But instead of asking ourselves, "should we push
>  in the VERY latest Python and hope it's ok?", we just patch the build system to
> accept it anyway and hope for the best. 
>  
>  Qt (et all) is a pretty organized upstream, so when asked about Python 3.8 support in
> 5.13.x, they said, "Nope. Wait for
>  5.14.x." 
>   
BTW this is scholar example of use-case for Modularity.

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Richard Shaw]

On Wed, May 20, 2020 at 2:39 PM Miroslav Suchý <msuchy(a)redhat.com&gt; wrote:

>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
>  > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
>  But instead of asking ourselves, "should we push
>  > in the VERY latest Python and hope it's ok?", we just patch the build
>  system to accept it anyway and hope for the best.
>  >
>  > Qt (et all) is a pretty organized upstream, so when asked about Python
>  3.8 support in 5.13.x, they said, "Nope. Wait for
>  > 5.14.x."
>  >
>
>  BTW this is scholar example of use-case for Modularity.
>  
Perhaps, but they are still too cumbersome for the average packager. So I
would have to create a module for Python 3.7, python-pyside2, and freecad,
correct?

Thanks,
Richard

[Martin Langhoff]

On Wed, May 20, 2020 at 3:47 PM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:
>  Perhaps, but they are still too cumbersome for the average packager.
>  So I would have to create a module for Python 3.7, python-pyside2, and freecad, correct?
Maybe Fedora needs parallel installable "python" and "python-next"
stacks?

Primary goal is that everything works on the "python" ("current")
stack, which powers DNF, etc.

Secondary goal is to get "everything" working on python-next.

Once "almost everything" works on python-next it gets promoted to
current python for the next release.

This description is obviously oversimplified, "almost everything"
would be a criteria with some critical packages/functionality being a
must, and some % of "leaf" packages broken being acceptable at
promotion time.

It also doesn't solve _every_ problem -- when various dependencies'
versions can combine in multiple ways, where some work some don't,
there's not much a distro can do without getting into boiling oceans.
OTOH, it does give you a target -- try to make _this_ set work.

Tooling might be needed to streamline the "promotion" process.

cheers,


m
-- 
 martin.langhoff(a)gmail.com
 - ask interesting questions  ~  http://linkedin.com/in/martinlanghoff
 - don't be distracted        ~  http://github.com/martin-langhoff
   by shiny stuff

[Stephen John Smoogen]

On Thu, 21 May 2020 at 08:49, Martin Langhoff <martin.langhoff(a)gmail.com
>  
wrote:

>  On Wed, May 20, 2020 at 3:47 PM Richard Shaw
> <hobbes1069(a)gmail.com&gt; wrote:
>  > Perhaps, but they are still too cumbersome for the average packager.
>  > So I would have to create a module for Python 3.7, python-pyside2, and
>  freecad, correct?
>
>  Maybe Fedora needs parallel installable "python" and "python-next"
> stacks?
>
>  
The issue is that a lot of other software would need python-older and
python-oldest stacks.

There is some amount of software is written for whatever languages is in
the oldest active Ubuntu LTS and the developers really only look to update
it to newer stuff AFTER it has gone EOL. There is another amount of
software written for the latest LTS, and there is another set of software
which is written for the latest. I expect that each of those groups is
large enough to think they are the 'majority' of users of the languages in
question.. so have no urge to move slower/faster because they really don't
have the time to do so. [If you are focusing on the latest features it
takes as much time and energy to slow down and find alternative methods
which work in older stacks.. if you are focusing on long term stability or
your main job is some other thing.. porting some script you wrote to
something newer because XYZ now needs foobaz methods is not a high priority
either.]


>  
-- 
Stephen J Smoogen.

[Martin Langhoff]

Agreed. That a lot of Python is such in 2.7 land I know and understand well
and it's not something Fedora can fix without boiling an ocean or two.

Python 3.x is good but not compelling for many codebases. Maintainership
effort for many projects has dwindled, tech debt must now be paid in one
big lump.

But on the 3.x series, the fragmentation over 3.x releases is a tractable
problem, with a current and next stack.

IMHO :)



m


On Thu, May 21, 2020, 9:18 AM Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:

>  On Thu, 21 May 2020 at 08:49, Martin Langhoff <martin.langhoff(a)gmail.com&gt;
>  wrote:
>
> > On Wed, May 20, 2020 at 3:47 PM Richard Shaw <hobbes1069(a)gmail.com&gt;
> > wrote:
> > > Perhaps, but they are still too cumbersome for the average packager.
> > > So I would have to create a module for Python 3.7, python-pyside2, and
> > freecad, correct?
> >
> > Maybe Fedora needs parallel installable "python" and
> "python-next" stacks?
> >
> >
>  The issue is that a lot of other software would need python-older and
>  python-oldest stacks.
>
>  There is some amount of software is written for whatever languages is in
>  the oldest active Ubuntu LTS and the developers really only look to update
>  it to newer stuff AFTER it has gone EOL. There is another amount of
>  software written for the latest LTS, and there is another set of software
>  which is written for the latest. I expect that each of those groups is
>  large enough to think they are the 'majority' of users of the languages in
>  question.. so have no urge to move slower/faster because they really don't
>  have the time to do so. [If you are focusing on the latest features it
>  takes as much time and energy to slow down and find alternative methods
>  which work in older stacks.. if you are focusing on long term stability or
>  your main job is some other thing.. porting some script you wrote to
>  something newer because XYZ now needs foobaz methods is not a high priority
>  either.]
>
>
> >
>
>  --
>  Stephen J Smoogen.
>
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Stephen John Smoogen]

On Wed, 20 May 2020 at 15:39, Miroslav Suchý <msuchy(a)redhat.com&gt; wrote:

>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
>  > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
>  But instead of asking ourselves, "should we push
>  > in the VERY latest Python and hope it's ok?", we just patch the build
>  system to accept it anyway and hope for the best.
>  >
>  > Qt (et all) is a pretty organized upstream, so when asked about Python
>  3.8 support in 5.13.x, they said, "Nope. Wait for
>  > 5.14.x."
>  >
>
>  BTW this is scholar example of use-case for Modularity.
>
>  
Maybe if each python module is made parallel installable AND each app which
uses python is tied to using that exact version of python versus python3.
Otherwise you could end up with QT apps needing python3.6, XYZ (let us say
TeX) apps needing python3.8 and dnf needing python3.9 and ONLY ONE OF THESE
CAN BE INSTALLED. And since dnf needs to be there.. I guess you would have
no QT or whatever.

If each of them have been patched to be parallel installable and the apps
have been patched to call python3.8 versus python3.. the only thing
modularity is helping is with giving a list of builds orders to make ti
work correctly. It also adds to each of those groups the maintenance of
those python modules because the python team has just enough people to
maintain 1 module. Not 2, not 3, not one for every old version of python3.






>  --
>  Miroslav Suchy, RHCA
>  Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Stephen J Smoogen.

[Igor Raits]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Wed, 2020-05-20 at 21:38 +0200, Miroslav Suchý wrote:
>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
>  > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python
>  > 3.8. But instead of asking ourselves, "should we push
>  > in the VERY latest Python and hope it's ok?", we just patch the
>  > build system to accept it anyway and hope for the best. 
>  > 
>  > Qt (et all) is a pretty organized upstream, so when asked about
>  > Python 3.8 support in 5.13.x, they said, "Nope. Wait for
>  > 5.14.x." 
>  > 
>  
>  BTW this is scholar example of use-case for Modularity. 
Not sure if this is bad or good :)

But anyway, with modularity in this case you would either have python
which is used for all applications or the one which is compatible with
PySide2 because of lack of parallel installability.

>  -- 
>  Miroslav Suchy, RHCA
>  Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: 
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: 
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: 
>  https://lists.fedoraproject.org/archives/list/[e-mail] 
- -- 
Igor Raits <ignatenkobrain(a)fedoraproject.org&gt;
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCgAdFiEEcwgJ58gsbV5f5dMcEV1auJxcHh4FAl7Fi38ACgkQEV1auJxc
> Hh7A/A/8DJSuoUEpOqT6EB9nhHNPQ+d+HelBxQTyg0jaLDSt2tzC+MWV7NrUBwO4
> 0vl7YworVmO9y26FmDSA3QPHzUy/R0jSzXHPJ4+CN1IvpNYVT3vQRukfqYpBAsQr
> zUJlfQeETzBebR+KoQVAhJ4XFEF3pFGDZeXcXIcNKStezcyPCfBVYrl0zizkUbJi
> T5IopFau1DBgRySGMkUj+qiiB8V8KrOufl3HI9Qbo+g7DRw7+3f5qblpuBP4Eamt
> cxjclW5brfUv/96PeX/V/cqkj0P6A+tGPksvSeoAKvEhhVwYMp4yrX8jErnl45aP
> GO2fPcRgNa61Ss6uVQ4SUX+PA67GoN2fiZ12ZKWT/P7/QH6AAyFOWTQXL8LTT8sE
> TAtPtNn4xAG1P/zvjq9oW8iKhYDk6fG7R9serVNVldNLluNKcF4Dl97T+nkFofD/
> GbofiVmf9Lix/jb55n3qEPyK+o8hbY2sp1E0S+zx4DoeBEV9xihS7nR9gOaQ0uxE
> jqBehmFGuWU/xJ4MlbJtfQiXJwssFbIBHrClJvnxpLYkDMUANcw83OBsRUMPPf6/
> Yf4gzhraKR0ZHh/oFtOlczT/mmvKH70/0GO/QVEiFkv5FNBXrs1uZOONnyKJTj4G
> G4n8MU7yXL3qtFGlhbrC1rBw/kUWUwpMqyMeM7g6S/subp5BHZI=
> =8tUT
> -----END PGP SIGNATURE-----

[Miro Hrončok]

On 20. 05. 20 21:38, Miroslav Suchý wrote:
>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
> > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. But instead of
> asking ourselves, "should we push
> > in the VERY latest Python and hope it's ok?", we just patch the build system
> to accept it anyway and hope for the best.
> >
> > Qt (et all) is a pretty organized upstream, so when asked about Python 3.8 support in
> 5.13.x, they said, "Nope. Wait for
> > 5.14.x."
> >
>  
>  BTW this is scholar example of use-case for Modularity. 
I disagree: This is a scholar misconception about Modularity.

Modularity doesn't have parallel installability, hence (as Smooge says in this 
thread) you have a big problem:

- you either end up in an impossible "choose the stack" situation
- or you need to rebuild every Python package in every Python stream
   - that has the maintenance issue
   - you have the same problem for PySide in the 3.8 stream as you had before
- or you can workaround Modularity design by packaging each Python stack as a 
different module with namespaced PRM packages (and components) -- hence you get 
all the usability problems and maintainability problems of Modularity with no 
added value

We could have parallel installable multiple Python version stacks w/out 
Modularity just fine. The "only little problem" is we hardly have the enough 
maintainers to maintain the 3k+ existing Python packages.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Nicolas Mailhot]

Le mercredi 20 mai 2020 à 22:18 +0200, Miro Hrončok a écrit :
>  
>  We could have parallel installable multiple Python version stacks
>  w/out Modularity just fine. The "only little problem" is we hardly
>  have the enough maintainers to maintain the 3k+ existing Python
>  packages. 
BTW, that’s not a Fedora project limitation. The whole GNOME stack has
been trying for years to reduce the combination complexity, by forcing
the use of a limited number of stacks/runtimes.

-- 
Nicolas Mailhot

[Nico Kadel-Garcia]

On Wed, May 20, 2020 at 3:39 PM Miroslav Suchý <msuchy(a)redhat.com&gt; wrote:
>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
>  > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. But instead of
> asking ourselves, "should we push
>  > in the VERY latest Python and hope it's ok?", we just patch the build
> system to accept it anyway and hope for the best.
>  >
>  > Qt (et all) is a pretty organized upstream, so when asked about Python 3.8 support
> in 5.13.x, they said, "Nope. Wait for
>  > 5.14.x."
>  >
>
>  BTW this is scholar example of use-case for Modularity. 
For *python*? Just..... no. the overlap of python modules among and
alongside each  other would be extraordinarily destabilizing. That was
wha the "python2dist" and python3dist" macros provided when used
consistently, so that suites of software could be "python27" or
"python36" prefixed.

The software package name prefixing could be pesky, but was far
simpler to manage.

[Kevin Kofler]

Miroslav Suchý wrote:
>  Dne 19. 05. 20 v 14:03 Richard Shaw napsal(a):
> > Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8. But
> > instead of asking ourselves, "should we push in the VERY latest Python
> > and hope it's ok?", we just patch the build system to accept it anyway
> > and hope for the best.
> > 
> > Qt (et all) is a pretty organized upstream, so when asked about Python
> > 3.8 support in 5.13.x, they said, "Nope. Wait for 5.14.x."
> > 
>  
>  BTW this is scholar example of use-case for Modularity. 
No, this is actually exactly the scholar example of NON-use-case for 
Modularity.

As others have already explained, you just cannot replace Python with an 
older version from a module. That will break the entire distribution. Even 
DNF will stop working, unless you rebuild that too. But rebuilding all the 
Python packages in Fedora against the old version just does not scale.

This is the scholar example of use-case for parallel-installable 
compatibility packages (python3.7 and a python3.7-* for each dependency of 
FreeCAD, while all the other python3-* do not have to be touched at all).

        Kevin Kofler

[Przemo Firszt]

Hi,

W dniu wto, 19.05.2020 o godzinie 07∶03 -0500, użytkownik Richard Shaw
napisał:
>  So I get the whole Fedora first, but...
>  
>  Backstory:
>  
>  FreeCAD has been in a terrible state in Fedora for a couple of years
>  now and I've nearly given up on trying to maintain the package a few
>  times now. The previous battle was with the Coin3D stack which
>  finally got updated to Coin4 in f32 (then Rawhide). I constantly got
>  bugs submitted that freecad was broken in f31 and my answer was, "Be
>  patient it'll finally be fixed in Fedora 32 where the Coin3D stack is
>  updated!". 
>  
>  However, that was a lie. Unintentional, but a lie nonetheless. 
>  
>  Fedora 32 releases and I'm enthusiastic that freecad will finally be
>  *fixed*! And then I get my first BZ, now PySide2 is broken... Why?
>  
>  Because Qt 5.13.x / PySide2 5.13.x is NOT compatible with Python 3.8.
>  But instead of asking ourselves, "should we push in the VERY latest
>  Python and hope it's ok?", we just patch the build system to accept
>  it anyway and hope for the best. 
>  
>  Qt (et all) is a pretty organized upstream, so when asked about
>  Python 3.8 support in 5.13.x, they said, "Nope. Wait for 5.14.x." 
>  
>  What good does that do me? At the time it wasn't released, and once
>  it was only Rawhide got it. And I get why. Updating the whole Qt
>  stack and rebuilding all the dependencies is pretty a pretty painful
>  process.
>  
>  So all that to say, we seem to be taking "Fedora First" to mean,
>  we're going to update regardless of what it breaks. 
>  
>  This is a little more ranty that I intended it to be, and no, I'm not
>  going to go research and paste a bunch of BZ urls, this isn't about
>  asking for help, it's more policy related. 
>  
>  Hell, I still have one project that all but refuses to update to
>  Python 3!
>  
>  Fortuantly a user found a github project that had the needed patches
>  backported from 5.14.x! Fingers crossed freecad will work again for
>  more than 5 seconds.
>  
>  Now I'm being told PySide2 doesn't build with Python 3.9 pre-release. 
>  I'm not surprised! It's not even released yet. I can (and will)
>  inform upstream. But what do you think their answer is going to be?
>  Wait for 5.15?
>  
>  Ok, rant over. 
I think it's time for me to activate.. My name is Przemo, I'm long time
redhat/PLD/fedora user with some experience in packaging. debugging,
etc. I keep building FreeCAD since fedora 20, I also maintain FreeCAD
nightly builds on COPR [1]. I experienced first hand all the problems
Richard mentioned above and more. Richard's "rant" is a bit broader
that this, but setting up automated FreeCAD tests would alleviate the
problem and give heads up about upcoming issues. The problem with
PySide didn't show up during compilation, but FreeCAD self-tests
captured it. FreeCAD self-test are currently run on every commit on
travis-CI system (debian/ubuntu based) and it works great.

I'll be happy to help.

[1] https://copr.fedorainfracloud.org/coprs/g/freecad/nightly/
-- 
Kind regards,
Przemo Firszt

[Richard Shaw]

On Thu, May 21, 2020 at 5:48 AM Przemo Firszt <przemo(a)firszt.eu&gt; wrote:

>  I think it's time for me to activate.. My name is Przemo, I'm long time
>  redhat/PLD/fedora user with some experience in packaging. debugging,
>  etc. I keep building FreeCAD since fedora 20, I also maintain FreeCAD
>  nightly builds on COPR [1]. I experienced first hand all the problems
>  Richard mentioned above and more. Richard's "rant" is a bit broader
>  that this, but setting up automated FreeCAD tests would alleviate the
>  problem and give heads up about upcoming issues. The problem with
>  PySide didn't show up during compilation, but FreeCAD self-tests
>  captured it. FreeCAD self-test are currently run on every commit on
>  travis-CI system (debian/ubuntu based) and it works great.
>  
Any help would be appreciated. Are you already a packager or do you need a
sponsor?

I looked through the CMake config for testing and found a Test target but
it didn't seem to do anything, nor is there a "make test" or "make
check"
target in the Makefile.

Thanks,
Richard