How many tmt tests are onboarded to Fedora CI?

[Miroslav Vadkerti]

Hi Matt,

I wanted to follow up on this question you had on DevConf.US 22 [1].

We would like to make these stats available to the community, unfortunately
now it is just a DB query to our Testing Farm database.

So, currently that is: 449 packages

I am attaching lists for proof.

There are also a bunch of shared tmt tests in tests dist-git tests
namespace, seems 83 of them:

https://src.fedoraproject.org/projects/tests/%2A

Best regards,
/M

[1] https://youtu.be/M2FXrUs3YjE?t=3121

-- 
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE
IRC mvadkert #tft #tmt #osci :: Mobile [phone]
Remote Czech Republic :: Red Hat Czech s.r.o

[Miro Hrončok]

On 25. 08. 22 19:46, Miroslav Vadkerti wrote:
>  Hi Matt,
>  
>  I wanted to follow up on this question you had on DevConf.US 22 [1].
>  
>  We would like to make these stats available to the community, unfortunately now 
>  it is just a DB query to our Testing Farm database.
>  
>  So, currently that is: 449 packages
>  
>  I am attaching lists for proof. 

I recognize some false positives in the list:


>  https://src.fedoraproject.org/rpms/pyproject-rpm-macros
>  https://src.fedoraproject.org/rpms/pypy
>  https://src.fedoraproject.org/rpms/python-packaging 
The packages use STI, not TMT.



>  https://src.fedoraproject.org/rpms/pypy3 
This package is retired on all branches, cannot use anything, used STI before, 
not TMT.


>  https://src.fedoraproject.org/rpms/python-dotenv
>  https://src.fedoraproject.org/rpms/python-requests
>  https://src.fedoraproject.org/rpms/python-shellingham
>  https://src.fedoraproject.org/rpms/python-urllib3 
The TMT tests were removed ~1 year ago.



There might be more such cases, I only checked a handful of packages I know or 
co-maintain.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Miroslav Vadkerti]

Thanks Miro,

The false positives come from the fact that we gathered the requests data
of Testing Farm, a final check
to check if that corresponds with the current state would get rid of those
false positives.

I will not more spend now more time on it, I believe it is only a few
occasions.

But thanks for taking the time and looking into it.

Best regards,
/M

On Mon, Aug 29, 2022 at 1:02 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  On 25. 08. 22 19:46, Miroslav Vadkerti wrote:
>  > Hi Matt,
>  >
>  > I wanted to follow up on this question you had on DevConf.US 22 [1].
>  >
>  > We would like to make these stats available to the community,
>  unfortunately now
>  > it is just a DB query to our Testing Farm database.
>  >
>  > So, currently that is: 449 packages
>  >
>  > I am attaching lists for proof.
>
>
>  I recognize some false positives in the list:
>
>
>   > https://src.fedoraproject.org/rpms/pyproject-rpm-macros
>   > https://src.fedoraproject.org/rpms/pypy
>   > https://src.fedoraproject.org/rpms/python-packaging
>
>  The packages use STI, not TMT.
>
>
>
>   > https://src.fedoraproject.org/rpms/pypy3
>
>  This package is retired on all branches, cannot use anything, used STI
>  before,
>  not TMT.
>
>
>   > https://src.fedoraproject.org/rpms/python-dotenv
>   > https://src.fedoraproject.org/rpms/python-requests
>   > https://src.fedoraproject.org/rpms/python-shellingham
>   > https://src.fedoraproject.org/rpms/python-urllib3
>
>  The TMT tests were removed ~1 year ago.
>
>
>
>  There might be more such cases, I only checked a handful of packages I
>  know or
>  co-maintain.
>
>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>
>  
-- 
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE
IRC mvadkert #tft #tmt #osci :: Mobile [phone]
Remote Czech Republic :: Red Hat Czech s.r.o

[Matthew Miller]

Thanks to both of you. Could we get the tests onto the Packager Dashboard?
I can file a pr at https://pagure.io/fedora-qa/packager_dashboard/issues,
or co-sign one if someone else wants to. :)

On Fri, Sep 9, 2022 at 4:14 AM Miroslav Vadkerti <mvadkert(a)redhat.com&gt;
wrote:

>  Thanks Miro,
>
>  The false positives come from the fact that we gathered the requests data
>  of Testing Farm, a final check
>  to check if that corresponds with the current state would get rid of those
>  false positives.
>
>  I will not more spend now more time on it, I believe it is only a few
>  occasions.
>
>  But thanks for taking the time and looking into it.
>
>  Best regards,
>  /M
>
>  On Mon, Aug 29, 2022 at 1:02 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>
> > On 25. 08. 22 19:46, Miroslav Vadkerti wrote:
> > > Hi Matt,
> > >
> > > I wanted to follow up on this question you had on DevConf.US 22 [1].
> > >
> > > We would like to make these stats available to the community,
> > unfortunately now
> > > it is just a DB query to our Testing Farm database.
> > >
> > > So, currently that is: 449 packages
> > >
> > > I am attaching lists for proof.
> >
> >
> > I recognize some false positives in the list:
> >
> >
> >  > https://src.fedoraproject.org/rpms/pyproject-rpm-macros
> >  > https://src.fedoraproject.org/rpms/pypy
> >  > https://src.fedoraproject.org/rpms/python-packaging
> >
> > The packages use STI, not TMT.
> >
> >
> >
> >  > https://src.fedoraproject.org/rpms/pypy3
> >
> > This package is retired on all branches, cannot use anything, used STI
> > before,
> > not TMT.
> >
> >
> >  > https://src.fedoraproject.org/rpms/python-dotenv
> >  > https://src.fedoraproject.org/rpms/python-requests
> >  > https://src.fedoraproject.org/rpms/python-shellingham
> >  > https://src.fedoraproject.org/rpms/python-urllib3
> >
> > The TMT tests were removed ~1 year ago.
> >
> >
> >
> > There might be more such cases, I only checked a handful of packages I
> > know or
> > co-maintain.
> >
> > --
> > Miro Hrončok
> > --
> > Phone: [phone]
> > IRC: mhroncok
> >
> >
>
>  --
>  Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE
>  IRC mvadkert #tft #tmt #osci :: Mobile [phone]
>  Remote Czech Republic :: Red Hat Czech s.r.o
>
>
>

[Miro Hrončok]

On 13. 09. 22 16:20, Matthew Miller wrote:
>  Thanks to both of you. Could we get the tests onto the Packager
> Dashboard? I 
>  can file a pr at https://pagure.io/fedora-qa/packager_dashboard/issues 
>  <https://pagure.io/fedora-qa/packager_dashboard/issues>;, or co-sign one if 
>  someone else wants to. :) 
The dashboard shows active PRs, updates, overrides, bugzillas or otherwise 
actionable items. I have no idea why and how we would list tmt tests there. 
What do you have in mind?

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Matthew Miller]

On Tue, Sep 13, 2022 at 10:28 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  The dashboard shows active PRs, updates, overrides, bugzillas or otherwise
>  actionable items. I have no idea why and how we would list tmt tests
>  there.
>  What do you have in mind?
>
>  
* "This package has no tests. You can create them easily ..." [Link to
instructions for creating tests.]
* "Tests for this package have failed since [date]"

... and similar?

[Miroslav Vadkerti]

Hi,

Sounds reasonable to me, we even talked about that few years ago with
frantisekz.

Let's start with an issue.

Best regards,
/M

On Tue, Sep 13, 2022, 4:57 PM Matthew Miller <mattdm(a)redhat.com&gt; wrote:

>  On Tue, Sep 13, 2022 at 10:28 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>
> >
> > The dashboard shows active PRs, updates, overrides, bugzillas or
> > otherwise
> > actionable items. I have no idea why and how we would list tmt tests
> > there.
> > What do you have in mind?
> >
> >
>  * "This package has no tests. You can create them easily ..." [Link to
>  instructions for creating tests.]
>  * "Tests for this package have failed since [date]"
>
>  ... and similar?
>
>

[Matthew Miller]

https://pagure.io/fedora-qa/packager_dashboard/issue/161

On Tue, Sep 13, 2022 at 11:52 AM Miroslav Vadkerti <mvadkert(a)redhat.com&gt;
wrote:

>  Hi,
>
>  Sounds reasonable to me, we even talked about that few years ago with
>  frantisekz.
>
>  Let's start with an issue.
>
>  Best regards,
>  /M
>
>  On Tue, Sep 13, 2022, 4:57 PM Matthew Miller <mattdm(a)redhat.com&gt; wrote:
>
> >
> > On Tue, Sep 13, 2022 at 10:28 AM Miro Hrončok <mhroncok(a)redhat.com&gt;
> > wrote:
> >
> >>
> >> The dashboard shows active PRs, updates, overrides, bugzillas or
> >> otherwise
> >> actionable items. I have no idea why and how we would list tmt tests
> >> there.
> >> What do you have in mind?
> >>
> >>
> > * "This package has no tests. You can create them easily ..." [Link to
> > instructions for creating tests.]
> > * "Tests for this package have failed since [date]"
> >
> > ... and similar?
> >
> >