What to do when Fedora CI run failed?

[Jan Pazdziora]

Hello,

I have run fedora-rawhide-build-pipeline

	https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...

that failed in the

	cloud-image-compose

stage, something which (I believe) have very little control over.

My other build on f30

	https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...

resulted in orange exclamation mark status, in spite of the fact that
the checkboxes in each of the states are all green.

What does it mean and how should I fix it?

What is the recommended way to remedy the situation?

Thank you,

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat

[Johnny Bieren]

On Tue, Jun 4, 2019 at 10:16 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt; wrote:

>  Hello,
>
>  I have run fedora-rawhide-build-pipeline
>
>
>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>
>  that failed in the
>
>          cloud-image-compose
>
>  stage, something which (I believe) have very little control over.
>  
I do agree that this failure does not appear to be due to anything you did.
It appears (to me) to be:
*13:09:06* process apparently never started in
/workDir/workspace/fedora-rawhide-build-pipeline@tmp/durable-3b02e0b5
@Bruno Goncalves <bgoncalv(a)redhat.com&gt; what is the proper way to retrigger
tests for the build pipeline? I could manually restart the jenkins build,
but I am not sure if there is a more official way to do it

>  My other build on f30
>
>
>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>
>  resulted in orange exclamation mark status, in spite of the fact that
>  the checkboxes in each of the states are all green.
>
>  What does it mean and how should I fix it?
>  
The orange exclamation mark status means unstable, which means everything
in the build completed, but the standard-test-roles test failed. You can
see the logs here.. some are prepended with FAIL
https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...


>  What is the recommended way to remedy the situation?
>
>  Thank you,
>
>  --
>  Jan Pazdziora
>  Senior Principal Software Engineer, Security Engineering, Red Hat
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Miroslav Vadkerti]

Hi,

On Tue, Jun 4, 2019 at 8:13 PM Johnny Bieren <jbieren(a)redhat.com&gt; wrote:

>  On Tue, Jun 4, 2019 at 10:16 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt;
>  wrote:
>
> >
> > Hello,
> >
> > I have run fedora-rawhide-build-pipeline
> >
> >
> >
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
> >
> > that failed in the
> >
> >         cloud-image-compose
> >
> > stage, something which (I believe) have very little control over.
> >
>  I do agree that this failure does not appear to be due to anything you
>  did. It appears (to me) to be:
>  *13:09:06* process apparently never started in
>  /workDir/workspace/fedora-rawhide-build-pipeline@tmp/durable-3b02e0b5
>  @Bruno Goncalves <bgoncalv(a)redhat.com&gt; what is the proper way to
>  retrigger tests for the build pipeline? I could manually restart the
>  jenkins build, but I am not sure if there is a more official way to do it
>  
Unfortunately, afaik, currently only people with login access to the
openshift cluster have ability to rebuild (as OSCI we can do it), basically
we are missing jenkins to be kerberized with Fedora's kerberos, so we can
give this ability to the end users ...



> > My other build on f30
> >
> >
> >
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
> >
> > resulted in orange exclamation mark status, in spite of the fact that
> > the checkboxes in each of the states are all green.
> >
> > What does it mean and how should I fix it?
> >
>  The orange exclamation mark status means unstable, which means everything
>  in the build completed, but the standard-test-roles test failed. You can
>  see the logs here.. some are prepended with FAIL
>
>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>
>
> >
> > What is the recommended way to remedy the situation?
> >
> > Thank you,
> >
> > --
> > Jan Pazdziora
> > Senior Principal Software Engineer, Security Engineering, Red Hat
> > _______________________________________________
> > CI mailing list -- ci(a)lists.fedoraproject.org
> > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]
> >
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile [phone]
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR

[Bruno Goncalves]

>  Hi,
>  
>  On Tue, Jun 4, 2019 at 8:13 PM Johnny Bieren <jbieren(a)redhat.com&gt; wrote:
>  
>  
>  Unfortunately, afaik, currently only people with login access to the
>  openshift cluster have ability to rebuild (as OSCI we can do it), basically
>  we are missing jenkins to be kerberized with Fedora's kerberos, so we can
>  give this ability to the end users ... 
And regarding the fail of
https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
it seems to be related to https://pagure.io/fedora-ci/general/issue/54

[Jan Pazdziora]

On Tue, Jun 04, 2019 at 02:12:45PM -0400, Johnny Bieren wrote:
>  On Tue, Jun 4, 2019 at 10:16 AM Jan Pazdziora
> <jpazdziora(a)redhat.com&gt; wrote:
>  
>  > My other build on f30
>  >
>  >
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>  >
>  > resulted in orange exclamation mark status, in spite of the fact that
>  > the checkboxes in each of the states are all green.
>  >
>  > What does it mean and how should I fix it?
>
>  The orange exclamation mark status means unstable, which means everything
>  in the build completed, but the standard-test-roles test failed. You can
>  see the logs here.. some are prepended with FAIL
>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
Ah, thank you.

I've incorrectly assumed that /tmp/package-test.sh itself will be
shown as failed.

It seems repositories in the test environments are configured with
gpgcheck even if the packages are not singed:

	https://lists.fedoraproject.org/archives/list/[e-mail]/...

-- 
Jan Pazdziora | adelton at #brno, #swid
Sr. Principal Software Engineer, Security Engineering, Red Hat

[Jan Pazdziora]

On Wed, Jun 05, 2019 at 12:03:01AM +0200, Miroslav Vadkerti wrote:
>  
>  Unfortunately, afaik, currently only people with login access to the
>  openshift cluster have ability to rebuild (as OSCI we can do it), basically
>  we are missing jenkins to be kerberized with Fedora's kerberos, so we can
>  give this ability to the end users ... 
Is that requirement / effort tracked somewhere / somehow? What is the
timeline of providing that functionality? And what is the expected
current workflow for reruning the test? Filing some ticket with the
OSCI team somehow?

-- 
Jan Pazdziora | adelton at #brno, #swid
Sr. Principal Software Engineer, Security Engineering, Red Hat

[Tim Flink]

On Wed, 5 Jun 2019 00:03:01 +0200
Miroslav Vadkerti <mvadkert(a)redhat.com&gt; wrote:

>  Hi,
>  
>  On Tue, Jun 4, 2019 at 8:13 PM Johnny Bieren <jbieren(a)redhat.com&gt;
>  wrote:
>  
>  >
>  >
>  > On Tue, Jun 4, 2019 at 10:16 AM Jan Pazdziora
>  > <jpazdziora(a)redhat.com&gt; wrote:
>  >  
>  >>
>  >> Hello,
>  >>
>  >> I have run fedora-rawhide-build-pipeline
>  >>
>  >>
>  >>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>  >>
>  >> that failed in the
>  >>
>  >>         cloud-image-compose
>  >>
>  >> stage, something which (I believe) have very little control over.
>  >>  
>  > I do agree that this failure does not appear to be due to anything
>  > you did. It appears (to me) to be:
>  > *13:09:06* process apparently never started in
>  > /workDir/workspace/fedora-rawhide-build-pipeline@tmp/durable-3b02e0b5
>  > @Bruno Goncalves <bgoncalv(a)redhat.com&gt; what is the proper way to
>  > retrigger tests for the build pipeline? I could manually restart the
>  > jenkins build, but I am not sure if there is a more official way to
>  > do it 
>  
>  Unfortunately, afaik, currently only people with login access to the
>  openshift cluster have ability to rebuild (as OSCI we can do it),
>  basically we are missing jenkins to be kerberized with Fedora's
>  kerberos, so we can give this ability to the end users ... 
As I recall, the infra folks don't want Fedora's kerberos to be used
for anything beyond koji. I do not recall the details behind that,
though and it's possible that things could have changed.

That being said, there is support for Fedora OAuth logins in Jenkins as
shown in this template:

https://pagure.io/fedora-infra-jenkins/

That support does allow restricting functions to FAS groups:

https://pagure.io/fedora-infra-jenkins/blob/master/f/s2i/jenkins/master/c...

Tim

<snip>

[Miroslav Vadkerti]

On Wed, Jun 5, 2019 at 5:49 PM Tim Flink <tflink(a)redhat.com&gt; wrote:

>  On Wed, 5 Jun 2019 00:03:01 +0200
>  Miroslav Vadkerti <mvadkert(a)redhat.com&gt; wrote:
>
>  > Hi,
>  >
>  > On Tue, Jun 4, 2019 at 8:13 PM Johnny Bieren <jbieren(a)redhat.com&gt;
>  > wrote:
>  >
>  > >
>  > >
>  > > On Tue, Jun 4, 2019 at 10:16 AM Jan Pazdziora
>  > > <jpazdziora(a)redhat.com&gt; wrote:
>  > >
>  > >>
>  > >> Hello,
>  > >>
>  > >> I have run fedora-rawhide-build-pipeline
>  > >>
>  > >>
>  > >>
>
> https://jenkins-continuous-infra.apps.ci.centos.org/blue/organizations/je...
>  > >>
>  > >> that failed in the
>  > >>
>  > >>         cloud-image-compose
>  > >>
>  > >> stage, something which (I believe) have very little control over.
>  > >>
>  > > I do agree that this failure does not appear to be due to anything
>  > > you did. It appears (to me) to be:
>  > > *13:09:06* process apparently never started in
>  > > /workDir/workspace/fedora-rawhide-build-pipeline@tmp/durable-3b02e0b5
>  > > @Bruno Goncalves <bgoncalv(a)redhat.com&gt; what is the proper way to
>  > > retrigger tests for the build pipeline? I could manually restart the
>  > > jenkins build, but I am not sure if there is a more official way to
>  > > do it
>  >
>  > Unfortunately, afaik, currently only people with login access to the
>  > openshift cluster have ability to rebuild (as OSCI we can do it),
>  > basically we are missing jenkins to be kerberized with Fedora's
>  > kerberos, so we can give this ability to the end users ...
>
>  As I recall, the infra folks don't want Fedora's kerberos to be used
>  for anything beyond koji. I do not recall the details behind that,
>  though and it's possible that things could have changed.
>
>  That being said, there is support for Fedora OAuth logins in Jenkins as
>  shown in this template:
>
>  https://pagure.io/fedora-infra-jenkins/
>
>  That support does allow restricting functions to FAS groups:
>
>
>
> https://pagure.io/fedora-infra-jenkins/blob/master/f/s2i/jenkins/master/c...

Thanks for the pointers Tim! I have filled taiga issue to track this, maybe
Bruno or somebody could help with this:

https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114

Or anybody else :)

Best regards,
/M



>  Tim
>
>  <snip>
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile [phone]
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR

[Jan Pazdziora]

On Wed, Jun 05, 2019 at 06:07:51PM +0200, Miroslav Vadkerti wrote:
>  
>  Thanks for the pointers Tim! I have filled taiga issue to track this, maybe
>  Bruno or somebody could help with this:
>  
>  https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114 
Thank you for tracking this.

However, could you please also provide guidance for people who might
need their Jenkins build/run restarted now? How do they get hold of
the OSCI team to to it? Or what is the process?

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat

[Miroslav Vadkerti]

On Thu, Jun 6, 2019 at 11:57 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt; wrote:

>  On Wed, Jun 05, 2019 at 06:07:51PM +0200, Miroslav Vadkerti wrote:
>  >
>  > Thanks for the pointers Tim! I have filled taiga issue to track this,
>  maybe
>  > Bruno or somebody could help with this:
>  >
>  > https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114
>
>  Thank you for tracking this.
>
>  However, could you please also provide guidance for people who might
>  need their Jenkins build/run restarted now? How do they get hold of
>  the OSCI team to to it? Or what is the process?
>  
Sure, sorry, so AFAIK you send your gpg encrypted password according to
your preference
to Brian

@brian is that so?



>  --
>  Jan Pazdziora
>  Senior Principal Software Engineer, Security Engineering, Red Hat
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile [phone]
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR

[Brian Stinson]

For now it's best to keep these requests to #fedora-ci on IRC. If we
have a few folks that are interested in picking up some packager
response duties here, let me know and we can discuss with the current
group of admins.

The Jenkins instances dedicated to the Fedora-CI package tests aren't
really meant for wider authentication beyond a few on the admin team.
That is to say, we won't allow open access to any packager.

What I'd rather see is a RFE somewhere (not sure where yet) such that
we have UX to retrigger from one of the flagship applications (Bodhi
or dist-git).

On Thu, Jun 6, 2019 at 12:56 PM Miroslav Vadkerti <mvadkert(a)redhat.com&gt; wrote:
>
>
>
> On Thu, Jun 6, 2019 at 11:57 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt; wrote:
>>
>> On Wed, Jun 05, 2019 at 06:07:51PM +0200, Miroslav Vadkerti wrote:
>> >
>> > Thanks for the pointers Tim! I have filled taiga issue to track this, maybe
>> > Bruno or somebody could help with this:
>> >
>> > https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114
>>
>> Thank you for tracking this.
>>
>> However, could you please also provide guidance for people who might
>> need their Jenkins build/run restarted now? How do they get hold of
>> the OSCI team to to it? Or what is the process?
>
>
> Sure, sorry, so AFAIK you send your gpg encrypted password according to your
preference
> to Brian
>
> @brian is that so?
>
>
>>
>>
>> --
>> Jan Pazdziora
>> Senior Principal Software Engineer, Security Engineering, Red Hat
>> _______________________________________________
>> CI mailing list -- ci(a)lists.fedoraproject.org
>> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
>
>
>
> --
> Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
> IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
> TPB-C 2C215 :: Mobile [phone]
> Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR
>
>

[Clement Verna]

On Thu, 6 Jun 2019 at 20:36, Brian Stinson <bstinson(a)redhat.com&gt; wrote:
>  For now it's best to keep these requests to #fedora-ci on IRC. If we
>  have a few folks that are interested in picking up some packager
>  response duties here, let me know and we can discuss with the current
>  group of admins.
>
>  The Jenkins instances dedicated to the Fedora-CI package tests aren't
>  really meant for wider authentication beyond a few on the admin team.
>  That is to say, we won't allow open access to any packager.
>
>  What I'd rather see is a RFE somewhere (not sure where yet) such that
>  we have UX to retrigger from one of the flagship applications (Bodhi
>  or dist-git). 
We are planning to add this feature to Bodhi as part of the Rawhide
Gating work going on there.
If you want to follow that we have ticket for it
(https://github.com/fedora-infra/bodhi/issues/3285)

>
> On Thu, Jun 6, 2019 at 12:56 PM Miroslav Vadkerti <mvadkert(a)redhat.com&gt; wrote:
> >
> >
> >
> > On Thu, Jun 6, 2019 at 11:57 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt;
wrote:
> >>
> >> On Wed, Jun 05, 2019 at 06:07:51PM +0200, Miroslav Vadkerti wrote:
> >> >
> >> > Thanks for the pointers Tim! I have filled taiga issue to track this,
maybe
> >> > Bruno or somebody could help with this:
> >> >
> >> > https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114
> >>
> >> Thank you for tracking this.
> >>
> >> However, could you please also provide guidance for people who might
> >> need their Jenkins build/run restarted now? How do they get hold of
> >> the OSCI team to to it? Or what is the process?
> >
> >
> > Sure, sorry, so AFAIK you send your gpg encrypted password according to your
preference
> > to Brian
> >
> > @brian is that so?
> >
> >
> >>
> >>
> >> --
> >> Jan Pazdziora
> >> Senior Principal Software Engineer, Security Engineering, Red Hat
> >> _______________________________________________
> >> CI mailing list -- ci(a)lists.fedoraproject.org
> >> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> >> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> >
> >
> >
> > --
> > Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
> > IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
> > TPB-C 2C215 :: Mobile [phone]
> > Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR
> >
> >
> _______________________________________________
> CI mailing list -- ci(a)lists.fedoraproject.org
> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Miroslav Vadkerti]

Cool, thanks! Will reply there

On Thu, Jun 6, 2019 at 9:26 PM Clement Verna <cverna(a)fedoraproject.org&gt;
wrote:

>  On Thu, 6 Jun 2019 at 20:36, Brian Stinson
> <bstinson(a)redhat.com&gt; wrote:
>  >
>  > For now it's best to keep these requests to #fedora-ci on IRC. If we
>  > have a few folks that are interested in picking up some packager
>  > response duties here, let me know and we can discuss with the current
>  > group of admins.
>  >
>  > The Jenkins instances dedicated to the Fedora-CI package tests aren't
>  > really meant for wider authentication beyond a few on the admin team.
>  > That is to say, we won't allow open access to any packager.
>  >
>  > What I'd rather see is a RFE somewhere (not sure where yet) such that
>  > we have UX to retrigger from one of the flagship applications (Bodhi
>  > or dist-git).
>
>  We are planning to add this feature to Bodhi as part of the Rawhide
>  Gating work going on there.
>  If you want to follow that we have ticket for it
>  (https://github.com/fedora-infra/bodhi/issues/3285)
>
>  >
>  > On Thu, Jun 6, 2019 at 12:56 PM Miroslav Vadkerti <mvadkert(a)redhat.com&gt;
>  wrote:
>  > >
>  > >
>  > >
>  > > On Thu, Jun 6, 2019 at 11:57 AM Jan Pazdziora <jpazdziora(a)redhat.com&gt;
>  wrote:
>  > >>
>  > >> On Wed, Jun 05, 2019 at 06:07:51PM +0200, Miroslav Vadkerti wrote:
>  > >> >
>  > >> > Thanks for the pointers Tim! I have filled taiga issue to track
>  this, maybe
>  > >> > Bruno or somebody could help with this:
>  > >> >
>  > >> > https://teams.fedoraproject.org/project/ci/us/12?kanban-status=114
>  > >>
>  > >> Thank you for tracking this.
>  > >>
>  > >> However, could you please also provide guidance for people who might
>  > >> need their Jenkins build/run restarted now? How do they get hold of
>  > >> the OSCI team to to it? Or what is the process?
>  > >
>  > >
>  > > Sure, sorry, so AFAIK you send your gpg encrypted password according
>  to your preference
>  > > to Brian
>  > >
>  > > @brian is that so?
>  > >
>  > >
>  > >>
>  > >>
>  > >> --
>  > >> Jan Pazdziora
>  > >> Senior Principal Software Engineer, Security Engineering, Red Hat
>  > >> _______________________________________________
>  > >> CI mailing list -- ci(a)lists.fedoraproject.org
>  > >> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  > >> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  > >> List Guidelines:
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > >> List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  > >
>  > >
>  > >
>  > > --
>  > > Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
>  > > IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
>  > > TPB-C 2C215 :: Mobile [phone]
>  > > Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR
>  > >
>  > >
>  > _______________________________________________
>  > CI mailing list -- ci(a)lists.fedoraproject.org
>  > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Miroslav Vadkerti :: Principal QE :: BaseOS QE / OSCI
IRC mvadkert #osci #baseosci #qe :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile [phone]
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR