Re: Trying to get some tests working

[Andrei Stepanov]

On Fri, Jul 20, 2018 at 1:54 PM Igor Gnatenko <
ignatenkobrain(a)fedoraproject.org&gt; wrote:

>  On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov
> <astepano(a)redhat.com&gt;
>  wrote:
>
> > On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko <
> > ignatenkobrain(a)fedoraproject.org&gt; wrote:
> >
> >> I'm trying to have shared tests between all 400+ rust packages.
> >>
> >> I have few problems though:
> >>
> >> * There is no package NEVRA in test environment so the shared test
> >> doesn't know what to test - it is necessary to know which test to run.
> >>
> >
> > We trigger tests for dist-git.
> > I think about dist-git as a SRPM.
> > SRPM can produce many RPMs.
> > CI pipeline: install all RPMS from Koji build and run tests from /tests
> > for all installed RPMs.
> > The same approach is for manual testing.
> > Conclusion: test-environment-system has installed RPMS.
> >
>
>  Just want to mention that this won't work if your subpackages conflict
>  (e.g. curl and curl-minimal.
>  
Right, and I would ask contra@ team or other guys for clarification, how CI
pipeline does care about above moment?


>  Tests from /tests dir check test-environment-system as a whole.
> >
>
> > https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't define
> > NEVRA.
> > Reason: STI tests are for all test-environment-system (all installed
> > packages).
> > /tests directory from dist-git knows that packages goes from dist-git/pkg
> > /tests/tests.yml is written for each SRPM package.
> > /tests/tests.yml knows that it will run against: SRPM + distro version.
> > /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files.
> > If you have common tests, you have different /tests/tests.yml where you
> > can specify exactly that this tests are for this SRPM and for Branch 28
> > (for example).
> >
>
>   This is wwhat I have done so far, however test runner has all this
>  information, why not to simply pass it to the tests rather than asking
>  every maintainer to do that manually.
>
>
> > * No dist-git sources of a package - while trying to workaround previous
> >> problem I have found that dist-git doesn't get into the environment so I
> >> can't do something like dnf builddep
> >>
> >> Please help me ;)
> >>
> >
> >
> > We should decide: if tests can exist independently without dist-git.
> > For now, we presume they can.
> > This is current state.
> > If yes. There is nothing to do.
> >
>
>  Why would they do?
>
>  
If you think that:

"parent dir for /tests must be contents from dist-git branch" , then:
please file a issue at https://pagure.io/standard-test-roles/issues
That I can announce discussion about this changes to STI to our team.
Please remember, this change should be consistent for all dist-git repos.




>  If no. We should tight together dist-git branch + /tests as one
> entity.
> > And copy /tests to test-environment system together with dist-git.
> >
> >  If we decide to change this, than we need to make correspondent changes
> > to STI.
> > https://fedoraproject.org/wiki/CI/Standard_Test_Interface
> >
> >
> >
> >>
> >> P.S> I'm using standard-test-basic role.
> >>
> >> Thanks in advance!
> >> --
> >>
> >> -Igor Gnatenko
> >>
> > _______________________________________________
> >> CI mailing list -- ci(a)lists.fedoraproject.org
> >> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> >> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives:
> >>
> https://lists.fedoraproject.org/archives/list/[e-mail]/...
> >>
> > _______________________________________________
> > CI mailing list -- ci(a)lists.fedoraproject.org
> > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> >
> https://lists.fedoraproject.org/archives/list/[e-mail]/...
> >
>  --
>
>  -Igor Gnatenko
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>
> https://lists.fedoraproject.org/archives/list/[e-mail]/...
>