On Fri, Jul 20, 2018 at 1:54 PM Igor Gnatenko <ignatenkobrain@fedoraproject.org> wrote:
On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov <astepano@redhat.com> wrote:
On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko <ignatenkobrain@fedoraproject.org> wrote:
I'm trying to have shared tests between all 400+ rust packages.

I have few problems though:

* There is no package NEVRA in test environment so the shared test doesn't know what to test - it is necessary to know which test to run.

We trigger tests for dist-git.
I think about dist-git as a SRPM.
SRPM can produce many RPMs.
CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs.
The same approach is for manual testing.
Conclusion: test-environment-system has installed RPMS.

Just want to mention that this won't work if your subpackages conflict (e.g. curl and curl-minimal. 

Right, and I would ask contra@ team or other guys for clarification, how CI pipeline does care about above moment?
 

Tests from /tests dir check test-environment-system as a whole.

Reason: STI tests are for all test-environment-system (all installed packages).
/tests directory from dist-git knows that packages goes from dist-git/pkg
/tests/tests.yml is written for each SRPM package.
/tests/tests.yml knows that it will run against: SRPM + distro version.
/tests/tests.yml for Fedora 27 and for Fedora 28 can are different files.
If you have common tests, you have different /tests/tests.yml where you can specify exactly that this tests are for this SRPM and for Branch 28 (for example).

 This is wwhat I have done so far, however test runner has all this information, why not to simply pass it to the tests rather than asking every maintainer to do that manually.
 
* No dist-git sources of a package - while trying to workaround previous problem I have found that dist-git doesn't get into the environment so I can't do something like dnf builddep

Please help me ;)


We should decide: if tests can exist independently without dist-git.
For now, we presume they can.
This is current state.
If yes. There is nothing to do.

Why would they do?
 

If you think that:

"parent dir for /tests must be contents from dist-git branch" , then:
please file a issue at https://pagure.io/standard-test-roles/issues
That I can announce discussion about this changes to STI to our team.
Please remember, this change should be consistent for all dist-git repos.
 

 
If no. We should tight together dist-git branch + /tests as one entity.
And copy /tests to test-environment system together with dist-git.

 If we decide to change this, than we need to make correspondent changes to STI. 
 
 

P.S> I'm using standard-test-basic role.

Thanks in advance!
--

-Igor Gnatenko

_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/message/45YUVGGKW3BSFRAFPRGAZ2PKPTFHXLTP/
--

-Igor Gnatenko

_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/message/QCO7WV2N5LHUBPN62PZMEGBJL2SKUI33/