FYI: "scap-security-guide-pull-requests" Jenkins CI job has been configured to test the specific PR on all three slaves (el6, el7, and fedora23)

[Jan Lieskovsky]

Hello folks,

jFYI I have recently configured the Jenkins 'scap-security-guide-pull-requests' CI job
to be always running in parallel on all three of the following Jenkins slaves:
* el6,
* el7, and
* fedora23

Why?

* OVAL-5.11 / OVAL-5.11.1 is not an official SCAP-1.2 standard => we still need to produce
  valid OVAL-5.10 files (IOW "make validate" target still needs to pass with openscap-1.0.x),

* OVAL-5.11.1 might become a SCAP-1.3 part => we need to produce valid OVAL-5.11 files
  (IOW "make validate" target need to pass with openscap-1.2.x),

* The "ShellCheck" test (remediation scripts bash syntax verification) is performed only
  on Fedora systems. In the past it happened "make validate" target stopped to PASS
  due the ShellCheck executable syntax change / expectations.

Therefore the only way how to satisfy all three of the above requirements and not to introduce
regressions in the future SSG content, we need to test PRs on all three of the aforementioned
systems (in the past we experienced PRs that succeeded only e.g. with "OVAL-5.11", but due
to random nature how Jenkins schedules the tests on the slaves, this PRs were improperly reported
as passing, even when they didn't met the condition the "make validate" target to pass
also with OVAL-5.10. This kind of PRs is making e.g. "scap-security-guide-nightly-oval510-zip"
to fail:
  https://jenkins.open-scap.org/job/scap-security-guide-nightly-oval510-zip/)

Therefore to prevent these regressions in the future, from now on every "scap-security-guide-pull-requests"
CI job will be tested on all three of the above systems.

Example PR where this happened already is:
  https://github.com/OpenSCAP/scap-security-guide/pull/1048

The particular testing jobs then being:
  * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/394/ (el6)
  * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/395/ (el7)
  * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/396/ (fedora23)

Right now "All checks have passed" section is not able to report results of all
three tests though (and possibly fail of at least one of those three jobs has
failed). I have filed:
  https://github.com/OpenSCAP/jenkins/issues/11

This to be possible to inspect in the future.

Thank you && Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team