8:49 a.m.
Hello folks,
in relation with recent SCAP content failures:
[1]
https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-Novembe...
(I will come with a final fix for this one yet)
[2]
https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-Novembe...
(confirmed as valid report)
[3]
https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-Novembe...
(confirmed as valid)
[4]
https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-Novembe...
(pending review)
the more urgent requirement / need how to have a way for automated current RHEL6 (and
Fedora)
SSG SCAP content testing available has arisen.
While these issues (when noticed) are mainly easy to be fixed, if went unnoticed they
could
cause serious harm (false sense of security for failing RHEL6's sshd rule claiming it
passed,
not working configuration for the login.defs / auditd cases).
To summary looks there's is an urgent need to have the content automatically tested
somehow.
We were thinking about this yesterday a bit with Simon. Below are the (mainly in form
of brainstorming) proposals / comments:
* looks it would not be possible to have the test suite directly as part of SSG content
performing the tests during scap-security-guide package builds (for example to check
sshd rules it would require to modify underlying system configuration which would not
be straightforward on the system to be scanned),
* selected subset of the rules (mainly checks and remediations changing configuration)
could be tested via chroot directory. There's openscap offline scan feature:
[5] https://www.redhat.com/archives/open-scap-list/2013-May/msg00007.html
which could be used for this purpose (set up desired test directory structure somewhere
on the filesystem [possibly /tmp?] and instruct oscap via those environment variables
to perform rule evaluation for that directory structure). Have expected evaluation
result for each of such configurations, and compare the obtained results with the
expected ones.
* the previous proposal wouldn't still catch case when for example
'PermitRootLogin'
would be allowed by default in the sshd config. Rules like this require the OVAL
rule evaluation to be performed on one hand, on the other hand some kind of bash
script to be performed on the other (for this concrete case try to login remotely
for root via SSH to see if it's allowed), and compare result of the oval scan with
actual result from the system.
For this case we have been thinking to have set of predefined system configurations
(hopefully via kickstart files), and scripts to perform actual system behaviour checks.
The party responsible for the testing would then use the kickstart file to
automatically
create the guest image of particular system, and would run those scripts on it (there
could be some unified interface to simplify the test runs yet).
We think that this could:
* prevent occurrence of issues like the above [1] up to [4] in the future,
* allow us to notice when something in the test outputs changed very early, so
we can possibly adjust the OVAL checks and / or remediations,
* speedup the certification process - since many of the failures would be
caught yet sooner than the content is provided to some independent party for
certification
* besides all the above it would be possible right in the moment of providing
new OVAL check / remediation to provide testing script for them too.
The proposal would still require us to perform the tests manually. But would at least
provided unified environment, the tests are expected to be performed at, and would
provide
testing scripts (IOW requirements these concrete scripts must pass for the OVAL check /
remediation rule definition to be acceptable).
To be honest I see that this is just brainstorming proposal. Many things would need
to be changed during the way. Hopefully at the end we would end up combining
the above two approaches (running set of tests via oscap chroot's / offline scan
mode, another set via live testing operating system images and scripts modifying
their behaviour in desired way).
This has been written mainly with the intention, if there would be willingness
SSG upstream / members to participate in this effort.
If you have proposals, how this could be improved yet, we would like to hear
about them.
Once we have agreed on the preliminary form of the testing suite, we could
move on how it would be implemented, and where the content (the testing configuration
files, kickstart files, testing scripts) would be hosted.
Another possibility is us to start doing this for current Fedora content, and
apply it back against RHEL-6 content only in moment it has shown to be proper
approach, mature enough.
Comments / feedback welcome.
Thank you && Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team