results validation

[Peter Vrabec]

Hi all,

I have found that If I validate OVAL results by schematron rules:

$ oscap oval validate-xml --results --schematron 
rhel6-oval-scap-security-guide.xml.result.xml

I get two types of errors:

oval:scap-security-guide:tst:960 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:959 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:811 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:787 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:786 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:359 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:267 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:221 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:tst:1085 - No state should be referenced when 
check_existence has a value of 'none_exist'.
oval:scap-security-guide:var:2663 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2655 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2651 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2649 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2648 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2644 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2636 - inconsistent datatype between the 
variable and an associated var_ref
oval:scap-security-guide:var:2629 - inconsistent datatype between the 
variable and an associated var_ref

Peter.

[Peter Vrabec]

Hi,

I'd like to clarify how I created 
"rhel6-oval-scap-security-guide.xml.result.xml"

# oscap xccdf eval --profile common --results ssg-xccdf-results.xml 
--oval-results ssg/RHEL6/output/rhel6-xccdf-scap-security-guide.xml

"--oval-results" option do all the magic. man page:
...
Generate OVAL Result file for each OVAL session used for evaluation. 
File with name 'original-oval-definitions-filename.result.xml' will be 
generated for each referenced OVAL file.
...

Peter.

On 08/21/2012 03:54 PM, Peter Vrabec wrote:
> Hi all,
>
> I have found that If I validate OVAL results by schematron rules:
>
> $ oscap oval validate-xml --results --schematron
> rhel6-oval-scap-security-guide.xml.result.xml
>
> I get two types of errors:
>
> oval:scap-security-guide:tst:960 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:959 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:811 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:787 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:786 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:359 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:267 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:221 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:1085 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:var:2663 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2655 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2651 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2649 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2648 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2644 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2636 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2629 - inconsistent datatype between the
> variable and an associated var_ref
>
> Peter.
> _______________________________________________
> scap-security-guide mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide

[Jeffrey Blank]

Hi Peter,

Thanks for the explanation.  I had finally figured the right options,
but only yesterday!  Using (from RPM) openscap-0.8.0-2.el6.x86_64, I got
different results.

$ oscap xccdf eval --profile common --results ssg-xccdf-results.xml
--oval-results rhel6-xccdf-scap-security-guide.xml

$ oscap oval validate-xml --results --schematron
rhel6-oval-scap-security-guide.xml.result.xml

xmlXPathCompOpEval: function exists not found
XPath error : Unregistered function
xmlXPathCompiledEval: 1 objects left on the stack.

Should I re-try with the source build to replicate the issue below?

Jeff

On 08/28/2012 11:59 AM, Peter Vrabec wrote:
> Hi,
>
> I'd like to clarify how I created
> "rhel6-oval-scap-security-guide.xml.result.xml"
>
> # oscap xccdf eval --profile common --results ssg-xccdf-results.xml
> --oval-results ssg/RHEL6/output/rhel6-xccdf-scap-security-guide.xml
>
> "--oval-results" option do all the magic. man page:
> ...
> Generate OVAL Result file for each OVAL session used for evaluation.
> File with name 'original-oval-definitions-filename.result.xml' will be
> generated for each referenced OVAL file.
> ...
>
> Peter.
>
> On 08/21/2012 03:54 PM, Peter Vrabec wrote:
> > Hi all,
> >
> > I have found that If I validate OVAL results by schematron rules:
> >
> > $ oscap oval validate-xml --results --schematron
> > rhel6-oval-scap-security-guide.xml.result.xml
> >
> > I get two types of errors:
> >
> > oval:scap-security-guide:tst:960 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:959 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:811 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:787 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:786 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:359 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:267 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:221 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:tst:1085 - No state should be referenced when
> > check_existence has a value of 'none_exist'.
> > oval:scap-security-guide:var:2663 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2655 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2651 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2649 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2648 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2644 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2636 - inconsistent datatype between the
> > variable and an associated var_ref
> > oval:scap-security-guide:var:2629 - inconsistent datatype between the
> > variable and an associated var_ref
> >
> > Peter.
> > _______________________________________________
> > scap-security-guide mailing list
> > [e-mail]
> > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
>
> scap-security-guide mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide

[Jeffrey Blank]

(Finally I have had some time to look into this!)

Thanks Peter, you're right, this indicates a problem with the OVAL checks.

They were using the check_existence attribute which is an
ExistenceEnumeration type  (which shouldn't be used if states are being
used):
http://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation/ov...

Instead, tests which reference states should be using only the check=
attribute which is a CheckEnumeration type:
http://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation/ov...

Given that the OVAL check I've corrected so far
(accounts_nologin_for_system.xml) didn't even behave correctly, I can
only guess that whatever tool was previously used for testing somehow
produced the correct results by chance...

More fixes to come for this.

However, I do not understand the problem with the "inconsistent datatype
between the variable and associated var_ref".  There is a number
datatype used in the XCCDF values, and an int datatype for the
external_variable type in OVAL.  Anyone else have any ideas?

Jeff

On 08/21/2012 09:54 AM, Peter Vrabec wrote:
> Hi all,
>
> I have found that If I validate OVAL results by schematron rules:
>
> $ oscap oval validate-xml --results --schematron
> rhel6-oval-scap-security-guide.xml.result.xml
>
> I get two types of errors:
>
> oval:scap-security-guide:tst:960 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:959 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:811 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:787 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:786 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:359 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:267 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:221 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:tst:1085 - No state should be referenced when
> check_existence has a value of 'none_exist'.
> oval:scap-security-guide:var:2663 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2655 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2651 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2649 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2648 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2644 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2636 - inconsistent datatype between the
> variable and an associated var_ref
> oval:scap-security-guide:var:2629 - inconsistent datatype between the
> variable and an associated var_ref
>
> Peter.
> _______________________________________________
> scap-security-guide mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide