On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler mpreisle@redhat.com wrote:
----- Original Message -----
From: "Andrew Gilmore" agilmore2@gmail.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Thursday, August 28, 2014 8:29:48 PM Subject: Re: New report and guide in openscap 1.1.0
I like the new look and functionality.
Two first blush comments:
- On the report document, I can imagine my security officials freaking
out
over the in-your-face "*The system is not compliant!*" text. What is the recommended course to ensure this text does not appear if you're running the scan on a webserver, for example? Is it as simple as creating a
custom
profile derived from the STIG profile? Does anyone directly use the STIG profile, have a completely compliant system, and have a server that actually does anything useful? Up to now, I've left tests in that I have waivers for, and then pointed
at
the waivers to justify the test failures. Perhaps I will need to change that practice.
Isn't that a good thing? They should freak out, their system is not compliant! The recommended course is to tailor the profile, leaving out rules that make no sense on your system. Then you fix the remaining rules using remediation. In the end the machine will be compliant.
I would maybe add or modify the message here to be something along the lines:
- "The system is not compliant! Please review rule results, site/network security requirements, and consider applying remediation."
--- or ---
- "The system may not be compliant! Please review rule results, site/network security requirements, and consider applying remediation."
I personally would prefer the last one as it says, "Hey. Check your system as well as check your security requirements to see if what you are seeing from the scan matches with those security requirements."
The job of openscap is to check your machines for compliance over and over.
When the machines are suddenly not compliant you really want to know that!
- On the guide document, the text beginning "Providing system
administrators" occurs twice.
Looks like an issue with SSG but I will look more into it.
-- Martin Preisler -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
Martin,
Thank you for the work on cleaner, modern, interactive report.
The three feature improvements that are most valuable are: 1. Including the priority of a control in the summary listing b/c it saves a lot of time looking things up. I needed this. 2. The interactive filtering b/c it also saves a lot of time b/c it allows me to use the report in planning and to have a conversation. I needed this. 3. Modern look and pop-up details. Also saves me time b/c it keeps my place while I look deeper (less cumbersome). I will use this.
Least needed features: 1. I'm less sure if I currently need the collapsing groups feature. I've been playing with it. On the plus side, the grouping has utility and it's particularly nice to collapse things to less than a handful of summary groups. It is also a big nerd knob whose presence is felt in everything I do in the detail section. The filters provide me with optional controls; the collapsing sections I have to actively manage. Even fully expanded, I am cognitively managing the groups presence as I scan down and look at items. I have to coordinate the filters and the grouping collapse/expansion. It's a powerful feature, and I might like it at times, but I also might find it cumbersome and bothersome in the future, too. I'd almost rather try the report a while with just the filters and other improvements first. (Or maybe make it optional...) 2. I keep wondering if the detail section should even be part of the report, or should be online by default. Does it make sense to include by default static information? I suppose running the report on a system on segregated network would be one reason -- but is that the most common scenario? Why have the detail information in the same report vs a second-related report?
Features I still want: 1. I still want an even higher level executive summary that begins to tie the details of the scan to actual system and organizational risk. I want a report that creates shared awareness between Dev, Ops, Sec, and Mission. 2. Diff'ing: Comparing result with previous report. If I am using the repot to lock down a system, I want to easily see what's changed since the previous scan. 3. Inclusion of comments and waivers. If I am consciously ignoring a test that is in the profile, it would be helpful to see and share that in the report (and see the result therefore as 'passed*' or 'pass waiver". 4. Time estimates and prioritization. How long will it take me to fix different items? Which ones have automated fixes?
Greg Elin http://govready.org - Making FISMA compliance easier for innovators
email: gregelin@gitmachines.com phone: 917-304-3488
On Fri, Aug 29, 2014 at 9:28 AM, Gabe Alford redhatrises@gmail.com wrote:
On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler mpreisle@redhat.com wrote:
----- Original Message -----
From: "Andrew Gilmore" agilmore2@gmail.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Thursday, August 28, 2014 8:29:48 PM Subject: Re: New report and guide in openscap 1.1.0
I like the new look and functionality.
Two first blush comments:
- On the report document, I can imagine my security officials freaking
out
over the in-your-face "*The system is not compliant!*" text. What is the recommended course to ensure this text does not appear if you're running the scan on a webserver, for example? Is it as simple as creating a
custom
profile derived from the STIG profile? Does anyone directly use the STIG profile, have a completely compliant system, and have a server that actually does anything useful? Up to now, I've left tests in that I have waivers for, and then pointed
at
the waivers to justify the test failures. Perhaps I will need to change that practice.
Isn't that a good thing? They should freak out, their system is not compliant! The recommended course is to tailor the profile, leaving out rules that make no sense on your system. Then you fix the remaining rules using remediation. In the end the machine will be compliant.
I would maybe add or modify the message here to be something along the lines:
- "The system is not compliant! Please review rule results, site/network
security requirements, and consider applying remediation."
--- or ---
- "The system may not be compliant! Please review rule results,
site/network security requirements, and consider applying remediation."
I personally would prefer the last one as it says, "Hey. Check your system as well as check your security requirements to see if what you are seeing from the scan matches with those security requirements."
The job of openscap is to check your machines for compliance over and over.
When the machines are suddenly not compliant you really want to know that!
- On the guide document, the text beginning "Providing system
administrators" occurs twice.
Looks like an issue with SSG but I will look more into it.
-- Martin Preisler -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
----- Original Message -----
From: "Greg Elin" gregelin@gitmachines.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Friday, August 29, 2014 11:18:53 PM Subject: Re: New report and guide in openscap 1.1.0
Martin,
Thank you for the work on cleaner, modern, interactive report.
The three feature improvements that are most valuable are:
- Including the priority of a control in the summary listing b/c it
saves a lot of time looking things up. I needed this. 2. The interactive filtering b/c it also saves a lot of time b/c it allows me to use the report in planning and to have a conversation. I needed this. 3. Modern look and pop-up details. Also saves me time b/c it keeps my place while I look deeper (less cumbersome). I will use this.
Thanks.
Least needed features:
- I'm less sure if I currently need the collapsing groups feature. I've
been playing with it. On the plus side, the grouping has utility and it's particularly nice to collapse things to less than a handful of summary groups. It is also a big nerd knob whose presence is felt in everything I do in the detail section. The filters provide me with optional controls; the collapsing sections I have to actively manage. Even fully expanded, I am cognitively managing the groups presence as I scan down and look at items. I have to coordinate the filters and the grouping collapse/expansion. It's a powerful feature, and I might like it at times, but I also might find it cumbersome and bothersome in the future, too. I'd almost rather try the report a while with just the filters and other improvements first. (Or maybe make it optional...)
I am against making things optional. We want one report with a simple quality assurance matrix. We want one set of features and we want to keep them working.
This is the first time I have heard somebody not liking the collapse/expand feature and it was one of the features people asked for so I am keeping it for now.
- I keep wondering if the detail section should even be part of the
report, or should be online by default. Does it make sense to include by default static information? I suppose running the report on a system on segregated network would be one reason -- but is that the most common scenario? Why have the detail information in the same report vs a second-related report?
No, details will not be uploaded anywhere on the web. They are generated locally, you can keep them local. Uploading them anywhere or even querying things from the web leaks information about your infrastructure.
Features I still want:
- I still want an even higher level executive summary that begins to tie
the details of the scan to actual system and organizational risk. I want a report that creates shared awareness between Dev, Ops, Sec, and Mission.
Mock-up please.
- Diff'ing: Comparing result with previous report. If I am using the
repot to lock down a system, I want to easily see what's changed since the previous scan.
People ask for this all the time. We plan to do it but it requires changes in more areas than just the report. Stay tuned.
- Inclusion of comments and waivers. If I am consciously ignoring a test
that is in the profile, it would be helpful to see and share that in the report (and see the result therefore as 'passed*' or 'pass waiver".
Same as above. This is not as simple as it seems.
- Time estimates and prioritization. How long will it take me to fix
different items? Which ones have automated fixes?
If you can tell us where we can get these times, I am all for it. If you really want this the first step is to propose a change to XCCDF and get the metadata in there. Second step is to convince content authors that they want to use the new metadata. In a few years we may have time estimates.
In the end I think this can't possibly be automated. We can pull a number out of a hat and call it the time estimation but that wouldn't be honest.
On 8/29/14, 9:28 AM, Gabe Alford wrote:
On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler <mpreisle@redhat.com mailto:mpreisle@redhat.com> wrote:
----- Original Message ----- > From: "Andrew Gilmore" <agilmore2@gmail.com <mailto:agilmore2@gmail.com>> > To: "SCAP Security Guide" <scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org>> > Sent: Thursday, August 28, 2014 8:29:48 PM > Subject: Re: New report and guide in openscap 1.1.0 > > I like the new look and functionality. > > Two first blush comments: > 1) On the report document, I can imagine my security officials freaking out > over the in-your-face "*The system is not compliant!*" text. What is the > recommended course to ensure this text does not appear if you're running > the scan on a webserver, for example? Is it as simple as creating a custom > profile derived from the STIG profile? Does anyone directly use the STIG > profile, have a completely compliant system, and have a server that > actually does anything useful? > Up to now, I've left tests in that I have waivers for, and then pointed at > the waivers to justify the test failures. Perhaps I will need to change > that practice. Isn't that a good thing? They should freak out, their system is not compliant! The recommended course is to tailor the profile, leaving out rules that make no sense on your system. Then you fix the remaining rules using remediation. In the end the machine will be compliant.I would maybe add or modify the message here to be something along the lines:
- "The system is not compliant! Please review rule results,
site/network security requirements, and consider applying remediation."
--- or ---
- "The system may not be compliant! Please review rule results,
site/network security requirements, and consider applying remediation."
I personally would prefer the last one as it says, "Hey. Check your system as well as check your security requirements to see if what you are seeing from the scan matches with those security requirements."
Systems are scanned against a specific profile (STIG, USGCB....) which represent defined requirements. It's fair to say "The system /is not/ compliant" vs "may not."
Recognizing deployments may have exceptions, the override/tailoring file can be user (e.g. "my site uses 5 char passwords, not 12, so don't fail me").
The job of openscap is to check your machines for compliance over and over. When the machines are suddenly not compliant you really want to know that! > 2) On the guide document, the text beginning "Providing system > administrators" occurs twice. Looks like an issue with SSG but I will look more into it. -- Martin Preisler -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
I'm going to start a new thread regarding language of compliance and baseline threads.
On Sun, Aug 31, 2014 at 2:30 AM, Shawn Wells shawn@redhat.com wrote:
On 8/29/14, 9:28 AM, Gabe Alford wrote:
On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler mpreisle@redhat.com wrote:
----- Original Message -----
From: "Andrew Gilmore" agilmore2@gmail.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Thursday, August 28, 2014 8:29:48 PM Subject: Re: New report and guide in openscap 1.1.0
I like the new look and functionality.
Two first blush comments:
- On the report document, I can imagine my security officials freaking
out
over the in-your-face "*The system is not compliant!*" text. What is
the
recommended course to ensure this text does not appear if you're running the scan on a webserver, for example? Is it as simple as creating a
custom
profile derived from the STIG profile? Does anyone directly use the STIG profile, have a completely compliant system, and have a server that actually does anything useful? Up to now, I've left tests in that I have waivers for, and then pointed
at
the waivers to justify the test failures. Perhaps I will need to change that practice.
Isn't that a good thing? They should freak out, their system is not compliant! The recommended course is to tailor the profile, leaving out rules that make no sense on your system. Then you fix the remaining rules using remediation. In the end the machine will be compliant.
I would maybe add or modify the message here to be something along the lines:
- "The system is not compliant! Please review rule results, site/network
security requirements, and consider applying remediation."
--- or ---
- "The system may not be compliant! Please review rule results,
site/network security requirements, and consider applying remediation."
I personally would prefer the last one as it says, "Hey. Check your system as well as check your security requirements to see if what you are seeing from the scan matches with those security requirements."
Systems are scanned against a specific profile (STIG, USGCB....) which represent defined requirements. It's fair to say "The system *is not* compliant" vs "may not."
Recognizing deployments may have exceptions, the override/tailoring file can be user (e.g. "my site uses 5 char passwords, not 12, so don't fail me").
The job of openscap is to check your machines for compliance over and
over. When the machines are suddenly not compliant you really want to know that!
- On the guide document, the text beginning "Providing system
administrators" occurs twice.
Looks like an issue with SSG but I will look more into it.
-- Martin Preisler -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
-- Shawn Wells Director, Innovation Programsshawn@redhat.com | 443.534.0130 @shawndwells
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
----- Original Message -----
From: "Gabe Alford" redhatrises@gmail.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Friday, August 29, 2014 3:28:20 PM Subject: Re: New report and guide in openscap 1.1.0
On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler mpreisle@redhat.com wrote:
[snip]
I would maybe add or modify the message here to be something along the lines:
- "The system is not compliant! Please review rule results, site/network
security requirements, and consider applying remediation."
--- or ---
- "The system may not be compliant! Please review rule results,
site/network security requirements, and consider applying remediation."
The thing is, you should have reviewed your security requirements before you chose the benchmark and profile and decided to run the scan :-) The only thing openscap knows is that the machine is not compliant with regards to the benchmark and profile combination you evaluated.
We have to be more generic than site/network security requirements. And I think saying that you are not compliant with regards to the selected benchmark and profile is redundant. That should be apparent from the report already.
scap-security-guide@lists.fedorahosted.org
-
Gabe Alford -
Greg Elin -
Martin Preisler -
Shawn Wells