Hello,
I am writing this email to get feedback from the members of the Fedora development community about OpenScanHub for Fedora.
# tl;dr
OpenScanHub does static and dynamic analysis of rpm packages and it may be helpful in the Fedora community. Please take a look at our staging proof of concept[4] and provide feedback. The proof of concept is in its early stages so there may be some bugs here or there! If the feedback is positive we may roll this out in official infrastructure and integrate with Fedora CI and Packit.
# What
OpenScanHub is a service for static and dynamic analysis. It has been in development inside Red Hat[1] for more than 12 years and was open sourced on GitHub[2] earlier this year. You can read a brief explanation of this service on my blog[3]. We would like to deploy this service on the Fedora infrastructure and start scanning packages shipped in the Fedora project through it.
# Why
I am sharing a prototype[4] of this service to get feedback from the community. This prototype is running on the staging instance of the Fedora infrastructure, so you would have to login[5] to the staging instance before submitting any scan. If you have never logged into that account, it may require you to do a password reset.
Once you are logged into the staging instance, you can login through the `krb5login` button[6] on the top right corner of the homepage and submit a scan through this form[7].
There are 3 different types of scans supported by OpenScanHub:
-
MockBuild performs a full scan of the package including downstream patches. Example[8] mockbuild for `openssl-3.1.1-4.fc39`. -
DiffBuild performs a differential scan on the downstream patches. So you can find only the defects that are introduced by the downstream patches. Example[9] diffbuild for `openssl-3.1.1-4.fc39`. This option would not work if the package fails to compile without patches. -
VersionDiffBuild performs a differential scan between 2 different versions of the package, and you can see defects introduced by the “newer” version of the package. Example[10] differential build between `openssl-3.1.1-4.fc39` and `openssl-3.0.9-2.fc38`.
All the submitted scans can be seen on the tasks[11] page.
This prototype is running on very limited resources, so please do not submit scan for any resource consuming package. Not all defects reported by OpenScanHub may be actual bugs, so please avoid fixing reported defects without careful examination. If we receive positive feedback on this prototype, there may be a possibility of integrating this service with the Fedora CI and Packit projects.
This is a very early stage prototype and may behave inconsistently. Please keep the discussion in this thread constructive. Thank you!
[1] https://kdudka.fedorapeople.org/muni23.pdf
[2] https://github.com/openscanhub/openscanhub
[3] https://situ.im/posts/openscanhub
[4] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/
[5] https://accounts.stg.fedoraproject.org
[6] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/auth/krb5login/ https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/auth/krb5login/?next=/
[7] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/scan/new/
[8] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/6/log/openss...
[9] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/9/log/openss...
[10] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/7/log/added.... [11] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/
On Tue, Dec 12, 2023 at 4:30 PM Siteshwar Vashisht svashisht@redhat.com wrote:
Hello,
I am writing this email to get feedback from the members of the Fedora development community about OpenScanHub for Fedora.
# tl;dr
OpenScanHub does static and dynamic analysis of rpm packages and it may be helpful in the Fedora community. Please take a look at our staging proof of concept[4] and provide feedback. The proof of concept is in its early stages so there may be some bugs here or there! If the feedback is positive we may roll this out in official infrastructure and integrate with Fedora CI and Packit.
# What
OpenScanHub is a service for static and dynamic analysis. It has been in development inside Red Hat[1] for more than 12 years and was open sourced on GitHub[2] earlier this year. You can read a brief explanation of this service on my blog[3]. We would like to deploy this service on the Fedora infrastructure and start scanning packages shipped in the Fedora project through it.
# Why
I am sharing a prototype[4] of this service to get feedback from the community. This prototype is running on the staging instance of the Fedora infrastructure, so you would have to login[5] to the staging instance before submitting any scan. If you have never logged into that account, it may require you to do a password reset.
I have received a couple of comments[1][2] from contributors inside and outside Red Hat. There were several scans submitted by community members that can be seen on the tasks[3] page. I may bring this prototype down at some point next week. So if anyone interested in this idea missed this email earlier, please try it before I bring the prototype down. Thank you!
Once you are logged into the staging instance, you can login through the `krb5login` button[6] on the top right corner of the homepage and submit a scan through this form[7].
There are 3 different types of scans supported by OpenScanHub:
MockBuild performs a full scan of the package including downstream patches. Example[8] mockbuild for `openssl-3.1.1-4.fc39`.
DiffBuild performs a differential scan on the downstream patches. So you can find only the defects that are introduced by the downstream patches. Example[9] diffbuild for `openssl-3.1.1-4.fc39`. This option would not work if the package fails to compile without patches.
VersionDiffBuild performs a differential scan between 2 different versions of the package, and you can see defects introduced by the “newer” version of the package. Example[10] differential build between `openssl-3.1.1-4.fc39` and `openssl-3.0.9-2.fc38`.
All the submitted scans can be seen on the tasks[11] page.
This prototype is running on very limited resources, so please do not submit scan for any resource consuming package. Not all defects reported by OpenScanHub may be actual bugs, so please avoid fixing reported defects without careful examination. If we receive positive feedback on this prototype, there may be a possibility of integrating this service with the Fedora CI and Packit projects.
This is a very early stage prototype and may behave inconsistently. Please keep the discussion in this thread constructive. Thank you!
[1] https://kdudka.fedorapeople.org/muni23.pdf
[2] https://github.com/openscanhub/openscanhub
[3] https://situ.im/posts/openscanhub
[4] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/
[5] https://accounts.stg.fedoraproject.org
[6] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/auth/krb5login/ https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/auth/krb5login/?next=/
[7] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/scan/new/
[8] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/6/log/openss...
[9] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/9/log/openss...
[10] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/7/log/added.... [11] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/
[1] https://github.com/openscanhub/openscanhub/issues/211
[2] https://github.com/openscanhub/openscanhub/issues/214
[3] https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/
On Tue, 2023-12-12 at 16:30 +0100, Siteshwar Vashisht wrote:
[...snip...]
Hi Siteshwar, thanks for working on this.
It looks like you're got the basic infrastructure of scanning working, but the prototype seems to be missing some things that IMHO would be essential to package maintainers actually using this, specifically:
- easy-to-read results (so that a maintainer can easily answer the questions "what is the tool telling me?", "is this a real problem?", "what can/should I do about this?")
- result management (e.g. "what issues have we seen in this past with this package?", "is this a flaky test that keeps coming and going?", etc)
[FWIW I'm the upstream author/maintainer of the GCC static analyzer; I'm also on the SARIF technical committee.]
There are 3 different types of scans supported by OpenScanHub:
-
MockBuild performs a full scan of the package including downstream patches. Example[8] mockbuild for `openssl-3.1.1-4.fc39`.
Looking at a result from the GCC static analyzer e.g. this -Wanalyzer- deref-before-check: https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/6/log/openss...
...I see that the scan-results.html view quoted the pertinent source code (which is good), but unfortunately it only shows the summary message from the diagnostic:
Error: GCC_ANALYZER_WARNING (CWE-465): [#def67] openssl-3.1.1/crypto/bn/bn_blind.c: scope_hint: In function 'BN_BLINDING_update' openssl-3.1.1/crypto/bn/bn_blind.c:108:12: warning[-Wanalyzer-deref-before-check]: check of 'b' for NULL after already dereferencing it # 106| !(b->flags & BN_BLINDING_NO_RECREATE)) { # 107| /* re-create blinding parameters */ # 108|-> if (!BN_BLINDING_create_param(b, NULL, NULL, ctx, NULL, NULL)) # 109| goto err; # 110| } else if (!(b->flags & BN_BLINDING_NO_UPDATE)) {
But each diagnostic from the GCC analyzer has a list of events associated with it, which contain essential information for a human to understand the warning and evaluate it.
For the example above, if I look at the raw log: https://staging-openscanhub.apps.ocp.stg.fedoraproject.org/task/6/log/openss...
...I see: /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c: In function 'BN_BLINDING_update': <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:108:12: warning: check of 'b' for NULL after already dereferencing it [-Wanalyzer-deref-before-check] <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:93:5: note: (1) entry to 'BN_BLINDING_update' <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:97:11: note: (2) pointer 'b' is dereferenced here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:97:8: note: (3) following 'false' branch... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:102:10: note: (4) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:105:8: note: (5) following 'true' branch... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:108:14: note: (6) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:108:14: note: (7) calling 'BN_BLINDING_create_param' from 'BN_BLINDING_update' <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:234:14: note: (8) entry to 'BN_BLINDING_create_param' <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:247:8: note: (9) following 'false' branch (when 'b' is non-NULL)... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:255:12: note: (10) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:255:8: note: (11) following 'false' branch... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:257:12: note: (12) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:257:8: note: (13) following 'false' branch... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:260:8: note: (14) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:260:8: note: (15) following 'false' branch (when 'e' is NULL)... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:264:12: note: (16) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:264:8: note: (17) following 'false' branch... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:267:8: note: (18) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:267:8: note: (19) following 'false' branch (when 'bn_mod_exp' is NULL)... <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:269:8: note: (20) ...to here <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:269:8: note: (21) following 'false' branch (when 'm_ctx' is NULL)... <--[gcc] cc1: note: (22) ...to here /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:307:8: note: (23) following 'false' branch (when 'b' is non-NULL)... <--[gcc] cc1: note: (24) ...to here /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:108:14: note: (25) returning to 'BN_BLINDING_update' from 'BN_BLINDING_create_param' <--[gcc] /builddir/build/BUILD/openssl-3.1.1/crypto/bn/bn_blind.c:108:12: note: (26) pointer 'b' is checked for NULL here but it was already dereferenced at (2) <--[gcc]
where events (2) and (26) contain the most pertinent information (but without quoting the pertinent source code).
I think this isn't going to be usable without some way for developers to view the chain of events, and for that view to show the source code at each event, so that it's trivial for the developer to review the above and answer the questions I mentioned above. I took a few minutes poking about on github trying to find the pertinent code to try to decide if the above was a true or false positive, and couldn't find the exact code, so I gave up.
Also, I think it would make sense to use SARIF as a serialization format for capturing results from the analyzers, since it captures the information of interest, and it widely supported; see e.g. https://github.com/oasis-tcs/sarif-spec/wiki/Known-Producers-and-Consumers Doing so might help with result management.
[...snip...]
Hope this is constructive; thanks again for working on this. Dave