Hi Fedorians,
I think the security tracking bug filing process needs to be amended. The current process is quite frustrating for me and other contributors. This is especially bad for Go CVEs, which there are lot of.
Red Hat Product Security creates a single tracking bug for Fedora{, EPEL} _and_ all Red Hat products and CCs a bunch of Fedora maintainers. They then create separate bugs for each package that they deem affected. The affected packages are oftened determined in a manner that appears overzealous and arbitrary.
After the bugs are created, we get spammed with a bunch of notifications about private bugs, RH product errata, and various other things that are completely irrelevant to Fedora. These messages flood my Bugzilla mailbox and obscure actual issues that I need to address. I do not really care whether a Go CVE has been mitigated in Red Hat Advanced Cluster Management for Kubernetes 2.4 for RHEL 8" or "Red Hat Advanced Cluster Management for Kubernetes 2.5 for RHEL 8" or "Red Hat Advanced Cluster Management for Kubernetes 2.6 for RHEL 8."
---
Some particularly egregious examples:
I maintain an Ansible kubernetes collection, and they reported it as vulnerable to some CVE with a specific Openshift component. The collection not vulnerable. They provided no actionable information, and the description was unclear. When I asked why it was reported, they said that the package "used OpenShift."
A couple Go CVEs ago[^1], they created bugs against hundreds of Go libraries. They arbitrarily chose branches and packages. The bugs were not actionable by packagers of individual go libraries. Only applications that provide binaries need to be rebuilt. They were reported shortly before the F34 EOL, so we got a huge amount of emails after the bugs were automatically closed. In fact, a Go packager reported that these messages from the _security_ team DOSed their mail server. To their credit, they have fixed this issue after one of the other Go SIG people talked to them. Now, these bugs are only filed against the golang component.
[^1]: Really, it was a couple Go releases ago. There are multiple CVEs reported with each Go release these days.
Another time, their automation posted the exact same comment over 200 times.
---
First and foremost, there needs to be a clear way for packagers to report problems with this process to prodsec.
I don't think Fedora packagers should be CCed on these global trackers. We could create a separate "Security Response" component under the "Fedora" Bugzilla product to create tracker bugs for CVEs that affect multiple Fedora components, or we could ask prodsec to only CC Fedora maintainers on the child, package-level bugs. I guess I could acomplish what I'm proposing by filtering out mails with "X-Bugzilla-Product: Security Response" headers and not have gone on this rant, but I still think this needs to be addressed.
Does anyone know how to reach prodsec about this? -- Best,
Maxwell G (@gotmax23) Pronouns: He/Him/His
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel devel@lists.fedoraproject.org wrote:
Does anyone know how to reach prodsec about this?
I'll reach out to the people I know and see what the best way to get them in this conversation is.
There's been some discussion in the security meeting about CVEs, and I've been meaning to get some time to chat with Ben about his thoughts on the best way to move forward. But I keep forgetting everytime I talk to him. I guess now is a good time as ever for him to read this and call me out at the next PGM Matrix/IRC meeting.
On Wed, Sep 7, 2022 at 2:45 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel devel@lists.fedoraproject.org wrote:
Does anyone know how to reach prodsec about this?
I'll reach out to the people I know and see what the best way to get them in this conversation is.
-- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Wed, Sep 7, 2022 at 8:45 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel devel@lists.fedoraproject.org wrote:
Does anyone know how to reach prodsec about this?
I'll reach out to the people I know and see what the best way to get them in this conversation is.
Yes, please.
I appreciate the fact that there's people who monitor security issues and file bugs for them, but the reporting tools they use are very broken. The last example I have is for a CVE (from 2020) in versions 0.1.x the "time" Rust crate, where bugs were filed a month ago, for the following packages:
- the correct bug for rust-time0.1: RHBZ#2119559 - bug for rust-timebomb (completely unrelated package): RHBZ#2119560 - bug for rust-time-macros0.1 (wrong package): RHBZ#2119561 - bug for rust-time-macros-impl (wrong package): RHBZ#2119562
Things like that result in lots of, basically spam, emails, because 3/4 opened bugs were filed for unrelated / wrong packages. It looks like the tooling they use does "prefix match" for component names, which is in many cases just *wrong*. This might also be the reason why dozens of bugs were opened for some golang CVEs.
Another time, their automation posted the exact same comment over 200 times.
Yup, I remember that, I was at the receiving end of this spam barrage, as well (for whatever reason I am getting CCd for all golang CVE bugs even though I am not maintainer of golang *or* member of the go-sig). As far as I remember, the tooling was broken because bugzilla queries for that specific bug timed out because it had so many comments / metadata / CC'd persons etc., and so it continued submitting the same comment over and over (making things worse and worse, of course).
Fabio
On Wed, Sep 7, 2022 at 7:45 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel devel@lists.fedoraproject.org wrote:
Does anyone know how to reach prodsec about this?
I'll reach out to the people I know and see what the best way to get them in this conversation is.
Has this conversation been started yet? Because the CVE reporting system doesn't seem to have been improved at all - in fact a recent CVE bug ( https://bugzilla.redhat.com/show_bug.cgi?id=2141029) was filed, had over 179 people added to the CC list, and there is no mention at all of which applications were identified as being affected or any other tracking bugs filed for those affected applications. So as a maintainer, I am then unsure why I was CC'd on the bug and which application prod sec wants me to examine for the vulnerability (especially since to my knowledge, none of the packages I maintain even use electron in any way or have its code contained inside of them).
-Ian
-- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Dne 09. 11. 22 v 3:10 Ian McInerney via devel napsal(a):
On Wed, Sep 7, 2022 at 7:45 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel <devel@lists.fedoraproject.org> wrote: > > Does anyone know how to reach prodsec about this? I'll reach out to the people I know and see what the best way to get them in this conversation is.Has this conversation been started yet? Because the CVE reporting system doesn't seem to have been improved at all - in fact a recent CVE bug (https://bugzilla.redhat.com/show_bug.cgi?id=2141029) was filed, had over 179 people added to the CC list, and there is no mention at all of which applications were identified as being affected or any other tracking bugs filed for those affected applications. So as a maintainer, I am then unsure why I was CC'd on the bug and which application prod sec wants me to examine for the vulnerability (especially since to my knowledge, none of the packages I maintain even use electron in any way or have its code contained inside of them).
Just FTR, when I was last time looking for answers why I was added on some tracker, and it was probably due to package.json included in source tarball, I was pointed to this project, which should be behind creating these trackers:
https://github.com/RedHatProductSecurity/component-registry
But hard to tell how it is used in practice :/
Vít
When I’ve been mass-CC’d on irrelevant CVEs, I have been able to determine that it was due to a package-lock.json file, which names and pins the versions of all recursive dependencies, that was included with some example NodeJS project in the source tarball. I’ve had trouble with this on a handful of packages.
I don’t recall whether it matters if the file is installed in a -doc subpackage with the other documentation and example files or only present in the sources, but I do remember that removing the package-lock.json file in %prep kept me from getting further irrelevant reports.
Obviously, it would be better if the “targeting” of these automated reports were better so that these workarounds weren’t required. When bugs are to be filed for dozens of packages, the standard of care in verifying their applicability should perhaps be a little higher.
On Wed, Nov 9, 2022, at 9:28 AM, Vít Ondruch wrote:
Dne 09. 11. 22 v 3:10 Ian McInerney via devel napsal(a):
On Wed, Sep 7, 2022 at 7:45 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Sep 7, 2022 at 2:05 PM Maxwell G via devel devel@lists.fedoraproject.org wrote:
Does anyone know how to reach prodsec about this?
I'll reach out to the people I know and see what the best way to get them in this conversation is.
Has this conversation been started yet? Because the CVE reporting system doesn't seem to have been improved at all - in fact a recent CVE bug (https://bugzilla.redhat.com/show_bug.cgi?id=2141029) was filed, had over 179 people added to the CC list, and there is no mention at all of which applications were identified as being affected or any other tracking bugs filed for those affected applications. So as a maintainer, I am then unsure why I was CC'd on the bug and which application prod sec wants me to examine for the vulnerability (especially since to my knowledge, none of the packages I maintain even use electron in any way or have its code contained inside of them).
Just FTR, when I was last time looking for answers why I was added on some tracker, and it was probably due to package.json included in source tarball, I was pointed to this project, which should be behind creating these trackers:
https://github.com/RedHatProductSecurity/component-registry
But hard to tell how it is used in practice :/
Vít
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Attachments:
- OpenPGP_signature
Maxwell G via devel wrote:
I don't think Fedora packagers should be CCed on these global trackers.
The problem is that, as it stands, those global trackers are the only place that actually explains (usually in one paragraph) what the security issue actually is. The [fedora-all] trackers are pretty useless considering that they contain no information whatsoever beyond the subject line. (Their only relevant content is the state, mainly whether they are open or closed.)
If we want independent Fedora-specific tracker bugs, they will need a copy of the introductory paragraph(s).
Kevin Kofler
V Thu, Sep 08, 2022 at 01:06:17AM +0200, Kevin Kofler via devel napsal(a):
Maxwell G via devel wrote:
I don't think Fedora packagers should be CCed on these global trackers.
The problem is that, as it stands, those global trackers are the only place that actually explains (usually in one paragraph) what the security issue actually is. The [fedora-all] trackers are pretty useless considering that they contain no information whatsoever beyond the subject line. (Their only relevant content is the state, mainly whether they are open or closed.)
[fedora-all] bugs links to the vulnerability tracker with Bugzilla dependencies. For me it's pretty obvious where to find the details. If it's not for obvious for others, then an additional sentence in the [fedora-all] description text ("More details about this vulnerability are in bug #NNN") could help.
-- Petr
On Thu, Sep 8, 2022 at 6:17 AM Petr Pisar ppisar@redhat.com wrote:
V Thu, Sep 08, 2022 at 01:06:17AM +0200, Kevin Kofler via devel napsal(a):
Maxwell G via devel wrote:
I don't think Fedora packagers should be CCed on these global trackers.
The problem is that, as it stands, those global trackers are the only place that actually explains (usually in one paragraph) what the security issue actually is. The [fedora-all] trackers are pretty useless considering that they contain no information whatsoever beyond the subject line. (Their only relevant content is the state, mainly whether they are open or closed.)
[fedora-all] bugs links to the vulnerability tracker with Bugzilla dependencies. For me it's pretty obvious where to find the details. If it's not for obvious for others, then an additional sentence in the [fedora-all] description text ("More details about this vulnerability are in bug #NNN") could help.
Fedora maintainers are CC'd often on the parent bug to bypass the private bug status while a bug is "under development". This has happened a few times for me as a maintainer of crypto-adjacent packages.
But yeah, some of it is definitely not right and last year I got spammed with so much that Gmail started rate limiting me. I had to turn several lists into digest mode to go back under.
On Thursday, September 8, 2022 Neal Gompa wrote:
Fedora maintainers are CC'd often on the parent bug to bypass the private bug status while a bug is "under development". This has happened a few times for me as a maintainer of crypto-adjacent packages.
That's a good point. I guess they could fix that by copying the description into the Fedora bug like Kevin said and/or making those "under development" security bugs private to all Fedora contributors instead of CCing specific people.
Dne 08. 09. 22 v 19:32 Maxwell G via devel napsal(a):
On Thursday, September 8, 2022 Neal Gompa wrote:
Fedora maintainers are CC'd often on the parent bug to bypass the private bug status while a bug is "under development". This has happened a few times for me as a maintainer of crypto-adjacent packages.
That's a good point. I guess they could fix that by copying the description into the Fedora bug like Kevin said and/or making those "under development" security bugs private to all Fedora contributors instead of CCing specific people.
However, I think that the idea is that whatever should be said about the CVE should be said in the main tracer. The fedora tracker should be used just to not forget to fix this in Fedora.
Nevertheless, this might soon become non issue given:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Vít
On Fri, Sep 9, 2022 at 10:47 AM Vít Ondruch vondruch@redhat.com wrote:
Nevertheless, this might soon become non issue given:
I think that that may depend on one's definition of "soon", but I do agree that it would be useful to understand how CVE tracking bug workflow is being considered to be handled in the future EL processes while supporting existing use of redhat.bugzilla.com for the remaining lifetimes of the existing EL's.
On Friday, September 9, 2022 Vít Ondruch wrote:
However, I think that the idea is that whatever should be said about the CVE should be said in the main tracer. The fedora tracker should be used just to not forget to fix this in Fedora.
Why not both? We shouldn't have to reference two different bugs to figure out what's going on.
Dne 09. 09. 22 v 17:09 Maxwell G via devel napsal(a):
On Friday, September 9, 2022 Vít Ondruch wrote:
However, I think that the idea is that whatever should be said about the CVE should be said in the main tracer. The fedora tracker should be used just to not forget to fix this in Fedora.
Why not both? We shouldn't have to reference two different bugs to figure out what's going on.
First of all, what is the information you would like to put to either of the trackers?
One information which comes to my mind is that one might have doubts about applicability of the CVE to Fedora, right? But if the CVE is not applicable to Fedora, then it is possibly not applicable to RHEL. Or if this is not convincing, the you can s/Fedora/Linux/ and applicability to Linux vs e.g. Windows. So should such information be put into Fedora tracker or into the main tracker?
So what other information, which is really specific to Fedora should be put into Fedora tracker?
Vít
On Mon Sep 12, 2022, Vít Ondruch wrote:
Dne 09. 09. 22 v 17:09 Maxwell G via devel napsal(a):
On Friday, September 9, 2022 Vít Ondruch wrote:
However, I think that the idea is that whatever should be said about the CVE should be said in the main tracer. The fedora tracker should be used just to not forget to fix this in Fedora.
Why not both? We shouldn't have to reference two different bugs to figure out what's going on.
First of all, what is the information you would like to put to either of the trackers?
Currently, the Fedora trackers contain
This is an automatically created tracking bug! It was created to ensure that one or more security vulnerabilities are fixed in affected versions of fedora-all.
For comments that are specific to the vulnerability please use bugs filed against the "Security Response" product referenced in the "Blocks" field.
For more information see: http://fedoraproject.org/wiki/Security/TrackingBugs
When submitting as an update, use the fedpkg template provided in the next comment(s). This will include the bug IDs of this tracking bug as well as the relevant top-level CVE bugs.
Please also mention the CVE IDs being fixed in the RPM changelog and the fedpkg commit message.
NOTE: this issue affects multiple supported versions of Fedora. While only one tracking bug has been filed, please correct all affected versions at the same time. If you need to fix the versions independent of each other, you may clone this bug as appropriate.
while the main bug has the actual description. I'm saying that the description should be copied to the Fedora tracker.
-- Maxwell G (@gotmax23) Pronouns: He/Him/His
I have started to ignore CVE bugs reports due to the low quality reporting. An outdated ffmpeg CVE was filed against nv-codec-headers, WTF!! It isn't the first time it's been totally bogus.
Hi all,
On Wed, Sep 07, 2022 at 06:04:14PM +0000, Maxwell G via devel wrote:
Hi Fedorians,
I think the security tracking bug filing process needs to be amended. The current process is quite frustrating for me and other contributors. This is especially bad for Go CVEs, which there are lot of.
Red Hat Product Security creates a single tracking bug for Fedora{, EPEL} _and_ all Red Hat products and CCs a bunch of Fedora maintainers. They then create separate bugs for each package that they deem affected. The affected packages are oftened determined in a manner that appears overzealous and arbitrary.
After the bugs are created, we get spammed with a bunch of notifications about private bugs, RH product errata, and various other things that are completely irrelevant to Fedora. These messages flood my Bugzilla mailbox and obscure actual issues that I need to address. I do not really care whether a Go CVE has been mitigated in Red Hat Advanced Cluster Management for Kubernetes 2.4 for RHEL 8" or "Red Hat Advanced Cluster Management for Kubernetes 2.5 for RHEL 8" or "Red Hat Advanced Cluster Management for Kubernetes 2.6 for RHEL 8."
An unrelated issue, but also not ideal:
some engineers at my company worked on fixing some Eternal Terminal (package: et) security issues. Those are fixed, we pushed out updated packages, then went through the CVE process...
Then CVEs get filed against both Fedora and EPEL, warning against versions < 6.2.0 ... while 6.2.1 has been in stable updates for months.
https://bugzilla.redhat.com/buglist.cgi?bug_status=__closed__&classifica...
Feedback to RH prodsec people -- if the process right now assumes every package built before the CVE is public is affected, this might not work well for fixes released while under embargo.
Thanks,
-
Ben Beasley -
Ben Cotton -
Fabio Valentini -
Gary Buhrmaster -
Ian McInerney -
JT -
Kevin Kofler -
Leigh Scott -
Maxwell G -
Michel Alexandre Salim -
Neal Gompa -
Petr Pisar -
Vít Ondruch