Package Review Process Policy

Wed Sep 12 14:40:54 UTC 2012

There are 26k++ packages in the fedora repos.
Daily, there are some more new packages to review
Daily, there are bugfixes to apply.
Daily, there are old packages orphaned, or repacked or rebuilt.

While i do agree that a code review would drasitcly increase security of
'our' packages,
it would be a departments day-to-day job to just do the code review.
What makes it even harder is the point, not all packages are written in the
same language,
further, the might be written in diffrent styles, at diffrent
know-how-levels of the developer of that package.

What i want to say is:
Even if you know the language to check/verify, some codes are very large,
and are hard to understand as long you havent written them.

One could say, just check the new packages code.
AFAIK, even today without checking the source code and its functionality,
its hard to get a new package reviewed.
And i guess it would get even harder if the review process calls you to
check all the source code and all of the packages functionality.

And keep in mind, noone's paid for all this work.

Fedora is about newest packages, not the securest/stable ones.

Just my 2 cents.

2012/9/8 <security-request at lists.fedoraproject.org>

> Send security mailing list submissions to
>         security at lists.fedoraproject.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://admin.fedoraproject.org/mailman/listinfo/security
> or, via email, send a message with subject or body 'help' to
>         security-request at lists.fedoraproject.org
>
> You can reach the person managing the list at
>         security-owner at lists.fedoraproject.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of security digest..."
>
>
> Today's Topics:
>
>    1. Package Review Process Policy (troopa)
>    2. Re: Package Review Process Policy (Adam Williamson)
>    3. Re: Re: Package Review Process Policy (troopa)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 8 Sep 2012 01:50:35 -0500
> From: troopa <troopa at gmail.com>
> To: security at lists.fedoraproject.org
> Subject: Package Review Process Policy
> Message-ID:
>         <
> CAMHtQkZ9jX98SG0O4J644rze5iqj7JZ0sMmE1i9aNyY9DVCwXA at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I have concern regarding Fedora's package review process and how its
> current policy enforcement seems to make security and code-correctness take
> a backseat to functionality and adoption of new packages. Specifically, I
> am having issues with the *lack of a mandatory code review* before a
> package is added to the official software repositories.
>
> Recently I was reading review process of an undisclosed forked project, and
> the results really made me think twice about trusting the official Fedora
> repository. It seems two people who are part of this process stated that a
> mandatory code review was not part of the underlying package review
> process.
>
>  Rahul Sundaram 2012-03-13 09:42:31 EDT
>
> @Christoph Wickert,  The quote doesn't mean what you think it does.  *We
> don't do code review as part of the review process clearly and there is no
> real history of even checking for functionality*.  If you want this to
> change, that is a reasonable position but any claim otherwise is
> overreaching. The checklist for instance focuses only on packaging policy.
> The worst that could happen is that the package gets abandoned after a
> while but that isn't a real problem.  It happens routinely anyway.
>
>
>  Christoph Wickert 2012-03-14 08:21:50 EDT
>
> (In reply to comment #35)
> > We don't
> > do code review as part of the review process clearly and there is no real
> > history of even checking for functionality.
>
> *I agree that a code review is not mandatory part of a package review*,
> nevertheless I consider it very useful. I recall a review that revealed
> serious bugs and even a security issue in one of my packages. Me and the
> reviewer worked on patches and I upstreamed them before the package was
> released in Fedora. This is how successful collaboration between developers
> and package maintainers should look like.
>
> Besides that, checking for basic functionality *is* definitely part of the
> review checklist: "The reviewer should test that the package functions as
> described. A package should not segfault instead of running, for example."
>
>
> This is a little alarming to me. Honestly, I expect anything that passes
> Fedora's package review process to be audited and checked to ensure there
> is no underlying malicious intent within software, especially when it is
> aiming at being added as part of Fedora's official repositories, which are
> generally considered a trusted source for installing new software.
>
> I mean, what if I decided to create a fork of XFCE with a few useful
> improvements or changes that were not directly accepted by the main
> branches policies, and in some obscure regions of the software I plant a
> malicious routine. According to the aforementioned quotes; as long as the
> package installed correctly, had at least the advertised functionality and
> didn't break anything then it would be able to pass a review, regardless of
> what surprises I may have hidden inside.
>
> According to Fedora, their underlying goal of this formal process is:
>
>  In order for a new package to be added to Fedora, the package must first
> undertake a formal review. *The purpose of this formal review is to try to
> ensure that the package meets the quality control requirements for Fedora.
> This does not mean that the package (or the software being packaged) is
> perfect, but it should meet baseline minimum requirements for quality.*
>
>
> I believe the *minimum requirements* for quality should most certainly
> include security as a highly important *"minimum requirement"* for their
> quality control.
>
> In this day and age, privacy and security should be the number one priority
> of all software. I don't care if the software is a calculator, desktop
> environment, service daemon or anything else - anything in the official
> Fedora repositories should be able to posses the following characteristics:
> *Trusted*, *safe*, and *stable* (within reason). Right now, the current
> policy enforcement only requires that packages meet the following
> characteristics: *Does not break*, *at least does what it claims*, *and
> seems stable enough for most people.*
>
> Personally, I find this to be an unacceptable standard. Especially coming
> from a project that is directly associated with a reputable project like
> RedHat. Sure, maybe security is more important to me than most everyone
> else, but security should at least be important enough to at least check
> the code to verify it provides advertised functionality and nothing more.
>
> Based on this information, just how "trusted" can Fedora's repositories be?
> I mean, it seems like any random person over the Internet willing to go
> through a review process can have their software added to the official
> repositories, without it being audited for major privacy or security
> violations.
> I can see an argument being made that "this is only the process for
> software which is optional and not directly security-related" and "it is
> also only the process for popular and well-known software".
>
> Well, just because something is optional and not directly security-related
> does not mean it shouldn't be able to be trusted in a secure environment,
> especially if it is being delivered by Fedora's official repositories.
> Also, just because something is popular does not mean someone won't try to
> slip something in it before asking for a "formal review".
>
> Am I honestly the only person that finds the current policy enforcement to
> be severely lacking?
>
> I suppose the only course of action is to create a ticket with FESCo, and
> hope they also feel that this method of formal review is lacking.
>
> I mean, I guess anyone that wanted to verify the integrity of their
> software could audit the code themselves, but that seems counter-productive
> to having a trusted central repository to begin with. Sure, the current
> process requires people to jump through a few hoops, but it does nothing to
> safeguard the privacy and security of its end-users.
>
> This is just something that should be looked at closer, in my humble
> opinion.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.fedoraproject.org/pipermail/security/attachments/20120908/0f4e62bf/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 08 Sep 2012 00:15:15 -0700
> From: Adam Williamson <awilliam at redhat.com>
> To: <security at lists.fedoraproject.org>
> Subject: Re: Package Review Process Policy
> Message-ID: <7433dc3653a92b79c2458334b0a1ce6b at www.happyassassin.net>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On 2012-09-07 23:50, troopa wrote:
>
> >  Personally, I find this to be an unacceptable standard. Especially
> > coming from a project that is directly associated with a reputable
> > project like RedHat. Sure, maybe security is more important to me
> > than
> > most everyone else, but security should at least be important enough
> > to at least check the code to verify it provides advertised
> > functionality and nothing more.
>
> Think about the practical implications of this. Take, say, MATE or
> Cinnamon, both recently added or trying to be added to Fedora repos.
> This would require both the packager and reviewer to be a) capable of
> and b) have enough time to review the _entire_ code base of each project
> and declare that they found no security issues. It's an incredibly
> difficult process.
>
> I'd take a high level summary and say that Fedora's processes and
> policies, broadly, assume an element of good faith. Your proposal
> appears to take the opposite tack: a defensive posture, assuming all new
> code is bad until proven otherwise. This is a very difficult stance for
> a project like Fedora to take convincingly. After all, if someone's
> trying to trojan in something evil, why would you expect them to leave
> it in plain sight? Surely they'd try to obfuscate it as much as
> possible. As the Obfuscated C Code Contest and others show, there's all
> sorts of possibilities down this line. If we're going to take a
> defensive posture to all proposed Fedora packages, we'd need a corps of
> elite coders to review every submitted package with a fine-toothed comb.
>
> In practice, we have enough trouble just finding people committed
> enough to perform the currently required review processes. It seems
> unrealistic to believe Fedora is capable of performing a comprehensive
> security audit on all the zillions of lines of code it contains and
> which are regularly added to it...
>
> Bodies with really serious security needs, like the NSA, have always
> taken 'consumer level' products like Fedora and performed their own
> security evaluations on them. I don't think that's an unreasonable
> approach. If you have really serious security requirements, then you may
> need to shoulder some of the burden of enforcing them yourself.
> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
> http://www.happyassassin.net
>
>
> ------------------------------
>
> Message: 3
> Date: Sat, 8 Sep 2012 05:28:43 -0500
> From: troopa <troopa at gmail.com>
> To: security at lists.fedoraproject.org
> Subject: Re: Re: Package Review Process Policy
> Message-ID:
>         <
> CAMHtQkZLHkT2nccbZM2S6OXbtfVNOZX2+6zkUEn7CaEhwfcDYA at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> >
> > Think about the practical implications of this. Take, say, MATE or
> > Cinnamon, both recently added or trying to be added to Fedora repos.
> > This would require both the packager and reviewer to be a) capable of
> > and b) have enough time to review the _entire_ code base of each project
> > and declare that they found no security issues. It's an incredibly
> > difficult process.
> >
> > While I understand that the practical implications of mandatory code
> review would be taxing and difficult on the team, it would still be far
> better to have trusted packages versus new packages, especially when it
> comes to forks of large projects. If a code review is not possible due to
> an extremely large project, then why not at least verify the identities of
> anyone that wants to submit a package to an official repository? That would
> at least make maintainers accountable if something like that did occur,
> which would make it less likely to happen. Only appropriate members of the
> Fedora team would have access to information regarding the identity of a
> maintainer in question. This would be a giant leap in the right direction
> without requiring such an extremely taxing overhead.
>
> For example, take MATE and Cinnamon as the example. Verify the maintainer
> that wishes to publish it within the official repository is indeed part of
> the development team, verify the team members identity, and then proceed
> with the normal package review. This would at least ensure that the package
> is going to be maintained by the developers, the person in charge of
> getting the package into the repository is identified and thus able to be
> verified, and that the package follows the existing guidelines. My point
> is: Just don't let any random person over the Internet be able to submit a
> package into the official repositories without having some accountability
> for nefarious actions. While it won't stop all possible instances of such
> an attack vector, it would at least be a more trusted solution than what we
> currently have in place.
>
> I'd take a high level summary and say that Fedora's processes and
> > policies, broadly, assume an element of good faith. Your proposal
> > appears to take the opposite tack: a defensive posture, assuming all new
> > code is bad until proven otherwise. This is a very difficult stance for
> > a project like Fedora to take convincingly. After all, if someone's
> > trying to trojan in something evil, why would you expect them to leave
> > it in plain sight? Surely they'd try to obfuscate it as much as
> > possible. As the Obfuscated C Code Contest and others show, there's all
> > sorts of possibilities down this line. If we're going to take a
> > defensive posture to all proposed Fedora packages, we'd need a corps of
> > elite coders to review every submitted package with a fine-toothed comb.
> >
> >
> While I agree if malicious code were embedded within a package it could be
> extremely difficult to detect, and I understand Fedora simply does not have
> the resources to perform security audits on large projects, there still
> needs to be a better way to allow packages into the official repositories.
> The current policy does not seem to take security into account at all, and
> takes everything on good faith. While I understand my suggested approach is
> not practical, but neither is taking everything in good faith.
>
> What would be so wrong with having a dedicated security team? Their job
> could be to ensure packages added into the repositories are more trusted by
> verifying the identity of the maintainer, ensuring the maintainer is part
> of the actual development team of the project in question, and auditing the
> code when possible and practical to ensure more trust in Fedora's
> repositories. While it is important not to disallow new developers to add
> their projects to the repositories, it is just as important to ensure they
> will have some form of accountability in the event that something like this
> did occur. The damage may already be done at that point, but it would
> likely deter most individuals. You could also offload formal reviews onto
> this team as well. If the resources just are not there for such a team,
> then I can understand that. However, if the resources were there for
> something like this to be done, it would be far better than the existing
> solution.
>
> It wouldn't take a "corpse of elite coders" to simply verify the source,
> and thus put more trust into the repositories.
>
> In practice, we have enough trouble just finding people committed
> > enough to perform the currently required review processes. It seems
> > unrealistic to believe Fedora is capable of performing a comprehensive
> > security audit on all the zillions of lines of code it contains and
> > which are regularly added to it...
> >
> > This is a big reason why I suspect you guys just don't have the resources
> to beef up your package review policy, and that is okay. None of my
> suggestions are to be taken as things you should do, but just my personal
> opinion regarding the situation. I understand that this is not a perfect
> world, and sometimes you have to make sacrifices to security to benefit the
> project and community as a whole. I am only asking that the policies be
> reviewed, and strengthened where possible with any practical solutions
> anyone can come up with. That is a big reason why I am still spit-balling
> ideas instead of just expecting someone else to "figure it out".
>
> I still think mandatory code reviews should be in place where possible,
> especially for smaller projects. They may not be mandatory for large
> projects, and that is fine. However, taking all packages on blind faith
> just seems a little silly to me, especially when it is within the reach of
> your existing resources to do so with smaller projects.
>
> There just has to be a better way to place trust in the repositories.
>
> > Bodies with really serious security needs, like the NSA, have always
> > taken 'consumer level' products like Fedora and performed their own
> > security evaluations on them. I don't think that's an unreasonable
> > approach. If you have really serious security requirements, then you may
> > need to shoulder some of the burden of enforcing them yourself.
> >
> > I can agree with this sentiment. If the need for security is greater than
> what is reasonably expected from the vendor, it would then fall to the
> end-user to increase security for themselves where possible. However, I
> believe that it is still okay to give feedback to the vendor, which is what
> I am doing.
>
>
> Anyway, thank you for taking the time to respond to me, Adam. I hope you do
> not think I am tying to be argumentative, because I am not. I just wanted
> to give some feedback, and hopefully get more people thinking about
> practical solutions that could help improve the package review process as a
> whole.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.fedoraproject.org/pipermail/security/attachments/20120908/289783db/attachment.html
> >
>
> ------------------------------
>
> --
> security mailing list
> security at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/security
>
> End of security Digest, Vol 56, Issue 1
> ***************************************
>

-- 
Simon A. Erat (sea)
Switzerland
----------------
FAS: sea
IRC: seasc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/security/attachments/20120912/7de39118/attachment-0001.html>