Signing built RPMs or how to create signed RPMs.

[Allen Hewes]

Hi,

I have my koji setup building RPMs from my subversion repo. I have mash sucking the RPMs out of koji and making yum repos. Right now, the RPMs in my yum repo aren't signed.

I have read some old posts from Jesse w.r.t adding signatures into/with koji. I found the commands for koji cli, but I am not clear as to what to do or how to do it.

My end goal is have the RPMs signed with my key when a client machine runs yum install/update from my repo. What I use to do before I started with koji was rpm --resign *.rpm (enter pass phrase) and then do createrepo -d.

I know I can build & sign stuff with rpmbuild --sign but where(how) would someone change this inside mock running inside kojid?

So my question is where does this happen with koji? Or does it happen outside of koji?

Thanks,

/allen

[Oliver Falk]

Hi Allen!

You might want to look at the following post:

http://www.mail-archive.com/[e-mail]/msg02187.html

-of

Am 13.12.2010 20:59, schrieb Allen Hewes:
> Hi,
>
> I have my koji setup building RPMs from my subversion repo. I have mash sucking the RPMs out of koji and making yum repos. Right now, the RPMs in my yum repo aren't signed.
>
> I have read some old posts from Jesse w.r.t adding signatures into/with koji. I found the commands for koji cli, but I am not clear as to what to do or how to do it.
>
> My end goal is have the RPMs signed with my key when a client machine runs yum install/update from my repo. What I use to do before I started with koji was rpm --resign *.rpm (enter pass phrase) and then do createrepo -d.
>
> I know I can build&  sign stuff with rpmbuild --sign but where(how) would someone change this inside mock running inside kojid?
>
> So my question is where does this happen with koji? Or does it happen outside of koji?
>
> Thanks,
> /allen
>
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Allen Hewes]

> Hi Allen!
>
> You might want to look at the following post:
>
> http://www.mail-archive.com/[e-mail]/ms

g02187.html
> -of

Hi Oliver,

Thanks for link. I had not come across this thread.

It would appear that currently there isn't any method to sign RPMs within koji or mash. You can import prebuilt RPMs with signatures into Koji. I don't know much about importing RPMs into koji because I haven't had a need.

Do the Fedora guys use the sign_unsigned.py script for the official Fedora yum repos? If so, how do they use mash? Because it looks to me that if you use this script, it does one of the steps mash does; fetching RPMs out of koji tags.

I would have guessed that the Fedora guys generate their yum repos via mash from koji tags and then sign RPMs.

I'd have to modify this script to suit my needs, but I think I could do it. It also looks like it relies on a newer version of RPM, the rpm command for key size == 4096 is one spot I noticed.

Also, I have to enter a passphrase when I sign my RPMs but this script doesn't have any provisions for that. Is there a way to make rpm --resign not prompt for a passphrase?

Has there been any talk about adding RPM signing to mash? It seems like that'd be a good place for it.

Thanks,

/allen

[Pierre Guillet]

Hi,

I'am using Koji + sign_unsigned.py + mash to build RPM on CentOS5

I have modified sign_unsigned.py to manage the passphrase. If option is not
used, sign_unsigned.py gives an empty passphrase to 'rpm --resign' command.

Add the Python expect module in import section (pexpect RPM must installed)
:

import getpass
+import pexpect

Add the --passwd option in __init__() from SignUnsigned class:

+        self.parser.add_option("--passwd", action="store_true")

Replace these lines in do_signing()

-                # loop in case password is mistyped
-                while os.system(cmd):
-                    # sleep briefly (give user a chance to ctrl-C)
-                    time.sleep(2)
+                # Use expect to give the passphrase
+                # LANG=C to have english question 'pass phrase'
+                os.environ['LC_ALL'] = 'C'
+                child = pexpect.spawn(cmd)
+                # Wait for 'pass phrase'
+                child.expect('phrase:')
+                if not self.options.passwd:
+                    child.sendline('\r')
+                else:
+                    child.sendline("%s" % self.options.passwd)
+                child.expect(pexpect.EOF)

Regards,
Pierre

2010/12/14 Allen Hewes
> > Hi Allen!
> >
> > You might want to look at the following post:
> >
> > http://www.mail-archive.com/[e-mail]/ms
>
> g02187.htmlhttp://www.mail-archive.com/[e-mail]/ms%0Ag02187.html
> > -of
>
> Hi Oliver,
>
> Thanks for link. I had not come across this thread.
>
> It would appear that currently there isn't any method to sign RPMs within
> koji or mash. You can import prebuilt RPMs with signatures into Koji. I
> don't know much about importing RPMs into koji because I haven't had a need.
>
> Do the Fedora guys use the sign_unsigned.py script for the official Fedora
> yum repos? If so, how do they use mash? Because it looks to me that if you
> use this script, it does one of the steps mash does; fetching RPMs out of
> koji tags.
>
> I would have guessed that the Fedora guys generate their yum repos via mash
> from koji tags and then sign RPMs.
>
> I'd have to modify this script to suit my needs, but I think I could do it.
> It also looks like it relies on a newer version of RPM, the rpm command for
> key size == 4096 is one spot I noticed.
>
> Also, I have to enter a passphrase when I sign my RPMs but this script
> doesn't have any provisions for that. Is there a way to make rpm --resign
> not prompt for a passphrase?
>
> Has there been any talk about adding RPM signing to mash? It seems like
> that'd be a good place for it.
>
> Thanks,
> /allen
>
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Oliver Falk]

Hi!

Thanks for answering this... I used to have a script doing something 
similar :-)

I guess this will help Allen.

-of

Am 14.12.2010 08:43, schrieb Pierre Guillet:
> Hi,
>
> I'am using Koji + sign_unsigned.py + mash to build RPM on CentOS5
>
> I have modified sign_unsigned.py to manage the passphrase. If option is
> not used, sign_unsigned.py gives an empty passphrase to 'rpm --resign'
> command.
>
> Add the Python expect module in import section (pexpect RPM must
> installed) :
>
> import getpass
> +import pexpect
>
> Add the --passwd option in __init__() from SignUnsigned class:
>    self.parser.add_option("--passwd", action="store_true")
>
> Replace these lines in do_signing()
>            # loop in case password is mistyped
>            while os.system(cmd):
>                # sleep briefly (give user a chance to ctrl-C)
>                time.sleep(2)
>            # Use expect to give the passphrase
>            # LANG=C to have english question 'pass phrase'
>            os.environ['LC_ALL'] = 'C'
>            child = pexpect.spawn(cmd)
>            # Wait for 'pass phrase'
>            child.expect('phrase:')
>            if not self.options.passwd:
>                child.sendline('\r')
>            else:
>                child.sendline("%s" % self.options.passwd)
>            child.expect(pexpect.EOF)
>
> Regards,
> Pierre
>
> 2010/12/14 Allen Hewes <[e-mail] >
>  >
>  > Hi Allen!
>  >
>  > You might want to look at the following post:
>  >
>  > http://www.mail-archive.com/[e-mail]/ms
> g02187.html
> <http://www.mail-archive.com/[e-mail]/ms%0Ag02187.html>
>  >
>  > -of
>
> Hi Oliver,
>
> Thanks for link. I had not come across this thread.
>
> It would appear that currently there isn't any method to sign RPMs
> within koji or mash. You can import prebuilt RPMs with signatures
> into Koji. I don't know much about importing RPMs into koji because
> I haven't had a need.
>
> Do the Fedora guys use the sign_unsigned.py script for the official
> Fedora yum repos? If so, how do they use mash? Because it looks to
> me that if you use this script, it does one of the steps mash does;
> fetching RPMs out of koji tags.
>
> I would have guessed that the Fedora guys generate their yum repos
> via mash from koji tags and then sign RPMs.
>
> I'd have to modify this script to suit my needs, but I think I could
> do it. It also looks like it relies on a newer version of RPM, the
> rpm command for key size == 4096 is one spot I noticed.
>
> Also, I have to enter a passphrase when I sign my RPMs but this
> script doesn't have any provisions for that. Is there a way to make
> rpm --resign not prompt for a passphrase?
>
> Has there been any talk about adding RPM signing to mash? It seems
> like that'd be a good place for it.
>
> Thanks,
>
> /allen
> --
> buildsys mailing list
> [e-mail]
> <mailto:[e-mail]>
> https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
>
>
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Allen Hewes]

Hi Pierre,
> I'am using Koji + sign_unsigned.py + mash to build RPM on CentOS5

I'm interested in hearing about this from you.

If you run sign_unsigned.py how can you be using mash to create yum repos?
> From what I can tell, mash can only work with Koji tags as a source and a file system path as a destination. Are you putting RPM headers back into koji then running mash?

Can you explain how you're doing this? (I am still a Koji neophyte, so go slow...)
> I have modified sign_unsigned.py to manage the passphrase. If 
> option is not used, sign_unsigned.py gives an empty 
> passphrase to 'rpm --resign' command.
>
> Add the Python expect module in import section (pexpect RPM 
> must installed) :
>
> import getpass
> +import pexpect
>
> Add the --passwd option in __init__() from SignUnsigned class:
>    self.parser.add_option("--passwd", action="store_true")
>
> Replace these lines in do_signing()
>            # loop in case password is mistyped
>            while os.system(cmd):
>                # sleep briefly (give user a chance to ctrl-C)
>                time.sleep(2)
>            # Use expect to give the passphrase
>            # LANG=C to have english question 'pass phrase'
>            os.environ['LC_ALL'] = 'C'
>            child = pexpect.spawn(cmd)
>            # Wait for 'pass phrase'
>            child.expect('phrase:')
>            if not self.options.passwd:
>                child.sendline('\r')
>            else:
>                child.sendline("%s" % self.options.passwd)
>            child.expect(pexpect.EOF)

And Thanks for patch! I think I will have a need for it.

/allen

[Oliver Falk]

Hi Allen!

I'm not sure how the Fedora guys do it... There's a lot of black 
(scripting) magic involved I guess. :-)

And yes, the script is already using the the larger key size, but that's 
not hard to "fix"...

Come on guys, show us your dirty little tricks! :-P

Best,
  Oliver

Am 14.12.2010 06:54, schrieb Allen Hewes:
> > Hi Allen!
> >
> > You might want to look at the following post:
> >
> > http://www.mail-archive.com/[e-mail]/ms
>
> g02187.html
> > -of
>
> Hi Oliver,
>
> Thanks for link. I had not come across this thread.
>
> It would appear that currently there isn't any method to sign RPMs within koji or mash. You can import prebuilt RPMs with signatures into Koji. I don't know much about importing RPMs into koji because I haven't had a need.
>
> Do the Fedora guys use the sign_unsigned.py script for the official Fedora yum repos? If so, how do they use mash? Because it looks to me that if you use this script, it does one of the steps mash does; fetching RPMs out of koji tags.
>
> I would have guessed that the Fedora guys generate their yum repos via mash from koji tags and then sign RPMs.
>
> I'd have to modify this script to suit my needs, but I think I could do it. It also looks like it relies on a newer version of RPM, the rpm command for key size == 4096 is one spot I noticed.
>
> Also, I have to enter a passphrase when I sign my RPMs but this script doesn't have any provisions for that. Is there a way to make rpm --resign not prompt for a passphrase?
>
> Has there been any talk about adding RPM signing to mash? It seems like that'd be a good place for it.
>
> Thanks,
> /allen
>
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Vijay N. Majagaonkar]

Hi All,

I m new to this system forgive me if i am doing something wrong here !

We can add sing key to build-tag at the time of creating tag..

[vijay@koji ~]$ koji add-tag --help
Usage: koji add-tag [options] name
(Specify the --help global option for a list of other help options)

Options:
  -h, --help       show this help message and exit
  --parent=PARENT  Specify parent
  --arches=ARCHES  Specify arches
*  --sigkey=SIGKEY  Specify signing key*

"Human Knowledge Belongs To World"

~thanks
Vijay.

On Tue, Dec 14, 2010 at 1:19 PM, Oliver Falk  wrote:
> Hi Allen!
>
> I'm not sure how the Fedora guys do it... There's a lot of black
> (scripting) magic involved I guess. :-)
>
> And yes, the script is already using the the larger key size, but that's
> not hard to "fix"...
>
> Come on guys, show us your dirty little tricks! :-P
>
> Best,
>  Oliver
>
> Am 14.12.2010 06:54, schrieb Allen Hewes:
> > > Hi Allen!
> > >
> > > You might want to look at the following post:
> > >
> > > http://www.mail-archive.com/[e-mail]/ms
> >
> > g02187.html
> > > -of
> >
> > Hi Oliver,
> >
> > Thanks for link. I had not come across this thread.
> >
> > It would appear that currently there isn't any method to sign RPMs within
>
> koji or mash. You can import prebuilt RPMs with signatures into Koji. I
> don't know much about importing RPMs into koji because I haven't had a need.
> > Do the Fedora guys use the sign_unsigned.py script for the official
>
> Fedora yum repos? If so, how do they use mash? Because it looks to me that
> if you use this script, it does one of the steps mash does; fetching RPMs
> out of koji tags.
> > I would have guessed that the Fedora guys generate their yum repos via
>
> mash from koji tags and then sign RPMs.
> > I'd have to modify this script to suit my needs, but I think I could do
>
> it. It also looks like it relies on a newer version of RPM, the rpm command
> for key size == 4096 is one spot I noticed.
> > Also, I have to enter a passphrase when I sign my RPMs but this script
>
> doesn't have any provisions for that. Is there a way to make rpm --resign
> not prompt for a passphrase?
> > Has there been any talk about adding RPM signing to mash? It seems like
>
> that'd be a good place for it.
> > Thanks,
> > /allen
> >
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Mike McLean]

On 12/14/2010 03:45 AM, Vijay N. Majagaonkar wrote:
> Hi All,
>
> I m new to this system forgive me if i am doing something wrong here !
>
> We can add sing key to build-tag at the time of creating tag..
>
> [vijay@koji ~]$ koji add-tag --help
> Usage: koji add-tag [options] name
> (Specify the --help global option for a list of other help options)
>
> Options:
>    -h, --help       show this help message and exit
>    --parent=PARENT  Specify parent
>    --arches=ARCHES  Specify arches
> --sigkey=SIGKEY  Specify signing key*

I'm not quite sure what you mean. Koji's add-tag subcommand does not and 
has never supported a --sigkey option. Is this actual output or are you 
proposing a new feature?

[Vijay N. Majagaonkar]

yea it has option --sigkey, in older version of koji,
"koji-1.2.6-1.20090109.1409.46.fc9.noarch" this version i have installed in
server,

I understand why you asked this question, even i don't find this option on
my client box that has newer version installed :)

V!jay

On Wed, Dec 15, 2010 at 12:50 AM, Mike McLean  wrote:
> On 12/14/2010 03:45 AM, Vijay N. Majagaonkar wrote:
> > Hi All,
> >
> > I m new to this system forgive me if i am doing something wrong here !
> >
> > We can add sing key to build-tag at the time of creating tag..
> >
> > [vijay@koji ~]$ koji add-tag --help
> > Usage: koji add-tag [options] name
> > (Specify the --help global option for a list of other help options)
> >
> > Options:
> >    -h, --help       show this help message and exit
> >    --parent=PARENT  Specify parent
> >    --arches=ARCHES  Specify arches
> > --sigkey=SIGKEY  Specify signing key*
>
> I'm not quite sure what you mean. Koji's add-tag subcommand does not and
> has never supported a --sigkey option. Is this actual output or are you
> proposing a new feature?
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Mike McLean]

On 12/14/2010 09:29 PM, Vijay N. Majagaonkar wrote:
> yea it has option --sigkey, in older version of koji,
> "koji-1.2.6-1.20090109.1409.46.fc9.noarch" this version i have installed in
> server,
>
> I understand why you asked this question, even i don't find this option on
> my client box that has newer version installed :)

I've search git and no commit has ever added it. In fact no command has 
ever had a --sigkey option (only --key, --keytab, or --sigs). The 
version you have installed on your server has either been strangely 
modified (it's not even clear what associating a sigkey with a tag would 
mean) or is experiencing some sort of bug.

You should compare /usr/bin/koji on your server to the version of 
cli/koji from the koji-1.2.6-1.fc9 tag in git.
https://fedorahosted.org/koji/browser/cli/koji?rev=7899fb704fd33ace3654b569e...

[Vijay N. Majagaonkar]

I can send you diff of koji python file :)
Just for update siging of RPM file is working with our buid system

V!jay

On Dec 16, 2010, at 1:49 AM, Mike McLean  wrote:
> On 12/14/2010 09:29 PM, Vijay N. Majagaonkar wrote:
> > yea it has option --sigkey, in older version of koji,
> > "koji-1.2.6-1.20090109.1409.46.fc9.noarch" this version i have installed in
> > server,
> >
> > I understand why you asked this question, even i don't find this option on
> > my client box that has newer version installed :)
>
> I've search git and no commit has ever added it. In fact no command has 
> ever had a --sigkey option (only --key, --keytab, or --sigs). The 
> version you have installed on your server has either been strangely 
> modified (it's not even clear what associating a sigkey with a tag would 
> mean) or is experiencing some sort of bug.
>
> You should compare /usr/bin/koji on your server to the version of 
> cli/koji from the koji-1.2.6-1.fc9 tag in git.
> https://fedorahosted.org/koji/browser/cli/koji?rev=7899fb704fd33ace3654b569e...
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Paul B Schroeder]

Vijay..

Your version of koji is using this:
ftp://ftp.pramberger.at/software/patches/koji-1.2.2-rpmsign.tar.gz

Cheers...Paul...

On 12/16/2010 02:25 AM, Vijay N Majagaonkar wrote:
> I can send you diff of koji python file :)
> Just for update siging of RPM file is working with our buid system
>
> V!jay
>
> On Dec 16, 2010, at 1:49 AM, Mike McLean  wrote:
> > On 12/14/2010 09:29 PM, Vijay N. Majagaonkar wrote:
> > > yea it has option --sigkey, in older version of koji,
> > > "koji-1.2.6-1.20090109.1409.46.fc9.noarch" this version i have installed in
> > > server,
> > >
> > > I understand why you asked this question, even i don't find this option on
> > > my client box that has newer version installed :)
> >
> > I've search git and no commit has ever added it. In fact no command has
> > ever had a --sigkey option (only --key, --keytab, or --sigs). The
> > version you have installed on your server has either been strangely
> > modified (it's not even clear what associating a sigkey with a tag would
> > mean) or is experiencing some sort of bug.
> >
> > You should compare /usr/bin/koji on your server to the version of
> > cli/koji from the koji-1.2.6-1.fc9 tag in git.
> > https://fedorahosted.org/koji/browser/cli/koji?rev=7899fb704fd33ace3654b569e...
> > --
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Vijay N. Majagaonkar]

http://comments.gmane.org/gmane.linux.redhat.fedora.buildsystem/2987

Thanks Paul, It cleared why there was difference in our koji sever compare
to git :)

Mike, Paul has done changes to our koji server and he submitted diff in the
above link

Thanks for the help
V!jay.

On Fri, Dec 17, 2010 at 12:27 PM, Paul B Schroeder wrote:
> Vijay..
>
> Your version of koji is using this:
> ftp://ftp.pramberger.at/software/patches/koji-1.2.2-rpmsign.tar.gz
>
> Cheers...Paul...
>
> On 12/16/2010 02:25 AM, Vijay N Majagaonkar wrote:
> > I can send you diff of koji python file :)
> > Just for update siging of RPM file is working with our buid system
> >
> > V!jay
> >
> > On Dec 16, 2010, at 1:49 AM, Mike McLean  wrote:
> >
> > On 12/14/2010 09:29 PM, Vijay N. Majagaonkar wrote:
> > > > yea it has option --sigkey, in older version of koji,
> > > > "koji-1.2.6-1.20090109.1409.46.fc9.noarch" this version i have installed
> > > > in
> > > > server,
> > > >
> > > > I understand why you asked this question, even i don't find this option
> > > > on
> > > > my client box that has newer version installed :)
> > >
> > > I've search git and no commit has ever added it. In fact no command has
> > > ever had a --sigkey option (only --key, --keytab, or --sigs). The
> > > version you have installed on your server has either been strangely
> > > modified (it's not even clear what associating a sigkey with a tag would
> > > mean) or is experiencing some sort of bug.
> > >
> > > You should compare /usr/bin/koji on your server to the version of
> > > cli/koji from the koji-1.2.6-1.fc9 tag in git.
> > > https://fedorahosted.org/koji/browser/cli/koji?rev=7899fb704fd33ace3654b569e...
> > >
> > > buildsys mailing list
> > > [e-mail]
> > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> >
> > --
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> --
>
> Paul B Schroeder
> <paulbsch "at" vbridges "dot" com>

[Mike McLean]

On 12/17/2010 07:44 AM, Vijay N. Majagaonkar wrote:
> http://comments.gmane.org/gmane.linux.redhat.fedora.buildsystem/2987
>
> Thanks Paul, It cleared why there was difference in our koji sever compare
> to git :)
>
> Mike, Paul has done changes to our koji server and he submitted diff in the
> above link

Perhaps, but the plugin listed there does not provide the command line 
option in question, nor does it provide a mechanism for such an option 
to work.

[Paul B Schroeder]

On 12/17/2010 12:18 PM, Mike McLean wrote:
> On 12/17/2010 07:44 AM, Vijay N. Majagaonkar wrote:
> > http://comments.gmane.org/gmane.linux.redhat.fedora.buildsystem/2987
> >
> > Thanks Paul, It cleared why there was difference in our koji sever compare
> > to git :)
> >
> > Mike, Paul has done changes to our koji server and he submitted diff in the
> > above link
>
> Perhaps, but the plugin listed there does not provide the command line
> option in question, nor does it provide a mechanism for such an option
> to work.

Vijay is confused.  I didn't write the code that his koji setup has been 
patched with.

Vijay, the plugin I sent to the list last night is something I just 
wrote this week and is completely different.

[Josh Boyer]

On Tue, Dec 14, 2010 at 2:49 AM, Oliver Falk  wrote:
> Hi Allen!
>
> I'm not sure how the Fedora guys do it... There's a lot of black
> (scripting) magic involved I guess. :-)
>
> And yes, the script is already using the the larger key size, but that's
> not hard to "fix"...
>
> Come on guys, show us your dirty little tricks! :-P

There are no dirty tricks.  It essentially goes:

1) RPMs built in koji
2) sign_unsigned.py is run against various koji tags.  Either
dist-f1x-candidates or dist-f1x-updates-testing, or whichever need to
be signed.  NOTE: rawhide is not signed
3) mash is run against the tag after the RPMs have all been signed.
4) Bodhi does some symlink switching after all the mashes have
completed successfully and the new repos are pushed to the mirrors.

That's it.  No tricks, nothing super efficient.

At some point, there was discussion on having koji do the signing
automatically after a build completes.  I think that is still a long
term plan, but it requires a project to use a single key for all
packages.

josh

[Oliver Falk]

On 12/14/2010 01:50 PM, Josh Boyer wrote:
> On Tue, Dec 14, 2010 at 2:49 AM, Oliver Falk  wrote:
> > Hi Allen!
> >
> > I'm not sure how the Fedora guys do it... There's a lot of black
> > (scripting) magic involved I guess. :-)
> >
> > And yes, the script is already using the the larger key size, but that's
> > not hard to "fix"...
> >
> > Come on guys, show us your dirty little tricks! :-P
>
> There are no dirty tricks.  It essentially goes:
> RPMs built in koji
> sign_unsigned.py is run against various koji tags.  Either
>
> dist-f1x-candidates or dist-f1x-updates-testing, or whichever need to
> be signed.  NOTE: rawhide is not signed
> 3) mash is run against the tag after the RPMs have all been signed.
> 4) Bodhi does some symlink switching after all the mashes have
> completed successfully and the new repos are pushed to the mirrors.
>
> That's it.  No tricks, nothing super efficient.
>
> At some point, there was discussion on having koji do the signing
> automatically after a build completes.  I think that is still a long
> term plan, but it requires a project to use a single key for all
> packages.

Sorry Josh. This wasn't meant as offence! I just never saw any 
documentation about this part - maybe I just didn't look hard enough. :-)

-of

[Josh Boyer]

On Tue, Dec 14, 2010 at 7:58 AM, Oliver Falk  wrote:
> > There are no dirty tricks.  It essentially goes:
> > RPMs built in koji
> > sign_unsigned.py is run against various koji tags.  Either
> >
> > dist-f1x-candidates or dist-f1x-updates-testing, or whichever need to
> > be signed.  NOTE: rawhide is not signed
> > 3) mash is run against the tag after the RPMs have all been signed.
> > 4) Bodhi does some symlink switching after all the mashes have
> > completed successfully and the new repos are pushed to the mirrors.
> >
> > That's it.  No tricks, nothing super efficient.
> >
> > At some point, there was discussion on having koji do the signing
> > automatically after a build completes.  I think that is still a long
> > term plan, but it requires a project to use a single key for all
> > packages.
>
> Sorry Josh. This wasn't meant as offence! I just never saw any
> documentation about this part - maybe I just didn't look hard enough. :-)

Oh, I wasn't offended in the slightest.  If anything I was wishing we
had dirty tricks, because how it is done right now is fairly
inefficient.

And yes, there should be more documentation in this area under the
RelEng SOPs.  I'll take the blame for that, as I never got around to
writing it.

josh

[Oliver Falk]

On 12/14/2010 02:03 PM, Josh Boyer wrote:
> On Tue, Dec 14, 2010 at 7:58 AM, Oliver Falk  wrote:
> > > There are no dirty tricks.  It essentially goes:
> > > RPMs built in koji
> > > sign_unsigned.py is run against various koji tags.  Either
> > >
> > > dist-f1x-candidates or dist-f1x-updates-testing, or whichever need to
> > > be signed.  NOTE: rawhide is not signed
> > > 3) mash is run against the tag after the RPMs have all been signed.
> > > 4) Bodhi does some symlink switching after all the mashes have
> > > completed successfully and the new repos are pushed to the mirrors.
> > >
> > > That's it.  No tricks, nothing super efficient.
> > >
> > > At some point, there was discussion on having koji do the signing
> > > automatically after a build completes.  I think that is still a long
> > > term plan, but it requires a project to use a single key for all
> > > packages.
> >
> > Sorry Josh. This wasn't meant as offence! I just never saw any
> > documentation about this part - maybe I just didn't look hard enough. :-)
>
> Oh, I wasn't offended in the slightest.

Fine.
> If anything I was wishing we had dirty tricks, because how it is
> done right now is fairly inefficient.

That's true!
> And yes, there should be more documentation in this area under the
> RelEng SOPs.  I'll take the blame for that, as I never got around to
> writing it.

:-) I know that problem!

-of

[Christos Triantafyllidis]

On Dec 14, 2010, at 2:50 PM, Josh Boyer wrote:
> On Tue, Dec 14, 2010 at 2:49 AM, Oliver Falk  wrote:
> > Hi Allen!
> >
> > I'm not sure how the Fedora guys do it... There's a lot of black
> > (scripting) magic involved I guess. :-)
> >
> > And yes, the script is already using the the larger key size, but that's
> > not hard to "fix"...
> >
> > Come on guys, show us your dirty little tricks! :-P
>
> There are no dirty tricks.  It essentially goes:
> RPMs built in koji
> sign_unsigned.py is run against various koji tags.  Either
>
> dist-f1x-candidates or dist-f1x-updates-testing, or whichever need to
> be signed.  NOTE: rawhide is not signed
> 3) mash is run against the tag after the RPMs have all been signed.
> 4) Bodhi does some symlink switching after all the mashes have
> completed successfully and the new repos are pushed to the mirrors.
>
> That's it.  No tricks, nothing super efficient.
>
> At some point, there was discussion on having koji do the signing
> automatically after a build completes.  I think that is still a long
> term plan, but it requires a project to use a single key for all
> packages.
>
> josh

Hi Josh, all,

i'm reading this thread and i think that i've missed some point. What is the purpose of signing an RPM if you sign it on an online machine? I haven't seen the sign_unsigned.py source yet but i guess what should be there is a mechanism that should download the unsigned RPMs, then a manual operation of RPM sign (possibly on an offline or at least access restricted node), and then another script to import the signed RPMs (or just the signatures).

Am i seeing this from a wrong perspective? does Fedora really sign the RPMs online? I guess this gets even worse if the sign operation is done more efficiently, automatically after each koji build.

I hope i don't sound offensive, but these were my thoughts as i want/need to implement something like this in our local koji installation and i hoped that you were using something more sophisticated.

Regards,
Christos

[Josh Boyer]

On Tue, Dec 14, 2010 at 8:20 AM, Christos Triantafyllidis
 wrote:
> Hi Josh, all,
>
> i'm reading this thread and i think that i've missed some point. What is the purpose of signing an RPM if you sign it on an online machine? I haven't seen the sign_unsigned.py source yet but i guess what should be there is a mechanism that should download the unsigned RPMs, then a manual operation of RPM sign (possibly on an offline or at least access restricted node), and then another script to import the signed RPMs (or just the signatures).

sign_unsigned.py uses sigul under the covers to do the actual RPM signing.
> Am i seeing this from a wrong perspective? does Fedora really sign the RPMs online? I guess this gets even worse if the sign operation is done more efficiently, automatically after each koji build.

No, currently the signing is done on a secure node.  There is a sigul
bridge that interfaces with sigul client requests and a secure node in
the datacenter that can only talk to that bridge.  It is not
accessible via http, ssh, etc.  The server signs the RPMs using the
keys.

Additionally, the server also generates those keys and stores them
locally.  Authenticated users can request it sign an RPM with a
particular key, but those users don't actually have access to that key
at all.  The gpg key never leaves the sigul server.  This is much
better than what was previously done, as that required sending the
key(s) to trusted individuals on multiple machines.

josh

[Jesse Keating]

On 12/14/10 5:20 AM, Christos Triantafyllidis wrote:
> Hi Josh, all,
>
> i'm reading this thread and i think that i've missed some point. What
> is the purpose of signing an RPM if you sign it on an online machine?

The purpose of the signature is to provide something downstream 
consumers can check to ensure that the build came from a Fedora source 
control and the Fedora build system.  We don't intend our signatures to 
provide anything beyond that.  The "online" machine in question is very 
secure, as secure as we can reasonably make it with open source tools. 
I'm not aware of a reasonable way to feed hundreds of thousands of rpms 
into an /offline/ machine to sign them, and then cart all of them back 
/off/ the offline machine and back onto a network.

Our package store, many TBs large, exists in a datacenter where the only 
access is remote.  This is a fact of our infrastructure and one that we 
have to deal with.  Creating sigul as Josh described is our effort to 
secure the process as much as possible, while remaining a open project 
that provides access to more than just Red Hat employees.
> I haven't seen the sign_unsigned.py source yet but i guess what
> should be there is a mechanism that should download the unsigned
> RPMs, then a manual operation of RPM sign (possibly on an offline or
> at least access restricted node), and then another script to import
> the signed RPMs (or just the signatures).

You really should start reading the sources then.  You've basically 
described how sigul and sigulsign_unsigned works.

The sigulsign_unsigned script takes in options and data such as what 
builds to sign, or what koji tag to sign, and what key to use.  It will 
prompt you for your personal passphrase for a particular key (every user 
has their own, nobody knows the real key passphrase).  This data is 
passed along to the sigul bridge which is semi-restricted.  The bridge 
operates against our account system to validate user certs, and with the 
"vault" which is a very secure and limited access machine where the 
actual gpg keys live.  The bridge fetches the unsigned rpms, passes them 
to the vault.  The vault signs them and passes back the signed header, 
which the bridge will import into koji.
> Am i seeing this from a wrong perspective? does Fedora really sign
> the RPMs online? I guess this gets even worse if the sign operation
> is done more efficiently, automatically after each koji build.

If "online" matches the above, then yes.  And we are moving to a point 
where we can sign each package as it completes a build in koji.  The 
only "worse" part is that we'd have one more extremely limited access 
machine with cached credentials to a "buildsystem" key so that it can 
detect a finished build, enact a sign+import of said build.  As I stated 
before the GPG key is only intended to validate that the build happened 
in an "official" way on "official" Fedora resources.  Nothing beyond that.
> I hope i don't sound offensive, but these were my thoughts as i
> want/need to implement something like this in our local koji
> installation and i hoped that you were using something more
> sophisticated.

[Christos Triantafyllidis]

Hi Jesse,
   thanks for these clarifications, actually what Josh already written in his last mail was more or less what i wanted to see. I'm glad that you have such a mechanism for secure "online" signatures already in place. Thus i'll just copy setup/idea and then try to make it fit my needs. :).

I just got a little bit disappointed in the thought of "we automatically sign whatever gets out from our koji" on an online machine (so why not the koji itself) and have consumers trust that this "build by Fedora based Fedora's SCM". What Josh and you described are totally fine for me.

Thanks again,
Christos

On Dec 14, 2010, at 8:26 PM, Jesse Keating wrote:
> On 12/14/10 5:20 AM, Christos Triantafyllidis wrote:
> > Hi Josh, all,
> >
> > i'm reading this thread and i think that i've missed some point. What
> > is the purpose of signing an RPM if you sign it on an online machine?
>
> The purpose of the signature is to provide something downstream 
> consumers can check to ensure that the build came from a Fedora source 
> control and the Fedora build system.  We don't intend our signatures to 
> provide anything beyond that.  The "online" machine in question is very 
> secure, as secure as we can reasonably make it with open source tools. 
> I'm not aware of a reasonable way to feed hundreds of thousands of rpms 
> into an /offline/ machine to sign them, and then cart all of them back 
> /off/ the offline machine and back onto a network.
>
> Our package store, many TBs large, exists in a datacenter where the only 
> access is remote.  This is a fact of our infrastructure and one that we 
> have to deal with.  Creating sigul as Josh described is our effort to 
> secure the process as much as possible, while remaining a open project 
> that provides access to more than just Red Hat employees.
> > I haven't seen the sign_unsigned.py source yet but i guess what
> > should be there is a mechanism that should download the unsigned
> > RPMs, then a manual operation of RPM sign (possibly on an offline or
> > at least access restricted node), and then another script to import
> > the signed RPMs (or just the signatures).
>
> You really should start reading the sources then.  You've basically 
> described how sigul and sigulsign_unsigned works.
>
> The sigulsign_unsigned script takes in options and data such as what 
> builds to sign, or what koji tag to sign, and what key to use.  It will 
> prompt you for your personal passphrase for a particular key (every user 
> has their own, nobody knows the real key passphrase).  This data is 
> passed along to the sigul bridge which is semi-restricted.  The bridge 
> operates against our account system to validate user certs, and with the 
> "vault" which is a very secure and limited access machine where the 
> actual gpg keys live.  The bridge fetches the unsigned rpms, passes them 
> to the vault.  The vault signs them and passes back the signed header, 
> which the bridge will import into koji.
> > Am i seeing this from a wrong perspective? does Fedora really sign
> > the RPMs online? I guess this gets even worse if the sign operation
> > is done more efficiently, automatically after each koji build.
>
> If "online" matches the above, then yes.  And we are moving to a point 
> where we can sign each package as it completes a build in koji.  The 
> only "worse" part is that we'd have one more extremely limited access 
> machine with cached credentials to a "buildsystem" key so that it can 
> detect a finished build, enact a sign+import of said build.  As I stated 
> before the GPG key is only intended to validate that the build happened 
> in an "official" way on "official" Fedora resources.  Nothing beyond that.
> > I hope i don't sound offensive, but these were my thoughts as i
> > want/need to implement something like this in our local koji
> > installation and i hoped that you were using something more
> > sophisticated.
>
> -- 
> Jesse Keating
> Fedora -- Freedom² is a feature!
> identi.ca: http://identi.ca/jkeating
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Allen Hewes]

Hi josh,
> RPMs built in koji
> sign_unsigned.py is run against various koji tags.  Either
>
> dist-f1x-candidates or dist-f1x-updates-testing, or whichever 
> need to be signed.  NOTE: rawhide is not signed
> 3) mash is run against the tag after the RPMs have all been signed.
> 4) Bodhi does some symlink switching after all the mashes 
> have completed successfully and the new repos are pushed to 
> the mirrors.

This is more for my understanding, because I think I am already out of my comfort zone for managing my little amount of packages with Koji. Koji really helps me a lot w.r.t having a handfull of custom packages, it allows me to focus on the package and not the mess of yum/rpm dependency.

But, how do you get from step 2 to step 3? I went ahead and read Jesse's e-mail (great explanation, btw) and is this where the sigul system comes into play? Because working from my knowledge of Koji, it would appear to me that

A) the output of step 2 would be signed RPMs on a filesystem

B) how do you get the signed RPMs on disk (the filesystem) back into Koji? I think this is the process I have come across in previous posts from Jesse/Mike. I don't understand what sigul is could be the issue...

C) does step 3 mean that you have taken twice as much space on disk because know you have two versions (one signed and one unsigned) of the same NVR build?

D) if I go to Fedora's Koji, I don't see two NVR RPMs per package. I think I am missing something here w.r.t getting signed RPMs back into Koji.

Thanks for outline Josh,

-Allen

[Jesse Keating]

On 12/14/10 7:57 PM, Allen Hewes wrote:
> B) how do you get the signed RPMs on disk (the filesystem) back into
> Koji? I think this is the process I have come across in previous
> posts from Jesse/Mike. I don't understand what sigul is could be the
> issue...

Sigul is calling koji import-sig in order to import the signed header 
from the signed rpm.  Koji can keep any number of signed headers for a 
package.  You can then ask koji to write out a version of rpms with 
signed headers.  This is actually done through the API, there is no 
command line option for it.  (koji list-api to get a list of all the 
possible API calls)
> C) does step 3 mean that you have taken twice as much space on disk
> because know you have two versions (one signed and one unsigned) of
> the same NVR build?

If you keep the signed one around yes.  You don't have to sign every 
build, or you don't have to keep the signed version around after you 
publish them somewhere.
> D) if I go to Fedora's Koji, I don't see two NVR RPMs per package. I
> think I am missing something here w.r.t getting signed RPMs back into
> Koji.

http://kojipkgs.fedoraproject.org/packages/pungi/2.1.4/1.fc14/data/signed/97... 
  You'll see signed rpms there.  The signature content gets put into the 
<package>/<version>/<release>/data/ directory structure.

[Mike Bonnet]

On 12/14/2010 11:23 PM, Jesse Keating wrote:
> On 12/14/10 7:57 PM, Allen Hewes wrote:
> > B) how do you get the signed RPMs on disk (the filesystem) back into
> > Koji? I think this is the process I have come across in previous
> > posts from Jesse/Mike. I don't understand what sigul is could be the
> > issue...
>
> Sigul is calling koji import-sig in order to import the signed header 
> from the signed rpm.  Koji can keep any number of signed headers for a 
> package.  You can then ask koji to write out a version of rpms with 
> signed headers.  This is actually done through the API, there is no 
> command line option for it.  (koji list-api to get a list of all the 
> possible API calls)

You can use koji write-signed-rpm to get it to write out a copy signed
with a previously imported signature.  The API works too though.
> > C) does step 3 mean that you have taken twice as much space on disk
> > because know you have two versions (one signed and one unsigned) of
> > the same NVR build?
>
> If you keep the signed one around yes.  You don't have to sign every 
> build, or you don't have to keep the signed version around after you 
> publish them somewhere.
> > D) if I go to Fedora's Koji, I don't see two NVR RPMs per package. I
> > think I am missing something here w.r.t getting signed RPMs back into
> > Koji.
>
> http://kojipkgs.fedoraproject.org/packages/pungi/2.1.4/1.fc14/data/signed/97... 
>   You'll see signed rpms there.  The signature content gets put into the 
> <package>/<version>/<release>/data/ directory structure.

[Jesse Keating]

On 12/15/10 5:48 AM, Mike Bonnet wrote:
> > > Sigul is calling koji import-sig in order to import the signed header 
> > > from the signed rpm.  Koji can keep any number of signed headers for a 
> > > package.  You can then ask koji to write out a version of rpms with 
> > > signed headers.  This is actually done through the API, there is no 
> > > command line option for it.  (koji list-api to get a list of all the 
> > > possible API calls)
>
> You can use koji write-signed-rpm to get it to write out a copy signed
> with a previously imported signature.  The API works too though.

Ah, I didn't see it in koji help --admin

[Jesse Keating]

On 12/13/10 9:54 PM, Allen Hewes wrote:
> > Hi Allen!
> >
> > You might want to look at the following post:
> >
> > http://www.mail-archive.com/[e-mail]/ms
>
> g02187.html
> > -of
>
> Hi Oliver,
>
> Thanks for link. I had not come across this thread.
>
> It would appear that currently there isn't any method to sign RPMs
> within koji or mash. You can import prebuilt RPMs with signatures
> into Koji. I don't know much about importing RPMs into koji because I
> haven't had a need.
>
> Do the Fedora guys use the sign_unsigned.py script for the official
> Fedora yum repos? If so, how do they use mash? Because it looks to me
> that if you use this script, it does one of the steps mash does;
> fetching RPMs out of koji tags.
>
> I would have guessed that the Fedora guys generate their yum repos
> via mash from koji tags and then sign RPMs.
>
> I'd have to modify this script to suit my needs, but I think I could
> do it. It also looks like it relies on a newer version of RPM, the
> rpm command for key size == 4096 is one spot I noticed.
>
> Also, I have to enter a passphrase when I sign my RPMs but this
> script doesn't have any provisions for that. Is there a way to make
> rpm --resign not prompt for a passphrase?
>
> Has there been any talk about adding RPM signing to mash? It seems
> like that'd be a good place for it.

I think there is some confusion here.  sign_unsigned.py was our old 
tool.  I wrote a new one when we started using the sigul secure signing 
backend. 
https://fedorahosted.org/rel-eng/browser/scripts/sigulsign_unsigned.py

This client interacts with the sigul bridge, which then interacts with 
the sigul server to actually rpmsign the files.  Then the signed headers 
get imported into koji, and we ask koji to write out a set of the rpms 
with the signed headers.  It's these signed copies that mash would fetch 
(if so configured).

Because we do composes in automated or semi-automated fashion, and often 
these composes re-use many existing packages, it doesn't make sense to 
mash and then some hours later come back to punch in a passphrase to 
(re)sign a ton of rpms.  We sign and store them in koji so that they can 
be fetched later by automated tools.

[Oliver Falk]

Hi Jesse!

Just want to mention, that sigul might be a bit too much effort for a 
private (or even corporate) koji setup...

-of

Am 14.12.2010 19:17, schrieb Jesse Keating:
> On 12/13/10 9:54 PM, Allen Hewes wrote:
> > > Hi Allen!
> > >
> > > You might want to look at the following post:
> > >
> > > http://www.mail-archive.com/[e-mail]/ms
> >
> > g02187.html
> > > -of
> >
> > Hi Oliver,
> >
> > Thanks for link. I had not come across this thread.
> >
> > It would appear that currently there isn't any method to sign RPMs
> > within koji or mash. You can import prebuilt RPMs with signatures
> > into Koji. I don't know much about importing RPMs into koji because I
> > haven't had a need.
> >
> > Do the Fedora guys use the sign_unsigned.py script for the official
> > Fedora yum repos? If so, how do they use mash? Because it looks to me
> > that if you use this script, it does one of the steps mash does;
> > fetching RPMs out of koji tags.
> >
> > I would have guessed that the Fedora guys generate their yum repos
> > via mash from koji tags and then sign RPMs.
> >
> > I'd have to modify this script to suit my needs, but I think I could
> > do it. It also looks like it relies on a newer version of RPM, the
> > rpm command for key size == 4096 is one spot I noticed.
> >
> > Also, I have to enter a passphrase when I sign my RPMs but this
> > script doesn't have any provisions for that. Is there a way to make
> > rpm --resign not prompt for a passphrase?
> >
> > Has there been any talk about adding RPM signing to mash? It seems
> > like that'd be a good place for it.
>
> I think there is some confusion here.  sign_unsigned.py was our old
> tool.  I wrote a new one when we started using the sigul secure signing
> backend.
> https://fedorahosted.org/rel-eng/browser/scripts/sigulsign_unsigned.py
>
> This client interacts with the sigul bridge, which then interacts with
> the sigul server to actually rpmsign the files.  Then the signed headers
> get imported into koji, and we ask koji to write out a set of the rpms
> with the signed headers.  It's these signed copies that mash would fetch
> (if so configured).
>
> Because we do composes in automated or semi-automated fashion, and often
> these composes re-use many existing packages, it doesn't make sense to
> mash and then some hours later come back to punch in a passphrase to
> (re)sign a ton of rpms.  We sign and store them in koji so that they can
> be fetched later by automated tools.

[Mike McLean]

On 12/14/2010 03:00 PM, Oliver Falk wrote:
> Just want to mention, that sigul might be a bit too much effort for a
> private (or even corporate) koji setup...

I'm not sure that's true. Koji is already a pretty complex system. sigul 
is not that much more work. I think it just needs better documentation.

[Oliver Falk]

Am 14.12.2010 21:33, schrieb Mike McLean:
> On 12/14/2010 03:00 PM, Oliver Falk wrote:
> > Just want to mention, that sigul might be a bit too much effort for a
> > private (or even corporate) koji setup...
>
> I'm not sure that's true. Koji is already a pretty complex system. sigul
> is not that much more work. I think it just needs better documentation.

From a security point of view, you are totally right. If you already 
use koji and did this complex setup, there's really no reason to not 
also do the sigul setup. In the world of virtual machines, a small 
signing host, might not be a big stunt anyway.

-of

[Florian La Roche]

On Tue, Dec 14, 2010 at 09:00:23PM +0100, Oliver Falk wrote:
> Hi Jesse!
>
> Just want to mention, that sigul might be a bit too much effort for a 
> private (or even corporate) koji setup...

sigul runs fine, also (with less security) on a koji server. And overall
it is also "just" about setting up a few config files and certs.
Signing rpms will increase the io load on the koji server...

regards,

Florian La Roche

[Oliver Falk]

On 12/15/2010 07:46 AM, Florian La Roche wrote:
> On Tue, Dec 14, 2010 at 09:00:23PM +0100, Oliver Falk wrote:
> > Hi Jesse!
> >
> > Just want to mention, that sigul might be a bit too much effort for a
> > private (or even corporate) koji setup...
>
> sigul runs fine, also (with less security) on a koji server. And overall
> it is also "just" about setting up a few config files and certs.
> Signing rpms will increase the io load on the koji server...

But if you want to run sigul, you have security in mind.. Why would one 
want to install it then on a less secured system? :-)

-of

Hello,

I'd like to chime in and say that I have actually tried to get sigul setup and would *love* to see more documentation on how to do this.   Particularly, I'd be interested in details on how to get it to auth via a 389 Server without having to setup pkgdb and fas.

Thanks,
Brian

On Dec 14, 2010, at 1:00 PM, Oliver Falk wrote:
> Hi Jesse!
>
> Just want to mention, that sigul might be a bit too much effort for a 
> private (or even corporate) koji setup...
>
> -of

<SNIP>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys