3:45 p.m.
Hey all,
Here's my first draft of a SourceURL guideline. This tries to
encapsulate current practices but a few new things had to be added
related to SRPMs where no upstream source exists. This draft will
probably need some touching up as I whipped it up pretty quickly but
hopefully it captures the spirit of what we're trying to achieve.
The latest version is available at:
http://www.fedoraproject.org/wiki/PackagingDrafts/SourceUrl
'''
= Referencing Source =
One of the design goals of rpm is to cleanly separate upstream source from vendor
modifications. For the Fedora packager, this means that sources used to build a package
should be the vanilla sources available from upstream. To help reviewers and QA scripts
verify this, the packager needs to indicate where a reviewer can find the source that was
used to make the rpm.
The most common case is where upstream distributes source as a tar.gz, tar.bz2 or zip
archive that we can download from an upstream website. In these cases you must use a full
URL to the package in the SourceX: line. For example::
{{{
Source0: http://download.sourceforge.net/%{name}/%{name}-%{version}.tar.gz
Source0:
http://ftp.gnome.org/pub/GNOME/sources/gnome-common/2.12/gnome-common-2.1...
}}}
There are several cases where upstream is not providing the source to you in an upstream
tarball. In these cases you must document how to generate the tarball used in the rpm
either through a spec file comment or a script included as a separate SourceX:.
Here are some specific examples:
== Using Revision Control ==
In some cases you may want to pull sources from upstream's revision control system
because there have been many changes since the last release and you think that a tarball
that you generate from there will more accurately show how the package relates to
upstream's development. Here's how you can use a comment to show where the source
came from::
{{{
# The source for this package was pulled from upstream's cvs. Use the
# following commands to generate the tarball:
# svn export -r 250 http://www.foo.org/svn/foo/trunk foo-20070221
# tar -czvf foo-20070221.tar.gz foo-20070221
Source0: foo-20070221.tar.gz
}}}
When pulling from revision control, please remember to use a Name-version-release
compatible with the release guidelines. In particular, check the section on Naming
Snapshots.
== When Upstream uses Prohibited Code ==
Some upstream packages include patents or trademarks that we are not allowed to ship even
as source code. In these cases you have to modify the source tarball to remove this code
before you even upload it to the build system. Here's an example of using a script to
document how you went from the upstream tarball to the one included in the package:
From the spec:
{{{
Source0: libfoo-1.0-nopatents.tar.gz
# libfoo contains patented code that we cannot ship. Therefore we use
# this script to remove the patented code before shipping it.
# Download the upstream tarball and invoke this script while in the
# tarball's directory:
# ./generate-tarball.sh 1.0
Source1: generate-tarball.sh
}}}
generate-tarball.sh:
{{{
#!/bin/sh
VERSION=$1
tar -xzvf libfoo-$VERSION.tar.gz
rm libfoo-$VERSION/src/patentedcodec.c
sed -i -e 's/patentedcodec.c//' libfoo-$VERSION/src/Makefile
tar -czvf libfoo-$VERSION-nopatents.tar.gz
}}}
== We are Upstream ==
For some packages where we are the upstream authors, for instance, the system-config-*
tools, the source rpm that we distribute is the canonical source of the files. There is
no public revision control system or publically released tarball for these programs so
there is no tarball to list. Add a comment like the following to the spec:
{{{
# This is a Red Hat maintained package which is specific to
# our distribution. Thus the source is only available from
# within this srpm.
Source0: system-config-foo-1.0.tar.gz
}}}
'''
-Toshio
3:57 p.m.
On Wed, Feb 14, 2007 at 01:45:25PM -0800, Toshio Kuratomi wrote:
Hey all,
Here's my first draft of a SourceURL guideline. This tries to
encapsulate current practices but a few new things had to be added
related to SRPMs where no upstream source exists. This draft will
probably need some touching up as I whipped it up pretty quickly but
hopefully it captures the spirit of what we're trying to achieve.
Looks OK. But since we're commenting on source origin could somewhere
a kind request ("SHOULD") to (srpm-)package upstream sources/patches
with original timestamps where possible be embedded?
--
Axel.Thimm at ATrpms.net
4:40 p.m.
On Wed, 2007-02-14 at 22:57 +0100, Axel Thimm wrote:
On Wed, Feb 14, 2007 at 01:45:25PM -0800, Toshio Kuratomi wrote:
> Hey all,
>
> Here's my first draft of a SourceURL guideline. This tries to
> encapsulate current practices but a few new things had to be added
> related to SRPMs where no upstream source exists. This draft will
> probably need some touching up as I whipped it up pretty quickly but
> hopefully it captures the spirit of what we're trying to achieve.
Looks OK. But since we're commenting on source origin could somewhere
a kind request ("SHOULD") to (srpm-)package upstream sources/patches
with original timestamps where possible be embedded?
That sounds like a good best practice. It sounds like a separate item
the way things are currently phrased. Do you have some wording to fit
it in, or do you just want to throw it in a separate recommendation.
-Toshio
5:30 p.m.
Toshio Kuratomi wrote:
On Wed, 2007-02-14 at 22:57 +0100, Axel Thimm wrote:
> Looks OK. But since we're commenting on source origin could
> somewhere a kind request ("SHOULD") to (srpm-)package upstream
> sources/patches with original timestamps where possible be
> embedded?
That sounds like a good best practice. It sounds like a separate
item the way things are currently phrased. Do you have some wording
to fit it in, or do you just want to throw it in a separate
recommendation.
Is this section from the packaging guidelines what you're after?
http://fedoraproject.org/wiki/Packaging/Guidelines#head-0239576e441f9ef53...
"When downloading sources, patches etc, consider using a client that
preserves the upstream timestamps. For example wget -N or curl -R. To
make the change global for wget, add this to your ~/.wgetrc:
timestamping = on, and for curl, add to your ~/.curlrc: -R."
--
Todd OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp
======================================================================
To have a successful relationship, I must learn to make it look like
I'm giving as much as I'm getting.
5:37 p.m.
On Wed, Feb 14, 2007 at 06:30:43PM -0500, Todd Zullinger wrote:
Toshio Kuratomi wrote:
> On Wed, 2007-02-14 at 22:57 +0100, Axel Thimm wrote:
>> Looks OK. But since we're commenting on source origin could
>> somewhere a kind request ("SHOULD") to (srpm-)package upstream
>> sources/patches with original timestamps where possible be
>> embedded?
>
> That sounds like a good best practice. It sounds like a separate
> item the way things are currently phrased. Do you have some wording
> to fit it in, or do you just want to throw it in a separate
> recommendation.
Is this section from the packaging guidelines what you're after?
http://fedoraproject.org/wiki/Packaging/Guidelines#head-0239576e441f9ef53...
"When downloading sources, patches etc, consider using a client that
preserves the upstream timestamps. For example wget -N or curl -R. To
make the change global for wget, add this to your ~/.wgetrc:
timestamping = on, and for curl, add to your ~/.curlrc: -R."
Indeed, thanks for pointing this out, Todd!
Toshio, forget my request then, unless you want to consolidate or
rearange parts of the guidelines talking about handling upstream
sources/patches.
--
Axel.Thimm at ATrpms.net
5:33 p.m.
On Wed, Feb 14, 2007 at 02:40:06PM -0800, Toshio Kuratomi wrote:
On Wed, 2007-02-14 at 22:57 +0100, Axel Thimm wrote:
> On Wed, Feb 14, 2007 at 01:45:25PM -0800, Toshio Kuratomi wrote:
> > Hey all,
> >
> > Here's my first draft of a SourceURL guideline. This tries to
> > encapsulate current practices but a few new things had to be added
> > related to SRPMs where no upstream source exists. This draft will
> > probably need some touching up as I whipped it up pretty quickly but
> > hopefully it captures the spirit of what we're trying to achieve.
>
> Looks OK. But since we're commenting on source origin could somewhere
> a kind request ("SHOULD") to (srpm-)package upstream sources/patches
> with original timestamps where possible be embedded?
That sounds like a good best practice. It sounds like a separate item
the way things are currently phrased. Do you have some wording to fit
it in, or do you just want to throw it in a separate recommendation.
since it's a weak suggestion and only a subsentence it would be nice
to interweave in the part that discusses unmodified upstream
sources. I agree the topic "URL" is not exactly describing timestamps :)
Maybe the general topic could be abstracted to "Dealing with sources
and patches" or similar, so it wouldn't be completely out of the water.
As a phrasing I would suggest "Whenever possible try to maintain
timestamps of sources or patches".
--
Axel.Thimm at ATrpms.net
4:06 p.m.
On Wed, Feb 14, 2007 at 01:45:25PM -0800, Toshio Kuratomi wrote:
Hey all,
Here's my first draft of a SourceURL guideline. This tries to
Overall it looks good to me. I see 2 other situations that may be worth
mentionning:
1. the tarball was found on a regular site, kept in a srpm, or a disk drive,
a mirror, the lookaside cache or whatever place, and afterwards the
initial download source disappeared. It is similar than the "we are
upstream" case but not exactly the same. The fedora maintainer should
maintain the package in that case, that is fix bugs and so on but
without being upstream, only in the fedora cvs (with patches). These
packages may be on the road to retirement, but not necessarily,
especially when these are old, stable, and little packages.
2. sometimes upstream don't have a properly versioned tarball. In that
case it should be asked to upstream to version properly, but it is
not always possible. I think that the rule for those cases should be
to provide the full url in a comment, and add a version to the tarball
based, for example on the timestamp of the file (but not necessarily).
For example:
# Source1 is only used for documentation
# renamed to tex4ht-all-YYYYMMDD.zip - based on last timestamp in
# directory
Source1: tex4ht-all-20050228.zip
# unversioned upstream source, downloaded with wget -N
#Source1 http://www.cse.ohio-state.edu/~gurari/TeX4ht/tex4ht-all.zip
--
Pat
4:37 p.m.
On Wed, 2007-02-14 at 13:45 -0800, Toshio Kuratomi wrote:
'''
= Referencing Source =
One of the design goals of rpm is to cleanly separate upstream
source from vendor modifications. For the Fedora packager, this
means that sources used to build a package should be the vanilla
sources available from upstream. To help reviewers and QA scripts
verify this, the packager needs to indicate where a reviewer can find
the source that was used to make the rpm.
caillon had this to say in the bug which spawned this:
'''
Looks good from the brief glance I took, but I strongly feel this whole
thing should be a "good practises" recommendation and not a requirement.
If you're trying to prevent against "bad" RPMs, well you're not going
to
do that if there are exceptions... Even for a good SRPM, someone could
simply fork an open source project, not have a repo other than the SRPM,
and distribute whatever code they want that way in extras,
theoretically. This has no bearing on the actual packaging or quality
of RPMs. It's only redeeming quality is that it might potentially help
with automated verification of upstream sources, but that does not exist
right now and that potential benefit should be enough to convince most
packagers to do this. There's simply no reason to make it a hard
requirement IMO other than because it's always been that way (which is
no real reason).
'''
6:46 a.m.
Toshio Kuratomi wrote:
On Wed, 2007-02-14 at 13:45 -0800, Toshio Kuratomi wrote:
> '''
> = Referencing Source =
>
> One of the design goals of rpm is to cleanly separate upstream
> source from vendor modifications. For the Fedora packager, this
> means that sources used to build a package should be the vanilla
> sources available from upstream. To help reviewers and QA scripts
> verify this, the packager needs to indicate where a reviewer can find
> the source that was used to make the rpm.
caillon had this to say in the bug which spawned this:
'''
Looks good from the brief glance I took, but I strongly feel this whole
thing should be a "good practises" recommendation and not a requirement.
Or, I don't know, something like "Packaging Guidelines"? (:
I like your draft, good work.
-- Rex
4:59 p.m.
On Wednesday 14 February 2007, Toshio Kuratomi wrote:
Tiny detail: s/download./downloads./ - the former has been shaky for quite a
while now, but I've never seen the latter fail.
5:06 p.m.
On Mittwoch 14 Februar 2007, Ville Skyttä wrote:
On Wednesday 14 February 2007, Toshio Kuratomi wrote:
> Source0:
> http://download.sourceforge.net/%{name}/%{name}-%{version}.tar.gz
Tiny detail: s/download./downloads./ - the former has been shaky for quite
a while now, but I've never seen the latter fail.
I just remembered, that in another thread I read prdownloads.sourceforge.net
as the recommended sourceforge download source. But I do not know, which
works better or if there are any differences.
Regards,
Till
1:17 a.m.
On Thursday 15 February 2007, Till Maas wrote:
On Mittwoch 14 Februar 2007, Ville Skyttä wrote:
> On Wednesday 14 February 2007, Toshio Kuratomi wrote:
> > Source0:
> > http://download.sourceforge.net/%{name}/%{name}-%{version}.tar.gz
>
> Tiny detail: s/download./downloads./ - the former has been shaky for
> quite a while now, but I've never seen the latter fail.
I just remembered, that in another thread I read
prdownloads.sourceforge.net as the recommended sourceforge download source.
But I do not know, which works better or if there are any differences.
prdownloads tarball URLs aren't suitable, they're not directly downloadable.
3:52 a.m.
On Donnerstag 15 Februar 2007, Ville Skyttä wrote:
prdownloads tarball URLs aren't suitable, they're not
directly
downloadable.
When I try to download
http://prdownloads.sourceforge.net/optipng/optipng-0.5.4.tar.gz, it works,
but it is a redirection to
http://downloads.sourceforge.net/optipng/optipng-0.5.4.tar.gz, so I guess
downloads is the better choice.
Regards,
Till
5:02 p.m.
On Mittwoch 14 Februar 2007, Toshio Kuratomi wrote:
The latest version is available at:
http://www.fedoraproject.org/wiki/PackagingDrafts/SourceUrl
I made some minor corrections, e.g. using example.com instead of foo.org and
links to the mentioned NamingGuidelines sections. I also added a section
about the correct sourceforge.net URL, afaik it is not documented anywhere
else, and a section why using %{version} in SourceX: is good. I hope this is
ok. :-)
I noticed, that in the section titled "When Upstream uses Prohibited Code"
there is not mentioned, that the URL of the original tarball must be included
in the generate-tarball.sh script, or, that it also must download it, e.g.
curl http://www.example.com/files/libfoo-$VERSION.tar.gz | xzvf
libfoo-$VERSION.tar.gz
Also, one could argue, that the script must always be
named "generate-tarball.sh" (or whatever name is appropriate) because then
QA-scripts could automatically use them.
Regards,
Till
10:48 a.m.
Toshio Kuratomi (a.badger(a)gmail.com) said:
{{{
# The source for this package was pulled from upstream's cvs. Use the
# following commands to generate the tarball:
# svn export -r 250 http://www.foo.org/svn/foo/trunk foo-20070221
# tar -czvf foo-20070221.tar.gz foo-20070221
Source0: foo-20070221.tar.gz
}}}
Might want to match your SCMs in the example. :)
As for the whole thing, I'd argue that they should be suggests-guidelines,
not required-guidelines.
Bill
11:42 a.m.
On Thu, 2007-02-15 at 11:48 -0500, Bill Nottingham wrote:
Toshio Kuratomi (a.badger(a)gmail.com) said:
> {{{
> # The source for this package was pulled from upstream's cvs. Use the
> # following commands to generate the tarball:
> # svn export -r 250 http://www.foo.org/svn/foo/trunk foo-20070221
> # tar -czvf foo-20070221.tar.gz foo-20070221
> Source0: foo-20070221.tar.gz
> }}}
Might want to match your SCMs in the example. :)
As for the whole thing, I'd argue that they should be suggests-guidelines,
not required-guidelines.
Why? Is there any reason why we don't want this information in the
package?
~spot
11:48 a.m.
Tom 'spot' Callaway (tcallawa(a)redhat.com) said:
> As for the whole thing, I'd argue that they should be
suggests-guidelines,
> not required-guidelines.
Why? Is there any reason why we don't want this information in the
package?
I dunno - it just feels like something that should be 'please do this',
not 'package rejected because you didn't do this.'
Bill
11:55 a.m.
Bill Nottingham wrote:
Tom 'spot' Callaway (tcallawa(a)redhat.com) said:
>> As for the whole thing, I'd argue that they should be suggests-guidelines,
>> not required-guidelines.
> Why? Is there any reason why we don't want this information in the
> package?
I dunno - it just feels like something that should be 'please do this',
not 'package rejected because you didn't do this.'
If folks want to be able to reliably verify upstream sources, this kind
of stuff is pretty much required. I can't think of very many legit
cases otherwise.
-- Rex
11:55 a.m.
On Thu, 2007-02-15 at 12:48 -0500, Bill Nottingham wrote:
Tom 'spot' Callaway (tcallawa(a)redhat.com) said:
> > As for the whole thing, I'd argue that they should be suggests-guidelines,
> > not required-guidelines.
>
> Why? Is there any reason why we don't want this information in the
> package?
I dunno - it just feels like something that should be 'please do this',
not 'package rejected because you didn't do this.'
Sorry. If you don't eat your meat, you cannot have any pudding.
HOW CAN YOU HAVE ANY PUDDING IF YOU DON'T EAT YOUR MEAT?!?
~spot
11:56 a.m.
Tom 'spot' Callaway (tcallawa(a)redhat.com) said:
Sorry. If you don't eat your meat, you cannot have any pudding.
HOW CAN YOU HAVE ANY PUDDING IF YOU DON'T EAT YOUR MEAT?!?
OK, just a little pinprick...
More seriously, if we're trying to encode this sort of metadata, it
would be nice to have a real format/mechanism, rather than 'a defined
comment syntax.'
Bill
12:01 p.m.
On Thu, 2007-02-15 at 12:56 -0500, Bill Nottingham wrote:
Tom 'spot' Callaway (tcallawa(a)redhat.com) said:
> Sorry. If you don't eat your meat, you cannot have any pudding.
>
> HOW CAN YOU HAVE ANY PUDDING IF YOU DON'T EAT YOUR MEAT?!?
OK, just a little pinprick...
More seriously, if we're trying to encode this sort of metadata, it
would be nice to have a real format/mechanism, rather than 'a defined
comment syntax.'
OK. Suggestions on format welcome. :)
~spot
12:19 p.m.
On Thu, Feb 15, 2007 at 07:11:45PM +0100, Till Maas wrote:
On Donnerstag 15 Februar 2007, Tom 'spot' Callaway wrote:
> OK. Suggestions on format welcome. :)
I suggest a shell script with the name "generate-tarball.sh" that generates
the desired tarball and is included as SourceX:.
There should be the package name prepended. Something like
foo-generate-tarball.sh
--
Pat
6277
days inactive
6278
days old
packaging@lists.fedoraproject.org
22 comments
9 participants
participants (9)
-
Axel Thimm -
Bill Nottingham -
Patrice Dumas -
Rex Dieter -
Till Maas -
Todd Zullinger -
Tom Callaway -
Toshio Kuratomi -
Ville Skyttä