[Disclaimer: This is probably going to end up as a blurb about dependencies - bear with me :)]
Firstly, I was just perusing through the FC4 package description list and was reminded of something I've thought of many times:
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
I think such a package dependency map would be quite useful to get an overview of the distribution, both for developers and users. "End nodes" in the graph could help identify candidates that could be moved to extras for instance (you might even tag vertices with the package size).
From the users perspective it would get a lot easier to get an overview
of the impact of removing a package, or if you want to rebuild a homebrew package for some reason ("I'd really like to just rebuild this package myself to the bleeding edge, but what else would it impact and maybe even break, if anything?").
Secondly I was thinking of a "DependencyLint" kind of tool - a small Python script maybe that would go through every installed RPM on a system and check if the requirements in the .spec files are correct and/or sufficient. Here I'm not quite sure about what the rules are for Fedora however - should absolutely everything that's needed to build a package be in BuildRequires:? Should every package that has C code require glibc? Or is it a goal to be as lax as possible to prevent (hopefully) unnecessary build problems?
In any case I was thinking of maybe using a combination of RPM querying and ldd (and possibly other tools). An example (from a FC1 box):
1. Get a list of all the RPM's on the system.
2. For each package, list the files and grep the bin/ and lib/ stuff. For instance 'rpm -ql mutt|grep bin' lists /usr/bin/mutt as one of it's files.
3. Check which dynamic libraries these binaries and libraries depend on using 'ldd'. 'ldd /usr/bin/mutt' lists /usr/lib/libz.so.1 among others.
4. For each of these, check which package provides them. 'rpm -q --whatprovides /usr/lib/libz.so.1' returns 'zlib-1.2.0.7-2'
5. Finally check that the original package depends on zlib. (This step I'm not sure what's the best way to query.) 'rpm -q --requires mutt' does not list zlib and 'rpm -e --test zlib' does not list mutt so we've actually found a missing dependency here.
These are just my first thoughts to maybe encourage some discussion. I'm sure this could be done in a better way and could probably also be expanded to do other stuff (I'm not very RPM-savvy). If you run a live testinstall of rawhide for instance such a tool could be run automatically after each update to detect any packaging problems wrt. dependencies.
Thoughts?
Regards, -- Tarjei
Tarjei Knapstad (tarjei.knapstad@predichem.com) said:
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
http://adrian.gimp.org/rpm-graph/
Bill
On Tue, Jun 21, 2005 at 09:45:20AM -0400, Bill Nottingham wrote:
Tarjei Knapstad (tarjei.knapstad@predichem.com) said:
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
That doesn't work with recent rpm python bindings. But I'm glad to see you refer to it, because I feel less dumb for not knowing that an equivalent program, "rpmgraph", is included in the rpm-devel package itself.
:)
On Tue, 2005-06-21 at 15:49, Matthew Miller wrote:
On Tue, Jun 21, 2005 at 09:45:20AM -0400, Bill Nottingham wrote:
Tarjei Knapstad (tarjei.knapstad@predichem.com) said:
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
That doesn't work with recent rpm python bindings. But I'm glad to see you refer to it, because I feel less dumb for not knowing that an equivalent program, "rpmgraph", is included in the rpm-devel package itself.
Thanks Bill, Matthew.
Looks like rpmgraph only works on a set of available packages though, and not on the local DB of already installed RPM's.
In any case, the example graph for RH 7.0 was a lot more of a clutter than I had feared/hoped and pretty much unusable as a reference. Think I'll just forget about this for now...
Thanks again, -- Tarjei
Tarjei Knapstad wrote:
In any case, the example graph for RH 7.0 was a lot more of a clutter than I had feared/hoped and pretty much unusable as a reference. Think I'll just forget about this for now...
Thanks again,
Tarjei
You don't need to give up.
Graph visualization tools that print out a static image are of limited use, as you've found.
There are tools out there for interactive graphing that let you pick out particular nodes and explore the relationships between them, and something like that is worth pursuing.
Tarjei Knapstad wrote:
In any case, the example graph for RH 7.0 was a lot more of a clutter than I had feared/hoped and pretty much unusable as a reference. Think I'll just forget about this for now...
Its interesting to me how often the value of a Graphviz rendering is just this, to indicate the unexpected complexity of a system.
I suppose it is more interesting to see a picture, but sometimes the output of gc (think wc on graphs) can be used as a measure of complexity with much less rendering time.
$ gc redhat-7.0-i386.dot 719 2681 redhat (redhat-7.0-i386.dot)
i.e. redhat-7.0-i386.dot contains 719 nodes (packages) and 2681 edges (dependencies).
John
On Tue, 2005-06-21 at 15:40 +0200, Tarjei Knapstad wrote:
[Disclaimer: This is probably going to end up as a blurb about dependencies - bear with me :)]
Firstly, I was just perusing through the FC4 package description list and was reminded of something I've thought of many times:
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
I think such a package dependency map would be quite useful to get an overview of the distribution, both for developers and users. "End nodes" in the graph could help identify candidates that could be moved to extras for instance (you might even tag vertices with the package size).
From the users perspective it would get a lot easier to get an overview
of the impact of removing a package, or if you want to rebuild a homebrew package for some reason ("I'd really like to just rebuild this package myself to the bleeding edge, but what else would it impact and maybe even break, if anything?").
Secondly I was thinking of a "DependencyLint" kind of tool - a small Python script maybe that would go through every installed RPM on a system and check if the requirements in the .spec files are correct and/or sufficient. Here I'm not quite sure about what the rules are for Fedora however - should absolutely everything that's needed to build a package be in BuildRequires:? Should every package that has C code require glibc? Or is it a goal to be as lax as possible to prevent (hopefully) unnecessary build problems?
In any case I was thinking of maybe using a combination of RPM querying and ldd (and possibly other tools). An example (from a FC1 box):
Get a list of all the RPM's on the system.
For each package, list the files and grep the bin/ and lib/ stuff.
For instance 'rpm -ql mutt|grep bin' lists /usr/bin/mutt as one of it's files.
- Check which dynamic libraries these binaries and libraries depend on
using 'ldd'. 'ldd /usr/bin/mutt' lists /usr/lib/libz.so.1 among others.
- For each of these, check which package provides them.
'rpm -q --whatprovides /usr/lib/libz.so.1' returns 'zlib-1.2.0.7-2'
- Finally check that the original package depends on zlib. (This step
I'm not sure what's the best way to query.) 'rpm -q --requires mutt' does not list zlib and 'rpm -e --test zlib' does not list mutt so we've actually found a missing dependency here.
These are just my first thoughts to maybe encourage some discussion. I'm sure this could be done in a better way and could probably also be expanded to do other stuff (I'm not very RPM-savvy). If you run a live testinstall of rawhide for instance such a tool could be run automatically after each update to detect any packaging problems wrt. dependencies.
1. Do an everything install of core. 2. install yum-utils 3. run package-cleanup --leaves
that'll tell you all the specific leaf nodes.
-sv
On Tue, 2005-06-21 at 15:46, seth vidal wrote:
On Tue, 2005-06-21 at 15:40 +0200, Tarjei Knapstad wrote:
These are just my first thoughts to maybe encourage some discussion. I'm sure this could be done in a better way and could probably also be expanded to do other stuff (I'm not very RPM-savvy). If you run a live testinstall of rawhide for instance such a tool could be run automatically after each update to detect any packaging problems wrt. dependencies.
- Do an everything install of core.
- install yum-utils
- run package-cleanup --leaves
that'll tell you all the specific leaf nodes.
Excellent, thanks Seth.
Cheers, -- Tarjei
Would it be possible to map out the package dependencies in Fedora by doing rpm queries and forming a directed graph of the results? (by using graphviz or something like that)
Yes you can. rpmgraph will give you the "runtime" dependencies. But if you are looking for the build dependencies, the Rookery build system has a tool in it for this purpose. http://www.web-insights.net/rookery/
What you will need to do is setup a repo of all the srpms. Then do the prep for build, and maybe even the check for build. Go into the analyze menu and it can do the whole repo, or just 1 package's dependencies. The 1 package can give you either the dependencies from that package back to nothing, or you can see what depends on that package, or both. This removes the clutter and lets you see the relationships that a particular package has.
I have an example for mkinitrd here: http://www.web-insights.net/rookery/mkinitrd.dep.gz
and server portion of FC3: http://www.web-insights.net/rookery/fc3.ps.gz
I have tools halfway complete to really tighten up the dependencies. Both in terms of build and runtime. I will try to finish these during the FC5 development cycle to get correct dependency information added back to the packages.
-Steve Grubb
__________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail
-
Bill Nottingham -
John Ellson -
Matthew Miller -
Paul A Houle -
seth vidal -
Steve G -
Tarjei Knapstad