It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Neal Becker wrote:
It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Are they the same? If so, my gut would be to use the one in libotf, if it's complete, maybe in a subpackage, and have openmpi remove it's own and Require it, sort of the way we handle bundled libraries.
-J
Jon Ciesla wrote:
Neal Becker wrote:
It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Are they the same? If so, my gut would be to use the one in libotf, if it's complete, maybe in a subpackage, and have openmpi remove it's own and Require it, sort of the way we handle bundled libraries.
-J
They have nothing in common, just unfortunately the same name.
On Wed, 2009-09-16 at 14:53 -0400, Neal Becker wrote:
It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Actually, in this case it shouldn't be a problem for very long, since every MPI compiler (& runtime) and MPI application should be reworked ASAP to conform to the new MPI packaging guidelines.
OpenMPI is currently under work at https://bugzilla.redhat.com/show_bug.cgi?id=521334
On Wed, Sep 16, 2009 at 10:04:17PM +0300, Jussi Lehtola wrote:
On Wed, 2009-09-16 at 14:53 -0400, Neal Becker wrote:
It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Actually, in this case it shouldn't be a problem for very long, since every MPI compiler (& runtime) and MPI application should be reworked ASAP to conform to the new MPI packaging guidelines.
OpenMPI is currently under work at https://bugzilla.redhat.com/show_bug.cgi?id=521334
I'm currently polishing up the new OpenMPI packaging in rawhide, and (barring surprises) I should have new openmpi and openmpi-devel packages in updates by the end of the week. These will obsolete the openmpi-libs and openmpi-vt packages, and otfdump will be in openmpi-devel: %{_libdir}/%{name}/bin/ where it won't interfere with libotf.
-- JF
Jay Fenlason wrote:
On Wed, Sep 16, 2009 at 10:04:17PM +0300, Jussi Lehtola wrote:
On Wed, 2009-09-16 at 14:53 -0400, Neal Becker wrote:
It seems both openmpi and libotf supply a %{_bindir}/otfdump. otf is either:
OpenTypeFont (libotf) or OpenTraceFormat (openmpi)
I maintain libotf. I'm not sure how to address this.
My only interest in libotf is so emacs can use it. For that, it doesn't need the binaries. Perhaps they could be put somewhere else?
I don't know how important otfdump is to openmpi, since I don't use it. My guess is that in both libotf and openmpi, neither is critical to function, but is a debug aid.
Actually, in this case it shouldn't be a problem for very long, since every MPI compiler (& runtime) and MPI application should be reworked ASAP to conform to the new MPI packaging guidelines.
OpenMPI is currently under work at https://bugzilla.redhat.com/show_bug.cgi?id=521334
I'm currently polishing up the new OpenMPI packaging in rawhide, and (barring surprises) I should have new openmpi and openmpi-devel packages in updates by the end of the week. These will obsolete the openmpi-libs and openmpi-vt packages, and otfdump will be in openmpi-devel: %{_libdir}/%{name}/bin/ where it won't interfere with libotf.
-- JF
Sounds like the optimal solution. Make sure to push at least to F-11, not sure if it affects F-10. . .
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
* Here, object means filesystem object. I'm not sure if there are any other types of objects to worry about.
On Wed, 2009-09-16 at 18:45 -0400, Neal Becker wrote:
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
- Here, object means filesystem object. I'm not sure if there are any other
types of objects to worry about.
Brainstorming: a script that walks the yum repo's filelist.tar.gz, and figures out a list of filename collisions, filtering by directories in the default PATH
Attached is a first pass at a python script that does this.
Output from the script when run upon [1] is below. Caveat: the script probably has bugs.
Does this look useful?
ulockmgr_server /bin/ulockmgr_server from fuse /usr/bin/ulockmgr_server from fuse
telnet /usr/bin/telnet from telnet /usr/kerberos/bin/telnet from krb5-workstation-clients
gzip /bin/gzip from gzip /usr/bin/gzip from gzip
fusermount /bin/fusermount from fuse /usr/bin/fusermount from fuse
stap-env /usr/bin/stap-env from systemtap-client /usr/bin/stap-env from systemtap /usr/bin/stap-env from systemtap-server
winemaker /usr/bin/winemaker from wine-devel /usr/bin/winemaker from wine-common
ftp /usr/bin/ftp from ftp /usr/kerberos/bin/ftp from krb5-workstation-clients
pinentry /usr/bin/pinentry from pinentry /usr/bin/pinentry from pinentry-gtk /usr/bin/pinentry from pinentry-qt
kadmin /usr/kerberos/bin/kadmin from krb5-workstation-servers /usr/kerberos/bin/kadmin from krb5-workstation
lzcmp /usr/bin/lzcmp from xz-lzma-compat /usr/bin/lzcmp from lzma
lzgrep /usr/bin/lzgrep from xz-lzma-compat /usr/bin/lzgrep from lzma
lzdiff /usr/bin/lzdiff from xz-lzma-compat /usr/bin/lzdiff from lzma
lzcat /usr/bin/lzcat from xz-lzma-compat /usr/bin/lzcat from lzma
lzmainfo /usr/bin/lzmainfo from xz-lzma-compat /usr/bin/lzmainfo from lzma
lzfgrep /usr/bin/lzfgrep from xz-lzma-compat /usr/bin/lzfgrep from lzma
plymouth /bin/plymouth from plymouth /usr/bin/plymouth from plymouth
gawk /bin/gawk from gawk /usr/bin/gawk from gawk
ex /bin/ex from vim-minimal /usr/bin/ex from vim-enhanced
ircd /usr/bin/ircd from ircd-ratbox /usr/bin/ircd from ircd-hybrid
cut /bin/cut from coreutils /usr/bin/cut from coreutils
towhee-mpi /usr/bin/towhee-mpi from towhee-mpi /usr/bin/towhee-mpi from towhee
pscp /usr/bin/pscp from putty /usr/bin/pscp from pssh
links /usr/bin/links from links /usr/bin/links from elinks
rsh /usr/kerberos/bin/rsh from krb5-workstation-clients /usr/bin/rsh from rsh
awk /bin/awk from gawk /usr/bin/awk from gawk
tmda-ofmipd /usr/bin/tmda-ofmipd from tmda-ofmipd /usr/bin/tmda-ofmipd from tmda
kvno /usr/kerberos/bin/kvno from krb5-workstation-servers /usr/kerberos/bin/kvno from krb5-workstation
sclient /usr/kerberos/bin/sclient from krb5-devel /usr/kerberos/bin/sclient from krb5-server
unlzma /usr/bin/unlzma from xz-lzma-compat /usr/bin/unlzma from lzma
ktutil /usr/kerberos/bin/ktutil from krb5-workstation-servers /usr/kerberos/bin/ktutil from krb5-workstation
lzegrep /usr/bin/lzegrep from xz-lzma-compat /usr/bin/lzegrep from lzma
ntfs-3g /bin/ntfs-3g from ntfs-3g /usr/bin/ntfs-3g from ntfs-3g
k5srvutil /usr/kerberos/bin/k5srvutil from krb5-workstation-servers /usr/kerberos/bin/k5srvutil from krb5-workstation
rlogin /usr/kerberos/bin/rlogin from krb5-workstation-clients /usr/bin/rlogin from rsh
stap-find-servers /usr/bin/stap-find-servers from systemtap-client /usr/bin/stap-find-servers from systemtap-server
lzma /usr/bin/lzma from xz-lzma-compat /usr/bin/lzma from lzma
kde4-doxygen.sh /usr/bin/kde4-doxygen.sh from kdelibs-devel /usr/bin/kde4-doxygen.sh from kdelibs
find /bin/find from findutils /usr/bin/find from findutils
jasper5-setclasspath.sh /usr/bin/jasper5-setclasspath.sh from tomcat5 /usr/bin/jasper5-setclasspath.sh from tomcat5-jasper
translate /usr/bin/translate from libtranslate /usr/bin/translate from surfraw
stap-gen-cert /usr/bin/stap-gen-cert from systemtap /usr/bin/stap-gen-cert from systemtap-server
stap-authorize-cert /usr/bin/stap-authorize-cert from systemtap /usr/bin/stap-authorize-cert from systemtap-server
rcp /usr/kerberos/bin/rcp from krb5-workstation-clients /usr/kerberos/bin/rcp from krb5-workstation-servers /usr/bin/rcp from rsh
env /bin/env from coreutils /usr/bin/env from coreutils
jspc5.sh /usr/bin/jspc5.sh from tomcat5 /usr/bin/jspc5.sh from tomcat5-jasper
synergyc /usr/bin/synergyc from synergy /usr/bin/synergyc from synergy-plus
synergys /usr/bin/synergys from synergy /usr/bin/synergys from synergy-plus
xemacs /usr/bin/xemacs from xemacs-nox /usr/bin/xemacs from xemacs
kill /bin/kill from util-linux-ng /usr/bin/kill from util-linux-ng
gettext /bin/gettext from gettext /usr/bin/gettext from gettext
winedump /usr/bin/winedump from wine-core /usr/bin/winedump from wine-devel
ntfsmount /bin/ntfsmount from ntfs-3g /usr/bin/ntfsmount from ntfs-3g
slideshow /usr/bin/slideshow from plt-scheme /usr/bin/slideshow from batik-slideshow
stap-report /usr/bin/stap-report from systemtap-runtime /usr/bin/stap-report from systemtap
gunzip /bin/gunzip from gzip /usr/bin/gunzip from gzip
servlink /usr/bin/servlink from ircd-ratbox /usr/bin/servlink from ircd-hybrid
lzless /usr/bin/lzless from xz-lzma-compat /usr/bin/lzless from lzma
jasper5.sh /usr/bin/jasper5.sh from tomcat5 /usr/bin/jasper5.sh from tomcat5-jasper
lzmore /usr/bin/lzmore from xz-lzma-compat /usr/bin/lzmore from lzma
lzmadec /usr/bin/lzmadec from xz-lzma-compat /usr/bin/lzmadec from lzma
[1] http://archive.linux.duke.edu/pub/fedora/linux/development/i386/os/repodata/...
Why not report all conflicts, instead of only those on your PATH?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/16/2009 05:08 PM, Neal Becker wrote:
Why not report all conflicts, instead of only those on your PATH?
Why not just have a system somewhere with Everything installed, including new stuff? That's what I almost have myself, and I noticed the libotf/openmpi conflict quite a while ago (it's why I don't have the latter installed anymore).
- -- J. Randall Owens | http://www.ghiapet.net/ ProofReading Markup Language | http://prml.sourceforge.net/
On Wed, 2009-09-16 at 17:32 -0700, J. Randall Owens wrote:
Why not just have a system somewhere with Everything installed, including new stuff? That's what I almost have myself, and I noticed the libotf/openmpi conflict quite a while ago (it's why I don't have the latter installed anymore).
Because that's difficult to maintain in an automated test world, where you'd like to run such tests after each and every build.
On Wed, 2009-09-16 at 20:08 -0400, Neal Becker wrote:
Why not report all conflicts, instead of only those on your PATH?
Because these aren't file level conflicts, as in they can both exist on the filesystem at the same time and RPM won't care. However they can lead to unexpected things due to PATH collision, if you type "foo" and there are multiple "foo"'s in your path, are you sure you know which one you'll get?
Jesse Keating wrote:
On Wed, 2009-09-16 at 20:08 -0400, Neal Becker wrote:
Why not report all conflicts, instead of only those on your PATH?
Because these aren't file level conflicts, as in they can both exist on the filesystem at the same time and RPM won't care. However they can lead to unexpected things due to PATH collision, if you type "foo" and there are multiple "foo"'s in your path, are you sure you know which one you'll get?
But the original problem was a file level conflict. Is it ever valid for 2 packages to own the same file?
On Wed, Sep 16, 2009 at 21:10:16 -0400, Neal Becker ndbecker2@gmail.com wrote:
But the original problem was a file level conflict. Is it ever valid for 2 packages to own the same file?
Yes. At a minimum the file's contents has to be identical in the two packages.
On Wed, 16 Sep 2009 21:10:16 -0400, Neal wrote:
But the original problem was a file level conflict.
I've been reporting file level conflicts for a long time (with a script on my fedorapeople page which is lacking automatic multilib support, however), but the openmpi/libotf conflict has been reported independently at least once: https://bugzilla.redhat.com/496131
Meanwhile the autoqa project is creating file level conflicts reports, too: https://fedorahosted.org/pipermail/autoqa-results/2009-September/thread.html
Is it ever valid for 2 packages to own the same file?
If their checksums are the same, yes, RPM won't complain.
One could apply the existing packaging guidelines to this case, however: https://fedoraproject.org/wiki/Packaging:Guidelines#Duplicate_Files
There would need to be a good reason for two packages to include the same file - instead of doing this via a shared package dependency.
Le Jeu 17 septembre 2009 02:47, Jesse Keating a écrit :
On Wed, 2009-09-16 at 20:08 -0400, Neal Becker wrote:
Why not report all conflicts, instead of only those on your PATH?
Because these aren't file level conflicts, as in they can both exist on the filesystem at the same time and RPM won't care. However they can lead to unexpected things due to PATH collision, if you type "foo" and there are multiple "foo"'s in your path, are you sure you know which one you'll get?
Well, for $PATH dirs you need to check the basename, for others you need to check the full name (and hash, and perms). Both checks are valid and useful
On Wed, 2009-09-16 at 20:08 -0400, Neal Becker wrote:
Why not report all conflicts, instead of only those on your PATH?
This is acting at the level of individual filenames (dropping the directory component from the path), and doesn't have any knowledge about a file beyond its full installation path.
The PATH filtering allows it to avoid lots of false-positives from all of the various "COPYING", "README" etc files we drop into various places in the filesystem.
Dave
On Wed, 16 Sep 2009, David Malcolm wrote:
On Wed, 2009-09-16 at 18:45 -0400, Neal Becker wrote:
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
- Here, object means filesystem object. I'm not sure if there are any other
types of objects to worry about.
Brainstorming: a script that walks the yum repo's filelist.tar.gz, and figures out a list of filename collisions, filtering by directories in the default PATH
Attached is a first pass at a python script that does this.
Output from the script when run upon [1] is below. Caveat: the script probably has bugs.
Does this look useful?
David, Yes it does look useful.
I wrote something similar:
http://skvidal.fedorapeople.org/misc/potential_conflict.py
which is what I believe autoqa is starting from for their file conflict checker.
-sv
On Thu, 2009-09-17 at 09:57 -0400, Seth Vidal wrote:
On Wed, 16 Sep 2009, David Malcolm wrote:
On Wed, 2009-09-16 at 18:45 -0400, Neal Becker wrote:
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
- Here, object means filesystem object. I'm not sure if there are any other
types of objects to worry about.
Brainstorming: a script that walks the yum repo's filelist.tar.gz, and figures out a list of filename collisions, filtering by directories in the default PATH
Attached is a first pass at a python script that does this.
Output from the script when run upon [1] is below. Caveat: the script probably has bugs.
Does this look useful?
David, Yes it does look useful.
I wrote something similar:
http://skvidal.fedorapeople.org/misc/potential_conflict.py
which is what I believe autoqa is starting from for their file conflict checker.
Aha! Your approach looks superior, as you're leveraging all that extra info from the RPM headers about file hashes etc. Thanks.
On Thu, 17 Sep 2009, David Malcolm wrote:
On Thu, 2009-09-17 at 09:57 -0400, Seth Vidal wrote:
On Wed, 16 Sep 2009, David Malcolm wrote:
On Wed, 2009-09-16 at 18:45 -0400, Neal Becker wrote:
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
- Here, object means filesystem object. I'm not sure if there are any other
types of objects to worry about.
Brainstorming: a script that walks the yum repo's filelist.tar.gz, and figures out a list of filename collisions, filtering by directories in the default PATH
Attached is a first pass at a python script that does this.
Output from the script when run upon [1] is below. Caveat: the script probably has bugs.
Does this look useful?
David, Yes it does look useful.
I wrote something similar:
http://skvidal.fedorapeople.org/misc/potential_conflict.py
which is what I believe autoqa is starting from for their file conflict checker.
Aha! Your approach looks superior, as you're leveraging all that extra info from the RPM headers about file hashes etc. Thanks.
maybe a bit more thourough but I wouldn't call it superior - it takes forever to complete b/c you have to look at all those headers :(
Magic alternatives welcome.
Actually, I sometimes wonder if we could do like git does and only take the first segment from the file's checksum and store it somewhere maybe with the other filelist sqlite metadata. That way we could tell if we were likely or unlikely to hit a conflict.
it is certainly not foolproof but I bet we'd be able to tell if two files were most likely different and occupying the same path in what? 85% of the cases? maybe more than that.
-sv
On Thu, 2009-09-17 at 14:14 -0400, Seth Vidal wrote:
On Thu, 17 Sep 2009, David Malcolm wrote:
On Thu, 2009-09-17 at 09:57 -0400, Seth Vidal wrote:
On Wed, 16 Sep 2009, David Malcolm wrote:
On Wed, 2009-09-16 at 18:45 -0400, Neal Becker wrote:
Which makes me wonder, how could this conflict have been avoided? Is there a tool that would check any new package to see if any object* in it would conflict with any existing package? If not, sounds like a good thing to have.
- Here, object means filesystem object. I'm not sure if there are any other
types of objects to worry about.
Brainstorming: a script that walks the yum repo's filelist.tar.gz, and figures out a list of filename collisions, filtering by directories in the default PATH
Attached is a first pass at a python script that does this.
Output from the script when run upon [1] is below. Caveat: the script probably has bugs.
Does this look useful?
David, Yes it does look useful.
I wrote something similar:
http://skvidal.fedorapeople.org/misc/potential_conflict.py
which is what I believe autoqa is starting from for their file conflict checker.
Aha! Your approach looks superior, as you're leveraging all that extra info from the RPM headers about file hashes etc. Thanks.
maybe a bit more thourough but I wouldn't call it superior - it takes forever to complete b/c you have to look at all those headers :(
Magic alternatives welcome.
Well, define "forever"... my script takes about 30 seconds (and ~1GB RAM) on my workstation; if that's a bit improvement over your runtimes, perhaps you could try a hybrid approach of walking the filelist.xml.gz to quickly find possible conflicts, then only opening the rpm headers as needed to reject the false conflicts? Dunno
[snip content-addressed storage/hashing ideas]
Dave
On Thu, 17 Sep 2009, David Malcolm wrote:
Well, define "forever"... my script takes about 30 seconds (and ~1GB RAM) on my workstation; if that's a bit improvement over your runtimes, perhaps you could try a hybrid approach of walking the filelist.xml.gz to quickly find possible conflicts, then only opening the rpm headers as needed to reject the false conflicts? Dunno
That's what I do. More or less - I do it by putting all the filenames as a key in a dict and the value is a list of the pkgs which own that file.
That portion takes 50s and 350M of ram on my laptop for f11 and updates.
then it take the list of files which have > 1 pkg owning them and grabs headers to sort them out.
[snip content-addressed storage/hashing ideas]
awww but that was the most interesting part. :)
-sv
David Malcolm wrote:
kde4-doxygen.sh /usr/bin/kde4-doxygen.sh from kdelibs-devel /usr/bin/kde4-doxygen.sh from kdelibs
This one was the same file accidentally shipped in both subpackages. Fixed in 4.3.1-4.
Kevin Kofler
On Wednesday 16 September 2009, Jay Fenlason wrote:
otfdump will be in openmpi-devel: %{_libdir}/%{name}/bin/ where it won't interfere with libotf.
IIUC this will only help wrt. the packaging conflict; interference still happens as $PATH changes e.g. when loading/unloading the openmpi modules. I don't think there's any other way around that issue besides renaming one of the executables, but whether that's worth doing/necessary is another thing.
On Wed, 2009-09-16 at 22:55 +0300, Ville Skyttä wrote:
On Wednesday 16 September 2009, Jay Fenlason wrote:
otfdump will be in openmpi-devel: %{_libdir}/%{name}/bin/ where it won't interfere with libotf.
IIUC this will only help wrt. the packaging conflict; interference still happens as $PATH changes e.g. when loading/unloading the openmpi modules. I don't think there's any other way around that issue besides renaming one of the executables, but whether that's worth doing/necessary is another thing.
Yes, that is true. If the openmpi module is loaded then Open MPI's otfdump will be used. I'm not sure whether this is a problem, though, since the module is not loaded by default and one does not run into any conflicts. Renaming binaries is a nuisance, too.