R package dependencies, translated into RPM land.
by Allen S. Rout
OK; I've been chewing and debating on this with several folks, in
several fora, for some time. Here are the facts as I've been able to
determine them so far; at least some of them will be stupidly obvious
to most of you. Please tear them apart as indicated.
+ The R community is much sysadmin-y and progammer-y than other
language communities. Several positions about correctness which
lots of admins take as Truth (i.e. dependency cycle == BAD) they
find to be more of an aesthetic call. This is reasonable.
+ Different repositories in the R community have independant lives and
attitudes. There is modest competition and grumbling between
maintainers associated with different repos.
+ Package dependencies cross repo boundaries; sticking with the
'Better' repositories just won't work, and discussion of these
variations tends to make R folks testy.
conclusion: The goal of evolving the R packages into a DAG is a
non-starter.
+ There are four classes of dependency in R-package land: Requires,
Imports, Suggests, and Enhances.
+ Requires and Imports are required to load the package. [1]
+ Suggests may be required to fully CMD CHECK the package [1]
+ The need for suggests at CMD CHECK can be deactivated by build
config file. [2]
+ Many of the dependency cycles can be avoided if we ignore Suggests
as an RPM dependency.
Now, on to opinion:
+ We would like all official packages to have passed a full R CMD CHECK
+ We would like an absolute minimum of manual special case handling.
It may not be possible to make that amount zero.
So: Here's my suggested procedure for building any single package,
gangked from a message I sent to R-core:
1) Express binary package dependencies according to Depends and Imports.
I'll call this the 'narrow dependency graph'.
2) As part of the binary package build process, run CHECK
with R_CHECK_FORCE_SUGGESTS = false.
I'll pull nomenclature out of my ear and call these "built" but not
"checked".
3) Build all binary packages which are downstream according to all of
Depends, Imports, Suggests, and Extends. I'll call this the 'broad
dependency graph'.
4) Install all the packages in the broad dependency graph.
5) for each package in the broad graph, run CHECK with
R_CHECK_FORCE_SUGGESTS=true.
Then the affected packages are "checked". Perhaps this can be noted
with a signature.
.... Whew!
- Allen S. Rout
[1] http://cran.r-project.org/doc/manuals/R-exts.html#The-DESCRIPTION-file
[2] http://cran.r-project.org/doc/manuals/R-exts.html#Customizing-checking-an...
14 years, 5 months
r2spec
by Allen S. Rout
So, I've modified r2spec fairly heavily on my own. Pierre-Yves
suggested I bring conversation straight here, so I'm doing that.
I'll put a SRPM of my r2spec up at the next point when it works. It
worked earlier today, but of course I'm doing more surgery right now.
Here are the most visible changes I've made so far; Please hear them
as suggestions, not demands. :)
+ Takes R package metadata from repositories directly through
available.packages. Can load from many repos at once.
+ Cleanly handles multi-line fields in package metadata
+ Recursively chases Depends
+ including across repositories (i.e. CRAN depends on BioC)
+ Or Depends and Suggests, and Imports and Extends.
There's a lot more to this. Separate topic, separate post.
+ Uses a single template for specfile, instead of two long functions
with literal strings.
+ Permits template to be specified on the command line
+ String manipulation by regexp instead of iterated 'replace' and such.
+ Maintains a dependency graph to permit building an automated rpmbuild order.
[ toy additions ]
outputs GraphViz (dot format) illustrations of dependency graphs.
I know this is lots and lots of change. I hope you don't see it as
encroaching.
- Allen S. Rout
14 years, 5 months
Any feedback? any at all?
by Allen S. Rout
Hi, folks.
I've now got my script in such a form that I can generate, build, and
check most of the 'broad' dependency tree for ggplot2. There are
several package build errors which I haven't started running down; I'm
not sure if they're relevant to my case.
Do any of you have any interest in this? I was thinking it might help
accellerate new R packages' introduction to e.g. EPEL, but my sense
now is that you'd much rather hand-craft every one. If our goals are
just skew, I'm fine with that.
I've put nearly a work week into this over the last few months, and am
very near the end of what I need for my local concerns. I can happily
put a bit more time to address things you-all want done, if your
perspective is something like "We could use this, if he'd just [x, y,
z]". Getting stuff accepted 'upstream' is worthwhile in my work
environment.
But if you're never going to be interested in the mass maintenance of
these packages, I'll just stop bothering you.
- Allen S. Rout
14 years, 5 months
Should rpy require R or R-core?
by José Abílio Matos
Hi all,
I got a feature request in bugzilla where the user request that rpy requires
R-core and not R.
I don't have any good reason to choose one or the other, what do others
think?
Would anyone oppose to such a move?
Regards,
--
José Abílio
14 years, 6 months
Reflections on R...
by Allen S. Rout
I've completed the "OK, now try to CMD CHECK all these" script, and
seeing how well it functions.
If you want to be impressed with what you can do with R, all out of
proportion with what you thought was keen yesterday, run R CMD check
on rgl.
Dang.
- Allen S. Rout
14 years, 6 months
R2spec kibitzing.
by Allen S. Rout
Sorry, those messages were supposed to have come out periodically over
the last week or two.
I've got the next version ready, tested as I had alluded to in my previous.
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.1-asr.src.rpm
So, here's what I did to get more or less useful results out of it:
After installing, I made a temp directory, /var/tmp/Rstuff.
There, I ran (as root):
# clear; R2spec -n "Allen S. Rout" -e "asr(a)ufl.edu" -c --force --suggests --verbose --package ggplot2
This downloaded the CRAN-like lists of packages, downloaded source
tarballs, and built (and copied) the specfiles and source.
It also generated a few dot files, including the broad and narrow
dependency graphs for the requested package.
It also generates a shell file: 'rpmbuild.sh'. This is intended to
make all the RPMs you need for the broad graph.
I haven't decided what the sane options are in terms of 'pwd'
throughout all this process. At the moment, the next step is:
# cd /usr/src/redhat/SPECS
# sh /var/tmp/Rstuff/rpmbuild.sh
This will build and install all of the RPMs in the broad graph. ... If
the individual builds work.
This script uses a tempdir extensively, again I'm not sure what the
Right Way is to do this, I haven't thought about it too much yet.
It's statically defined at the beginning of the script to be
/var/tmp/Rstuff
that's obviously inappropriate for release, but will do for the
moment.
The script drops a build log and an install log for every package
addressed.
Right now, for me to get the sysdeps to build and install all (most)
of ggplot2's broad depgraph, I must:
yum install openmpi openmpi-devel openmpi-libs unixODBC-devel pvm tcl tcl-devel perl-DateManip perl-Date-Calc perl-version
and then install from EPEL:
environment-modules-3.2.6-4.el5.x86_64.rpm
mpich2-1.1.1-1.el5.x86_64.rpm
perl-IO-stringy-2.110-5.el5.noarch.rpm
perl-Jcode-2.06-6.el5.noarch.rpm
perl-OLE-Storage_Lite-0.14-9.el5.noarch.rpm
perl-Parse-RecDescent-1.96-1.el5.noarch.rpm
perl-Spreadsheet-WriteExcel-2.18-1.el5.noarch.rpm
perl-Unicode-Map-0.112-12.el5.x86_64.rpm
and then install from the graphviz people:
graphviz-2.24.0-1.el5.x86_64.rpm
graphviz-devel-2.24.0-1.el5.x86_64.rpm
With these things done, I still fail to build (and haven't
investigated much yet):
tcltk2: Fails to install: need tclsh8.3, not 8.4 ?
multcomp: requires Survival >= 2.35.7 ? ...
flexmix: dep on multcomp
R-RandomFields, R-MCMCpack: uncaught debug files in the build tree.
MPI, sprng
As threatened, my next step will be a similar shell script to R CMD
CHECK all the packages, and populate yet another family of logfiles.
Once that is complete, I will re-announce and go back to try to figure
out why individual packages fail.
- Allen S. Rout
14 years, 6 months
collaborating on ggplot2 ?
by Jack Tanner
Howdy,
I'd like to get the ggplot2 library package-able. I cannot sign up as a
maintainer, but I can work on spec files. ggplot2 has a whole bunch of
dependencies:
According to http://cran.r-project.org/web/packages/ggplot2/ ,
Imports: reshape
<http://cran.r-project.org/web/packages/reshape/index.html> (≥ 0.8.0),
plyr <http://cran.r-project.org/web/packages/plyr/index.html> (≥ 0.1.6),
splines, MASS <http://cran.r-project.org/web/packages/VR/index.html>,
RColorBrewer
<http://cran.r-project.org/web/packages/RColorBrewer/index.html>, grid,
proto <http://cran.r-project.org/web/packages/proto/index.html>
Suggests: quantreg
<http://cran.r-project.org/web/packages/quantreg/index.html>, Hmisc
<http://cran.r-project.org/web/packages/Hmisc/index.html>, mapproj
<http://cran.r-project.org/web/packages/mapproj/index.html>, maps
<http://cran.r-project.org/web/packages/maps/index.html>, digest
<http://cran.r-project.org/web/packages/digest/index.html>, colorspace
<http://cran.r-project.org/web/packages/colorspace/index.html>, hexbin
<http://cran.r-project.org/web/packages/hexbin/index.html>, gpclib
<http://cran.r-project.org/web/packages/gpclib/index.html>, maptools
<http://cran.r-project.org/web/packages/maptools/index.html>
Would anyone be interested in taking this on with me and writing spec
files for some of the ggplot2 prereqs?
14 years, 6 months
Introduction; was Re: collaborating on ggplot2 ?
by Allen S. Rout
Well, it didn't get through GMANE. So I'm re-sending directly.
Pierre-Yves <pingou-E11Oz7VxvVOXCRStZZN3OA(a)public.gmane.org> writes:
> Did you look at the subdependencies ? I mean that for example quantreg
> requires SparseM tripack, akima, MASS, survival, rgl, logspline
> (Which already points out a circular dependency on MASS)...
I'm also working on this problem, albeit from a slightly different
perspective.
I started out wanting to just get ggplot2 installed. :)
I found r2spec, and have been folding, spindling, and mutilating it
for somewhere under a month. Pierre-Yves, somewhere in the ether
between me and you are several messages along the lines of "I've got
some suggestions, will you entertain them?"
I've also been talking with other R developer types; I came up with a
similar chunk of code to make "Ebuilds" (Gentoo artifact mostly like a
specfile) a few years back.
So, Hi, there!
I've got several different topics queued up, I figure I'll wait to see
if this gets through GMANE before I start on them.
- Allen S. Rout
14 years, 6 months