Catkin.. *sigh*

[Ankur Sinha]

Hi,

So I tried to build rpms for groovy by downloading the entire source at
once and letting catkin make it all, as documented here[1]. It doesn't
work. I'm not surprised. A few things:

1. The --install-space /opt/... suggestion won't work with mock, since
catkin uses this in all the setup scripts. So, the setup.sh files have
$RPM_BUILD_ROOT in them. I'm considering filing a bug requesting them to
provide a different --root option (like every other build system does!)

This link will only work when building manually with root permissions.
It isn't going to work for a system wide installation, and it certainly
cannot be converted to a spec file.

2. Seeing that this method isn't going to work, I dug up on catkin a
little more and found that one can actually bootstrap it all from
source[2][3]. I'll give this a try today and tomorrow and see if I can
"spec-ify" it.

The catkin package in Fedora appears to be missing some stuff though:
like the catkin_init_workspace script. Rich, is this a matter of just
updating the Fedora catkin package? Should I file a bug? We're carrying
0.4x and the latest seems to be 0.5+

I'll see if I can get some help on the build-sys SIG about extending
bloom to spew rpms. I think this needs to be dealt with once and for
all.

[1] http://www.ros.org/wiki/groovy/Installation/Fedora
[2]
http://answers.ros.org/question/44606/bootstrapping-a-groovy-install/
[3] http://ros.org/doc/api/catkin/html/adv_user_guide/underlay.html

[Rich Mattes]

On Wed, Jul 24, 2013 at 4:04 AM, Ankur Sinha  wrote:
> Hi,
>
> So I tried to build rpms for groovy by downloading the entire source at
> once and letting catkin make it all, as documented here[1]. It doesn't
> work. I'm not surprised. A few things:
>
> Are you targeting /usr or /opt as your installation point?
> The --install-space /opt/... suggestion won't work with mock, since
>
> catkin uses this in all the setup scripts. So, the setup.sh files have
> $RPM_BUILD_ROOT in them. I'm considering filing a bug requesting them to
> provide a different --root option (like every other build system does!)
>
> This link will only work when building manually with root permissions.
> It isn't going to work for a system wide installation, and it certainly
> cannot be converted to a spec file.

I did at one point get some of the groovy packages spec-ified and built as
RPMs installing to /usr, including catkin.  I will try to dig up the specs
and upload them to my fedorapeople account.  I don't remember many of the
details off the top of my head, but I will let you know when I reacquaint
myself with that effort.
> Seeing that this method isn't going to work, I dug up on catkin a
>
> little more and found that one can actually bootstrap it all from
> source[2][3]. I'll give this a try today and tomorrow and see if I can
> "spec-ify" it.
>
> The catkin package in Fedora appears to be missing some stuff though:
> like the catkin_init_workspace script. Rich, is this a matter of just
> updating the Fedora catkin package? Should I file a bug? We're carrying
> 0.4x and the latest seems to be 0.5+
>
> The version of catkin does seem to be tied to the ROS release, at least

for fuerte and groovy.  So yes, we will need to update to catkin 0.5 when
we want to shift to groovy.  My plan was to do this for f20 and leave f19
and f18 on fuerte, and then put hydro on f21 after it's had a little while
to stabilize.

I guess I was holding off on upgrading f20 until all of the fuerte base
packages are done, but we can start working on it on the side without
pushing any builds.
> I'll see if I can get some help on the build-sys SIG about extending
> bloom to spew rpms. I think this needs to be dealt with once and for
> all.

Sounds good.  I've been following that list for a little while, I'll chip
in where I can.

Rich

[Ankur Sinha]

On Wed, 2013-07-24 at 08:55 -0400, Rich Mattes wrote:
> On Wed, Jul 24, 2013 at 4:04 AM, Ankur Sinha 
> wrote:
>         Hi,
>     So I tried to build rpms for groovy by downloading the entire
>     source at
>     once and letting catkin make it all, as documented here[1]. It
>     doesn't
>     work. I'm not surprised. A few things:
>
> Are you targeting /usr or /opt as your installation point?

I just followed the docs blindly, so /opt.
>     1. The --install-space /opt/... suggestion won't work with
>     mock, since
>     catkin uses this in all the setup scripts. So, the setup.sh
>     files have
>     $RPM_BUILD_ROOT in them. I'm considering filing a bug
>     requesting them to
>     provide a different --root option (like every other build
>     system does!)
>
>     This link will only work when building manually with root
>     permissions.
>     It isn't going to work for a system wide installation, and it
>     certainly
>     cannot be converted to a spec file.
>
> I did at one point get some of the groovy packages spec-ified and
> built as RPMs installing to /usr, including catkin.  I will try to dig
> up the specs and upload them to my fedorapeople account.  I don't
> remember many of the details off the top of my head, but I will let
> you know when I reacquaint myself with that effort.

That'd be great. :)
>     2. Seeing that this method isn't going to work, I dug up on
>     catkin a
>     little more and found that one can actually bootstrap it all
>     from
>     source[2][3]. I'll give this a try today and tomorrow and see
>     if I can
>     "spec-ify" it.
>
>     The catkin package in Fedora appears to be missing some stuff
>     though:
>     like the catkin_init_workspace script. Rich, is this a matter
>     of just
>     updating the Fedora catkin package? Should I file a bug? We're
>     carrying
>     0.4x and the latest seems to be 0.5+
>
> The version of catkin does seem to be tied to the ROS release, at
> least for fuerte and groovy.  So yes, we will need to update to catkin
> 0.5 when we want to shift to groovy.  My plan was to do this for f20
> and leave f19 and f18 on fuerte, and then put hydro on f21 after it's
> had a little while to stabilize.

Maybe it's time to start tracking groovy or hydro in a fedorapeople
repo? That way, we don't pollute the official fedora repos with our
"experiments"? It'll also make it clear that any files in the
fedorapeople repo are completely in alpha state and may or may not work
with official fedora repos?
> I guess I was holding off on upgrading f20 until all of the fuerte
> base packages are done, but we can start working on it on the side
> without pushing any builds.
>     I'll see if I can get some help on the build-sys SIG about
>     extending
>     bloom to spew rpms. I think this needs to be dealt with once
>     and for
>     all.
>
> Sounds good.  I've been following that list for a little while, I'll
> chip in where I can.

Since posting, I've been reading on bloom, catkin and catkin_pkg.
catkin_pkg is supposed to be a stand alone catkin python API and I
expected catkin's methods available there to let me get info from the
package.xml files. Doesn't seem to be the case though. (I was trying to
see if I can find enough API to hack a packagexml2spec script since the
package.xml file should contain all the dep info we need (or so I
think)). Bloom is slightly more complex since it's really *tightly*
integrated with git. Not sure how well it'll fit with just generating
rpms, in mock etc. I'm still looking for docs on deploying bloom and
using it to generate anything myself to get a better hang of how it
works.

There's so much random documentation that I kinda keep getting confused.
Lately, I found this which makes it feel like we can build each package
from source manually:

http://ros.org/doc/api/catkin/html/adv_user_guide/underlay.html

[Rich Mattes]

On 07/24/2013 10:06 AM, Ankur Sinha wrote:
> That'd be great. :)

See [1].  I know catkin builds, and I'll see if I can get
> Maybe it's time to start tracking groovy or hydro in a fedorapeople
> repo? That way, we don't pollute the official fedora repos with our
> "experiments"? It'll also make it clear that any files in the
> fedorapeople repo are completely in alpha state and may or may not work
> with official fedora repos?

I kinda started doing that at one point at [2].  I did start tracking 
fuerte in a copr, and managed to keep the packages so that they don't 
interfere with the core packages [3].  I started generating groovy 
packages as well, but never got as far as setting up a copr.

I like coprs more than repos.fp.o because they handle doing all of the 
mock builds support chain builds by feeding previously buildt packages 
into the next build in the chain.  I think we should file a bug to see 
if we can get multiple people access to kick off builds, and then figure 
out a common space to upload the SRPMs to build so that more than one 
person can work on it.
> Since posting, I've been reading on bloom, catkin and catkin_pkg.
> catkin_pkg is supposed to be a stand alone catkin python API and I
> expected catkin's methods available there to let me get info from the
> package.xml files. Doesn't seem to be the case though. (I was trying to
> see if I can find enough API to hack a packagexml2spec script since the
> package.xml file should contain all the dep info we need (or so I
> think)). Bloom is slightly more complex since it's really *tightly*
> integrated with git. Not sure how well it'll fit with just generating
> rpms, in mock etc. I'm still looking for docs on deploying bloom and
> using it to generate anything myself to get a better hang of how it
> works.
>
> There's so much random documentation that I kinda keep getting confused.
> Lately, I found this which makes it feel like we can build each package
> from source manually:
>
> http://ros.org/doc/api/catkin/html/adv_user_guide/underlay.html

I think building each package manually isn't too hard, they're all 
basically cmake projects.  It'd be really nice to have something like 
bloom handle the annoying details of actually generating a spec 
(especially for the case where it's just generating one monster package 
to drop in /opt like how the ROS debs currently work).  It would be much 
harder to get a tool like bloom to create proper -devel subpackages, 
give each stack its own package, etc., and there's enough FHS issues 
with default ROS packages that they all pretty much need manual 
intervention before being suitable for Fedora proper.

Rich

[1] http://rmattes.fedorapeople.org/groovypackages/catkin/
[2] http://rmattes.fedorapeople.org/ros-groovy/
[3] http://rmattes.fedorapeople.org/ros-fuerte/SRPMS/

Software collections [Was: Re: Catkin.. *sigh*]

[Ankur Sinha]

On Sat, 2013-07-27 at 19:29 -0400, Rich Mattes wrote:
> and there's enough FHS issues 
> with default ROS packages that they all pretty much need manual 
> intervention before being suitable for Fedora proper.

What about using Software Collections[1,2] in the mean time, until all
of ROS is FHS compliant? Even after it does become compliant, we still
have the issue of having multiple ROS versions available.

Software collections seem to permit multiple versions of the same
software, so we can have the multiple supported versions of ROS for
Fedora in a third party repository.

At a glance, I think ROS fits software collections really well. We can
have *one* version of ROS supported in Fedora officially, say Fuerte (if
we manage to pull this off, that is), and have the other versions
in /opt as collections. Users can then install all versions of ROS and
use what they need, the way ROS upstream supports *buntu.

I haven't worked with software collections before. I just ran into a
mail on the packaging SIG list and decided to check it out.

BTW: I've started working on bloom for rpms. Just started hacking
though. Hopefully I'll have something up in a bit. [3]

[1] https://fedoraproject.org/wiki/SoftwareCollections
[2]
https://fedoraproject.org/wiki/Packaging:Guidelines#Software_Collection_Macr...
[3] https://github.com/sanjayankur31/bloom