Dealing with cyclic dependencies

[Bryan Chan]

As I mentioned at last Friday's meeting, we have tried building EPEL 7
packages on s390x, using a simple script that runs mock on all the
SRPMs, and populating a local yum repo with the generated binary RPMs.
SRPMs that fail to build due to missing dependencies will often succeed
on the second pass, when they find the necessary dependencies in the
local yum repo. However, RPMs that have cyclic dependencies cannot be
built this way, when none of the RPMs in the cycle currently exists for
a new architecture.

I suppose one could break the cycle by installing dependencies from
source, and then building one of the RPMs with rpmbuild -bb --nodeps,
outside of mock. Once the cycle is broken, the remaining packages
should be possible to build with mock.

I just wonder if any one knows of a better way to deal with this.

Bryan

[Dan Horák]

On Tue, 8 Sep 2015 13:02:41 -0400
"Bryan Chan" <bryan.chan(a)ca.ibm.com&gt; wrote:

>  
>  As I mentioned at last Friday's meeting, we have tried building EPEL 7
>  packages on s390x, using a simple script that runs mock on all the
>  SRPMs, and populating a local yum repo with the generated binary RPMs.
>  SRPMs that fail to build due to missing dependencies will often
>  succeed on the second pass, when they find the necessary dependencies
>  in the local yum repo. However, RPMs that have cyclic dependencies
>  cannot be built this way, when none of the RPMs in the cycle
>  currently exists for a new architecture.
>  
>  I suppose one could break the cycle by installing dependencies from
>  source, and then building one of the RPMs with rpmbuild -bb --nodeps,
>  outside of mock. Once the cycle is broken, the remaining packages
>  should be possible to build with mock.
>  
>  I just wonder if any one knows of a better way to deal with this. 
Often those packages contain a "bootstrap" macro that can be set to 1
for an initial build with limited dependencies and to 0 for production
build. And for building for EPEL there should be a older build (source
rpm) available (because it had to bootstrapped also for regular EPEL),
so first rebuild this, and then the latest, or next or whatever is
necessary. The changelog in the spec files and/or history in dist-git
will show the details about the versions.

Do you have a list of those problematic builds?


		Dan

[Bryan Chan]

Dan Horák <dan(a)danny.cz&gt; wrote on 2015-09-08 01:56:54 PM:

>  Often those packages contain a "bootstrap" macro that can
> be set to 1
>  for an initial build with limited dependencies and to 0 for production
>  build. And for building for EPEL there should be a older build (source
>  rpm) available (because it had to bootstrapped also for regular EPEL),
>  so first rebuild this, and then the latest, or next or whatever is
>  necessary. The changelog in the spec files and/or history in dist-git
>  will show the details about the versions.
>
>  Do you have a list of those problematic builds? 
Hi Dan, thanks for the suggestion. My team mate has found a similar
bootstrap macro in the ghc packages and is trying it out now.

I asked the person who did the original runs back in June but he doesn't
remember where the circular dependencies came from. He thought it might
be some Python- or Perl-related dependencies in a %test section of a
package but he isn't sure. We will watch out for the issue and report
if we see it again.

Thanks, Bryan

[Dan Horák]

On Thu, 10 Sep 2015 03:00:17 -0400
"Bryan Chan" <bryan.chan(a)ca.ibm.com&gt; wrote:

>  
>  Dan Horák <dan(a)danny.cz&gt; wrote on 2015-09-08 01:56:54 PM:
>  
>  > Often those packages contain a "bootstrap" macro that can be set to
>  > 1 for an initial build with limited dependencies and to 0 for
>  > production build. And for building for EPEL there should be a older
>  > build (source rpm) available (because it had to bootstrapped also
>  > for regular EPEL), so first rebuild this, and then the latest, or
>  > next or whatever is necessary. The changelog in the spec files
>  > and/or history in dist-git will show the details about the versions.
>  >
>  > Do you have a list of those problematic builds?
>  
>  Hi Dan, thanks for the suggestion. My team mate has found a similar
>  bootstrap macro in the ghc packages and is trying it out now. 
for ghc specifically I would recommend to contact Jens Petersen (juhp
on #fedora-devel IRC) directly, he is the ghc maintainer in Fedora and
knows best all the weirdness in ghc packaging.

>  I asked the person who did the original runs back in June but he
>  doesn't remember where the circular dependencies came from. He
>  thought it might be some Python- or Perl-related dependencies in a %
>  test section of a package but he isn't sure. We will watch out for
>  the issue and report if we see it again. 
yeah, tests in perl-* are possible candidate, let us know and I guess we
can help


		Dan

[Stephen John Smoogen]

On 8 September 2015 at 11:02, Bryan Chan <bryan.chan(a)ca.ibm.com&gt; wrote:
>  As I mentioned at last Friday's meeting, we have tried building
> EPEL 7
>  packages on s390x, using a simple script that runs mock on all the
>  SRPMs, and populating a local yum repo with the generated binary RPMs.
>  SRPMs that fail to build due to missing dependencies will often succeed
>  on the second pass, when they find the necessary dependencies in the
>  local yum repo. However, RPMs that have cyclic dependencies cannot be
>  built this way, when none of the RPMs in the cycle currently exists for
>  a new architecture.
>
>  I suppose one could break the cycle by installing dependencies from
>  source, and then building one of the RPMs with rpmbuild -bb --nodeps,
>  outside of mock. Once the cycle is broken, the remaining packages
>  should be possible to build with mock.
>
>  I just wonder if any one knows of a better way to deal with this.
>  
Normally you use 1-2 side repos where you put your packages.

Repo 1: All the package for s390 f17 (for example)
Repo 2: All the packages as they are built by your rebuild script.

As the packages in repo 1 are usually lesser EVR than the f18/f19 you
are wanting to rebuild you can usually meet the minimum for a circular
dependency. If you want to really get fancy you can remove packages
from Repo1 as they are rebuilt in Repo2 and then have a Repo3 which is
the rebuild of the rebuilds.

[Somewhere along the way you run out of turtles on the way down...]

>  Bryan
>
>
>  _______________________________________________
>  epel-devel mailing list
>  epel-devel(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/epel-devel
>  


-- 
Stephen J Smoogen.

[Bryan Chan]

Stephen John Smoogen <smooge(a)gmail.com&gt; wrote on 2015-09-08 04:48:09 PM:

>  Normally you use 1-2 side repos where you put your packages.
>
>  Repo 1: All the package for s390 f17 (for example)
>  Repo 2: All the packages as they are built by your rebuild script.
>
>  As the packages in repo 1 are usually lesser EVR than the f18/f19 you
>  are wanting to rebuild you can usually meet the minimum for a circular
>  dependency. If you want to really get fancy you can remove packages
>  from Repo1 as they are rebuilt in Repo2 and then have a Repo3 which is
>  the rebuild of the rebuilds. 
We used RHEL 7.1 as our read-only "repo 1". Using a Fedora repo might
have helped us avoid some dependency issues... I hadn't thought of that.

Thanks for the suggestions.

Bryan