[PATCH] Added a brute-force command to do a submodule-update after branch-checkout.

[Martin Wilhelm]

Reason:
1) We use mock to build a RPM. Mock does all the SCM-stuff via its plugin.
2) This plugin actually does NOT do a "git submodule update --init --recursive"
after the branch checkout.
3) This leads to wrong (old) submodule references.
4) Compile error.
---
 py/mockbuild/scm.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/py/mockbuild/scm.py b/py/mockbuild/scm.py
index e76293d..dc67bf0 100644
--- a/py/mockbuild/scm.py
+++ b/py/mockbuild/scm.py
@@ -39,6 +39,7 @@ class scmWorker(object):
 
         self.branch = None
         self.postget = None
+        self.afterpostget = None
         if 'branch' in opts:
             self.branch = opts['branch']
         if self.branch:
@@ -46,6 +47,7 @@ class scmWorker(object):
                 self.get = self.get.replace("SCM_BRN", "-r " +
self.branch)
             elif self.method == "git":
                 self.postget = "git checkout " + self.branch
+                self.afterpostget = "git submodule update --init --recursive"
             elif self.method == "svn":
                 self.get = self.get.replace("SCM_BRN", self.branch)
             else:
@@ -73,6 +75,7 @@ class scmWorker(object):
 
         self.log.debug("SCM checkout command: " + self.get)
         self.log.debug("SCM checkout post command: " + str(self.postget))
+        self.log.debug("SCM checkout after post command: " +
str(self.afterpostget))
         self.environ = os.environ.copy()
         # Set HOME properly while checking out from SCM since tools like
         # Subversion might have there settings needed to carry out checkout
@@ -90,6 +93,8 @@ class scmWorker(object):
         mockbuild.util.do(shlex.split(self.get), shell=False, cwd=self.wrk_dir,
env=self.environ)
         if self.postget:
             mockbuild.util.do(shlex.split(self.postget), shell=False, cwd=self.src_dir,
env=self.environ)
+        if self.afterpostget:
+            mockbuild.util.do(shlex.split(self.afterpostget), shell=False,
cwd=self.src_dir, env=self.environ)
         self.log.debug("Fetched sources from SCM")
 
     decorate(traceLog())
-- 
1.9.1

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 16 Sep 2014 16:23:40 +0200
Martin Wilhelm <martin(a)system4.org&gt; wrote:

>  Reason:
>  1) We use mock to build a RPM. Mock does all the SCM-stuff via its
>  plugin. 2) This plugin actually does NOT do a "git submodule update
>  --init --recursive" after the branch checkout. 3) This leads to wrong
>  (old) submodule references. 4) Compile error.
>  
Can you please explain this a little more. I am really not sure what
this is trying to do or why.  and what effect if will have on users not
using the functionality you're using.

Dennis
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBAgAGBQJUIvgVAAoJEH7ltONmPFDR1awQAJABBMa9BJBmTGgCWOrBR0Uk
> iaB1oamNbAQTE85v2vd70P5qGbHNDddyLGEu8l5yrTaBNKY2mmUgns3f7bcXjZU0
> OeA2OLU4WtuV4ZuELTTHLe1NNSVqUWi352363FVpgoscn6b7/1H1QaLwR2lu088H
> /hQFWzhCHWdhj0nEr03uT+/O6CKSgU4Wz4w5ka0Sx7jGjTjWXPYolYBcaPMmo4NT
> bgeBYp3PhjEqgw/ItlAiVqulkqzC+OEUS3Bx4OeiqGSxSh0bWMy4e8NrO0PVsvos
> Hed06qOJZ6KWosGPYKOHoeIz0nbeT7PWztK+93+39y8deegxhSaVOsGhLzsFC23m
> doJNBmCPriYLyvznVfXe5T8RtH19FeFVSqf942/uw74HWh1e76QFgwsJLd4NnZLc
> 21cAP2H4Vjr7uMbfYQZ1HUgCnS+CoFP3gehm997YwdK55MR+YR+je9u/icxspUVz
> 46RV1bAoNdo8Ec2zvzDvl1EtN1z6iAdJoyX9RKujJBx6CYu7dQV46SRmrS9p6rFu
> T/uE79K1x/233M8uWFQ6bd5JR114IICxno/i7S+1VPTR5HbbsegDJUwIHaDdSvNy
> X7ivwIAPSScZZllB0tZa/Nda5NsYeKQ3LNtXDftQnlvrkeIoZJtC1bfLokM46bXP
> OKWMyQC+srW1SuBncd08
> =+DLU
> -----END PGP SIGNATURE-----
>

[Martin Wilhelm]

Hi,

On 2014-09-24 18:57, Dennis Gilmore wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>  On Tue, 16 Sep 2014 16:23:40 +0200
>  Martin Wilhelm <martin(a)system4.org&gt; wrote:
>
> > Reason:
> > 1) We use mock to build a RPM. Mock does all the SCM-stuff via its
> > plugin. 2) This plugin actually does NOT do a "git submodule update
> > --init --recursive" after the branch checkout. 3) This leads to wrong
> > (old) submodule references. 4) Compile error.
>  Can you please explain this a little more. I am really not sure what
>  this is trying to do or why.  and what effect if will have on users not
>  using the functionality you're using.
>
>  Dennis
> >  -----BEGIN PGP SIGNATURE-----
> >  Version: GnuPG v2
> >
> >  iQIcBAEBAgAGBQJUIvgVAAoJEH7ltONmPFDR1awQAJABBMa9BJBmTGgCWOrBR0Uk
> >  iaB1oamNbAQTE85v2vd70P5qGbHNDddyLGEu8l5yrTaBNKY2mmUgns3f7bcXjZU0
> >  OeA2OLU4WtuV4ZuELTTHLe1NNSVqUWi352363FVpgoscn6b7/1H1QaLwR2lu088H
> >  /hQFWzhCHWdhj0nEr03uT+/O6CKSgU4Wz4w5ka0Sx7jGjTjWXPYolYBcaPMmo4NT
> >  bgeBYp3PhjEqgw/ItlAiVqulkqzC+OEUS3Bx4OeiqGSxSh0bWMy4e8NrO0PVsvos
> >  Hed06qOJZ6KWosGPYKOHoeIz0nbeT7PWztK+93+39y8deegxhSaVOsGhLzsFC23m
> >  doJNBmCPriYLyvznVfXe5T8RtH19FeFVSqf942/uw74HWh1e76QFgwsJLd4NnZLc
> >  21cAP2H4Vjr7uMbfYQZ1HUgCnS+CoFP3gehm997YwdK55MR+YR+je9u/icxspUVz
> >  46RV1bAoNdo8Ec2zvzDvl1EtN1z6iAdJoyX9RKujJBx6CYu7dQV46SRmrS9p6rFu
> >  T/uE79K1x/233M8uWFQ6bd5JR114IICxno/i7S+1VPTR5HbbsegDJUwIHaDdSvNy
> >  X7ivwIAPSScZZllB0tZa/Nda5NsYeKQ3LNtXDftQnlvrkeIoZJtC1bfLokM46bXP
> >  OKWMyQC+srW1SuBncd08
> >  =+DLU
> >  -----END PGP SIGNATURE----- 
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>  
sorry for the long delay - here is a more in-detail explanation:

Actually, we use mock + mock-scm to build a rpm out of some projects
hosted in git. An example looks like:


mock --resultdir $(PWD)/RPMS/ -v --scm-enable \
         --scm-option method=git \
         --scm-option package=$(SYSTEM) \
         --scm-option git_get='git clone --recursive $(GIT_URL) $(SYSTEM)' \
         --scm-option spec=./$(SYSTEM).spec \
         --scm-option branch=$(GIT_BRANCH) \
         --scm-option exclude_vcs=False \
         --scm-option write_tar=True $(RPM_DEFINES)


Doing so, a "git clone --recursive $(GIT_URL)" gets triggered
as specified with --scm-option git_get.

We also switch the branch to something non-master, done
with --scm-option branch=$(GIT_BRANCH). Inside the mock-scm code
you can see, that this would trigger:


        if self.branch:
             [...]
             elif self.method == "git":
                 self.postget = "git checkout " + self.branch"


Means: After a "git clone --recursive" we do a "git checkout branch".
This is unfortunately not enough when having a git-project with
*branched* submodules.
Means: Every submodule could have different branches beside of "master".
The root-project points to some of these branches (using the according
git-hash).

But: That can only work if you issue a "git submodule update --init
--recursive" on your checkout after the mentioned "git checkout branch"
or the root-project will point to undefined git-hashes.

Thats why I extended the self.postget with a second command which gets
executed after "git checkout branch".

Regarding the effect on other users:
"git submodule update --init --recursive" gets triggered only if
you use the option "--scm-option branch".
And: If there are no submodules or submodules with no branches except
master, git simply does nothing (exit 0). That means: Nothing breaks,
same behavior as before.

For now, the little patch fixed some issues at least for us.

Anyway - it could make sense to add another parameter to mock to
enable/disable that behavior / or to add a "--scm-option git_postget"
to set some arbitrary extra commands to be executed lastly instead of
hardcoding these.

What do you think? Shall we go with a new option like "--scm-option
git_postget"?



Martin


-- 
     martin wilhelm      /-/ system4
    martin(a)system4.org  /-/ http://www.system4.org
                       /-/ info(a)system4.org

[Martin Wilhelm]

On 2014-10-07 12:34, Martin Wilhelm wrote:

>  Anyway - it could make sense to add another parameter to mock to
>  enable/disable that behavior / or to add a "--scm-option git_postget"
>  to set some arbitrary extra commands to be executed lastly instead of
>  hardcoding these. 
I made this little change now - patch comes ASAP.


Martin