[PATCH] Add "rpmbuild_networking" key (False by default) for nspawn backend

[Colin Walters]

> From 1726c9b75abad3361e6207d2ea4a8e8cb78c85c4 Mon Sep 17 00:00:00 2001
From: Colin Walters <walters(a)verbum.org&gt;
Date: Mon, 27 Jul 2015 21:35:20 -0400
Subject: [PATCH] Add "rpmbuild_networking" key (False by default) for nspawn
 backend

Build systems like Koji and many others want to support one aspect of
reproducible builds by ensuring all input artifacts are cached within
the organization.  This is the "If Github goes down, you can still
build" aspect.

Traditionally, Koji installations implement this via firewalling
rules on the server.

However, now with the systemd nspawn backend, we can rely
on much more robust container features.  In particular, systemd has
a `--private-network` option which does exactly what we want.

Teach mock how to use it, and enable this by default if the nspawn
backend is in use.

This has the potential to break mock users who were relying on
networking - but it's easy to turn back on.  But I assert that the
majority of people using mock are exactly the kind of user who want to
do full packaging, and it's best if they find out about any network
issues before they do a Koji/whatever build.
---
 etc/mock/site-defaults.cfg |  3 +++
 py/mockbuild/backend.py    |  2 ++
 py/mockbuild/util.py       | 13 +++++++++----
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/etc/mock/site-defaults.cfg b/etc/mock/site-defaults.cfg
index 4ce0392..355870c 100644
--- a/etc/mock/site-defaults.cfg
+++ b/etc/mock/site-defaults.cfg
@@ -54,6 +54,9 @@
 # By default Mock use simple chroot(1). When you set this to True
 # it will use systemd-nspawn(1)
 # config_opts['use_nspawn'] = False
+# If you're using nspawn, then by default networking will be turned off
+# for rpmbuild.  This helps ensure more reproducible builds.
+# config_opts['rpmbuild_networking'] = False
 
 # The default package manager is Yum
 # config_opts['package_manager'] = 'yum'
diff --git a/py/mockbuild/backend.py b/py/mockbuild/backend.py
index c342573..8a5e644 100644
--- a/py/mockbuild/backend.py
+++ b/py/mockbuild/backend.py
@@ -428,10 +428,12 @@ class Commands(object):
         command = [rpmbuild_cmd]
         if not util.USE_NSPAWN:
             command = ["bash", "--login", "-c"] + command
+        print "network: %s" % (not
self.config['rpmbuild_networking'],)
         self.buildroot.doChroot(command,
             shell=False, logger=self.buildroot.build_log, timeout=timeout,
             uid=self.buildroot.chrootuid, gid=self.buildroot.chrootgid,
             user=self.buildroot.chrootuser,
+            private_network=not self.config['rpmbuild_networking'],
             printOutput=self.config['print_main_output'])
         bd_out = self.make_chroot_path(self.buildroot.builddir)
         results = glob.glob(bd_out + '/RPMS/*.rpm')
diff --git a/py/mockbuild/util.py b/py/mockbuild/util.py
index 58b4f90..36b2188 100644
--- a/py/mockbuild/util.py
+++ b/py/mockbuild/util.py
@@ -420,7 +420,8 @@ def selinuxEnabled():
 @traceLog()
 def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True,
        returnOutput=0, uid=None, gid=None, user=None, personality=None,
-       printOutput=False, env=None, pty=False, *args, **kargs):
+       printOutput=False, env=None, pty=False, private_network=False,
+       *args, **kargs):
 
     logger = kargs.get("logger", getLog())
     output = ""
@@ -437,7 +438,7 @@ def do(command, shell=False, chrootPath=None, cwd=None, timeout=0,
raiseExc=True
             command = ['/bin/sh', '-c'] + command
             shell = False
         if chrootPath and USE_NSPAWN:
-            command = _prepare_nspawn_command(chrootPath, user, command)
+            command = _prepare_nspawn_command(chrootPath, user, command,
private_network=private_network)
         logger.debug("Executing command: {0} with env {1} and shell
{2}".format(command, env, shell))
         child = subprocess.Popen(
             command,
@@ -531,7 +532,7 @@ def is_in_dir(path, directory):
 
     return os.path.commonprefix([path, directory]) == directory
 
-def _prepare_nspawn_command(chrootPath, user, cmd):
+def _prepare_nspawn_command(chrootPath, user, cmd, private_network=False):
     cmd_is_list = isinstance(cmd, list)
     if user:
         # needs to be /bin because of el5 and el6 targets
@@ -541,7 +542,10 @@ def _prepare_nspawn_command(chrootPath, user, cmd):
             cmd = ['/bin/su', '-l', user, '-c',
'"{0}"'.format(cmd)]
     elif not cmd_is_list:
         cmd = [ cmd, ]
-    cmd = ['/usr/bin/systemd-nspawn', '-M' , uuid.uuid4().hex,
'-D', chrootPath] + cmd
+    nspawn_argv = ['/usr/bin/systemd-nspawn', '-M' , uuid.uuid4().hex,
'-D', chrootPath]
+    if private_network:
+        nspawn_argv.append('--private-network')
+    cmd = nspawn_argv + cmd
     if cmd_is_list:
         return cmd
     else:
@@ -629,6 +633,7 @@ def setup_default_config_opts(unprivUid, version, pkgpythondir):
     config_opts['state_log_fmt_name'] = "state"
     config_opts['online'] = True
     config_opts['use_nspawn'] = False
+    config_opts['rpmbuild_networking'] = False
 
     config_opts['internal_dev_setup'] = True
     config_opts['internal_setarch'] = True
-- 
1.8.3.1

[Miroslav Suchý]

Dne 28.7.2015 v 03:41 Colin Walters napsal(a):
>  From 1726c9b75abad3361e6207d2ea4a8e8cb78c85c4 Mon Sep 17 00:00:00
> 2001
>  From: Colin Walters <walters(a)verbum.org&gt;
>  Date: Mon, 27 Jul 2015 21:35:20 -0400
>  Subject: [PATCH] Add "rpmbuild_networking" key (False by default) for nspawn
>   backend
>  
>  Build systems like Koji and many others want to support one aspect of
>  reproducible builds by ensuring all input artifacts are cached within
>  the organization.  This is the "If Github goes down, you can still
>  build" aspect.
>  
>  Traditionally, Koji installations implement this via firewalling
>  rules on the server.
>  
>  However, now with the systemd nspawn backend, we can rely
>  on much more robust container features.  In particular, systemd has
>  a `--private-network` option which does exactly what we want.
>  
>  Teach mock how to use it, and enable this by default if the nspawn
>  backend is in use.
>  
>  This has the potential to break mock users who were relying on
>  networking - but it's easy to turn back on.  But I assert that the
>  majority of people using mock are exactly the kind of user who want to
>  do full packaging, and it's best if they find out about any network
>  issues before they do a Koji/whatever build. 

Greate idea. I like it.


>   # The default package manager is Yum
>   # config_opts['package_manager'] = 'yum'
>  diff --git a/py/mockbuild/backend.py b/py/mockbuild/backend.py
>  index c342573..8a5e644 100644
>  --- a/py/mockbuild/backend.py
>  +++ b/py/mockbuild/backend.py
>  @@ -428,10 +428,12 @@ class Commands(object):
>           command = [rpmbuild_cmd]
>           if not util.USE_NSPAWN:
>               command = ["bash", "--login", "-c"] + command
>  +        print "network: %s" % (not
> self.config['rpmbuild_networking'],) 
There is hardly any prints in current mock code. Can you use log() call?
Either self.buildroot.build_log.debug() or
  log = getLog()
  log.debug()
Choose yourself which one fits you better.


-- 
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys

[Colin Walters]

On Tue, Jul 28, 2015, at 04:19 AM, Miroslav Suchý wrote:
>  Greate idea. I like it. 
Thanks.  FWIW, this is how Gnome Continuous has worked for several years
now, although using https://git.gnome.org/browse/linux-user-chroot which
is designed to run completely as non-root instead of nspawn.

>  There is hardly any prints in current mock code. Can you use log()
> call?
>  Either self.buildroot.build_log.debug() or
>    log = getLog()
>    log.debug()
>  Choose yourself which one fits you better. 
This was a leftover debug statement; I can resend without it, but
if you're happy with the patch as is, please apply without it, or
if you see any other issues, let me know and I'll fix those and
send an updated version then.

[Miroslav Suchý]

Dne 28.7.2015 v 15:35 Colin Walters napsal(a):
>  This was a leftover debug statement; I can resend without it, but
>  if you're happy with the patch as is, please apply without it, or
>  if you see any other issues, let me know and I'll fix those and
>  send an updated version then. 
Applied as be4b13a.
Thank you.

-- 
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys