Running FreeIPA server containers on Ubuntu docker setups / Travis CI

[Jan Pazdziora]

Hello,

I and Stanislav have been setting up Travis CI for the

	https://github.com/freeipa/freeipa-container

repository. We now run docker build operations for pull requests
and commits, and I've been trying to add docker run tests as well.
Example Travis build

	https://travis-ci.org/adelton/freeipa-container/builds/400275700

from

	https://github.com/adelton/freeipa-container/commit/16ced918f8492e583ff02...

seems to pass on CentOS and rawhide images but fails on other Fedoras.
All the failures are in the

	Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
	  [1/28]: configuring certificate server instance
	  [error] RuntimeError: CA configuration failed.

step. I remember that due to the way systemd was handling KEYRING
operations, seccomp:unconfined was one of the things that has been
causing issues in our tests on Fedora hosts. Running

	docker info

in Travis CI shows that it's running 17.09.0-ce and there are no
Security Options listed, unlike on my Fedora 28's
docker-1.13.1-59.gitaf6b32b.fc28.x86_64 which has

	Security Options:
	 seccomp
	  Profile: /etc/docker/seccomp.json
	 selinux

Are there people around with Ubuntu knowledge who could check what
the docker 17.09.0-ce behaviour on that system is and if seccomp can
somehow be tweaked?

Thank you,

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat

[Alexander Bokovoy]

On to, 05 heinä 2018, Jan Pazdziora via FreeIPA-users wrote:
> Hello,
>
> I and Stanislav have been setting up Travis CI for the
>
> 	https://github.com/freeipa/freeipa-container
>
> repository. We now run docker build operations for pull requests
> and commits, and I've been trying to add docker run tests as well.
> Example Travis build
>
> 	https://travis-ci.org/adelton/freeipa-container/builds/400275700
>
> from
>
> 	https://github.com/adelton/freeipa-container/commit/16ced918f8492e583ff02...
>
> seems to pass on CentOS and rawhide images but fails on other Fedoras.
> All the failures are in the
>
> 	Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
> 	  [1/28]: configuring certificate server instance
> 	  [error] RuntimeError: CA configuration failed.
>
> step. I remember that due to the way systemd was handling KEYRING
> operations, seccomp:unconfined was one of the things that has been
> causing issues in our tests on Fedora hosts. Running
>
> 	docker info
>
> in Travis CI shows that it's running 17.09.0-ce and there are no
> Security Options listed, unlike on my Fedora 28's
> docker-1.13.1-59.gitaf6b32b.fc28.x86_64 which has
>
> 	Security Options:
> 	 seccomp
> 	  Profile: /etc/docker/seccomp.json
> 	 selinux
>
> Are there people around with Ubuntu knowledge who could check what
> the docker 17.09.0-ce behaviour on that system is and if seccomp can
> somehow be tweaked? 
I cannot give you an answer on Ubuntu parts but may be you can
instead
remove use of KEYRING ccache from krb5.conf, thus removing the reason
for seccomp modification?

-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Jan Pazdziora]

On Thu, Jul 05, 2018 at 11:20:57AM +0300, Alexander Bokovoy wrote:
>  > the docker 17.09.0-ce behaviour on that system is and if seccomp
> can
>  > somehow be tweaked?
>  I cannot give you an answer on Ubuntu parts but may be you can instead
>  remove use of KEYRING ccache from krb5.conf, thus removing the reason
>  for seccomp modification? 
This is not about Kerberos caches using kernel keyring.

This is about session keyring created by systemd:

	https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1691096
	https://github.com/systemd/systemd/issues/6281
	https://bugzilla.redhat.com/show_bug.cgi?id=1492081

Presumably it assumed that if it can create the keyring, it will be
able to chown it as well, leading to

	systemd[593]: pki-tomcatd(a)pki-tomcat.service: Failed at step KEYRING spawning
/usr/sbin/pki-server: Permission denied

On Fedora and RHEL with docker from distribution/Extras, docker is
built and configured to use seccomp, and it prevents it from being
able to being able to create the keyring in the first place, so things
do not fail.

I can reproduce the problem on Fedora for example with

	docker run --security-opt=seccomp=unconfined --rm -ti -e DEBUG_NO_EXIT=true -e
PASSWORD=Secret123 -h ipa.example.test freeipa/freeipa-server:fedora-27 -U -r EXAMPLE.TEST
--no-ntp

-- the seccomp=unconfined allows the first operation but due to
lack of CAP_CHOWN, the second one then fails.

I believe Ubuntu used by Travis CI and the docker build that is there
do not support seccomp, so things run similar to seccomp=unconfined
on Fedora.

Container run from the centos-7 image does not fail because systemd there
did not have that logic yet, the fedora-rawhide image does not fail
because the logic in systemd has been fixed since.

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat

[Jan Pazdziora]

On Thu, Jul 05, 2018 at 02:14:20PM +0200, Jan Pazdziora wrote:
>  
>  I can reproduce the problem on Fedora for example with
>  
>  	docker run --security-opt=seccomp=unconfined --rm -ti -e DEBUG_NO_EXIT=true -e
> PASSWORD=Secret123 -h ipa.example.test freeipa/freeipa-server:fedora-27 -U -r EXAMPLE.TEST
> --no-ntp
>  
>  -- the seccomp=unconfined allows the first operation but due to
>  lack of CAP_CHOWN, the second one then fails. 
I can "fix" this on Fedora with

	--privileged --security-opt=seccomp=unconfined

but I don't consider it a good solution for obvious reasons.

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat

[Jan Pazdziora]

On Thu, Jul 05, 2018 at 02:20:48PM +0200, Jan Pazdziora via FreeIPA-users wrote:
>  On Thu, Jul 05, 2018 at 02:14:20PM +0200, Jan Pazdziora wrote:
>  > 
>  > I can reproduce the problem on Fedora for example with
>  > 
>  > 	docker run --security-opt=seccomp=unconfined --rm -ti -e DEBUG_NO_EXIT=true -e
> PASSWORD=Secret123 -h ipa.example.test freeipa/freeipa-server:fedora-27 -U -r EXAMPLE.TEST
> --no-ntp
>  > 
>  > -- the seccomp=unconfined allows the first operation but due to
>  > lack of CAP_CHOWN, the second one then fails.
>  
>  I can "fix" this on Fedora with
>  
>  	--privileged --security-opt=seccomp=unconfined
>  
>  but I don't consider it a good solution for obvious reasons. 
For the record, I went with --privileged in Travis CI for OSes that
need it:

	https://github.com/freeipa/freeipa-container/blob/master/.travis.yml#L34

CentOS 7 and Fedora rawhide do not, so we are testing at least
something unprivileged.

-- 
Jan Pazdziora
Senior Principal Software Engineer, Security Engineering, Red Hat