8:15 a.m.
I'm trying to follow the following guide:
https://fedoraproject.org/wiki/CI/Tests#Wrapping
Let's say I've done the following:
$ fedpkgclone gzip
$ cd gzip/tests/
Now I want to run the tests in Docker. The guide says:
Try running this example test against an Atomic Host or Docker
Container. It should pass.
"Docker Container" leads to:
https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Container
So I've done:
$ export TEST_SUBJECTS=docker:docker.io/library/fedora:26
$ ansible-playbook --tags=container tests.yml
I see a lot of cows (cute), yet the error is:
"Destination /usr/local/bin not writable"
It's also amazingly fast, so I think no Fedora 26 docker image is being
pulled in. I don't think docker is invoked here at all.
What piece am I missing?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
5:55 a.m.
Miro, Hi!
Please check how you run tests.
From the snippet I see that you run as ordinary user. ansible-playbook must be run as
root.
More logs also would be fine.
Thank you.
10:31 a.m.
On 23.4.2018 12:55, Andrei Stepanov wrote:
Miro, Hi!
Hi Andrei, thanks for your answer.
Please check how you run tests.
From the snippet I see that you run as ordinary user. ansible-playbook must be run as
root.
I run the tests as a regular user, I want them to be executed in a
container, being root inside. That should not require me to run it as root.
More logs also would be fine.
The logs are not helpful, because they indicate the problem: I'm not
root. Yet I've attached it.
So let me rephrase the question:
How do I, as a regular user of my developer machine, run the tests in a
docker container, being root in the container?
Note that I can run docker without sudo.
I don't want the ansible plabook to start creating files in my own
/usr/local/bin. Which is what I believe would happen if I run it as
root. I want it to:
* pull the fedora image if needed
* start the container
* go in, be root inside
* install the package (from where?)
* run the tests (inside the container, with the installed package)
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:45 a.m.
On Mon, Apr 23, 2018 at 5:31 PM, Miro Hrončok <mhroncok(a)redhat.com> wrote:
On 23.4.2018 12:55, Andrei Stepanov wrote:
> Miro, Hi!
>
Hi Andrei, thanks for your answer.
Please check how you run tests.
> From the snippet I see that you run as ordinary user. ansible-playbook
> must be run as root.
>
I run the tests as a regular user, I want them to be executed in a
container, being root inside. That should not require me to run it as root.
More logs also would be fine.
>
The logs are not helpful, because they indicate the problem: I'm not root.
Yet I've attached it.
So let me rephrase the question:
How do I, as a regular user of my developer machine, run the tests in a
docker container, being root in the container?
Note that I can run docker without sudo.
I don't want the ansible plabook to start creating files in my own
/usr/local/bin. Which is what I believe would happen if I run it as root. I
want it to:
From: https://fedoraproject.org/wiki/CI/Standard_Test_Interface
- MUST execute the playbook as root
...
- SHOULD drop privileges appropriately if the test suite should be run
as non-root.
Also there should be an env variable:
https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Inventory
export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo
/usr/share/ansible/inventory)
Miro, may I ask. How do you see system starts TEST_SUBJECTS=docker:
docker.io/library/fedora:26 with ordinary user credentials?
Thanks!
* pull the fedora image if needed
* start the container
* go in, be root inside
* install the package (from where?)
* run the tests (inside the container, with the installed package)
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
10:58 a.m.
On 23.4.2018 17:45, Andrei Stepanov wrote:
On Mon, Apr 23, 2018 at 5:31 PM, Miro Hrončok <mhroncok(a)redhat.com
<mailto:mhroncok@redhat.com>> wrote:
On 23.4.2018 12:55, Andrei Stepanov wrote:
Miro, Hi!
Hi Andrei, thanks for your answer.
Please check how you run tests.
From the snippet I see that you run as ordinary user.
ansible-playbook must be run as root.
I run the tests as a regular user, I want them to be executed in a
container, being root inside. That should not require me to run it
as root.
More logs also would be fine.
The logs are not helpful, because they indicate the problem: I'm not
root. Yet I've attached it.
So let me rephrase the question:
How do I, as a regular user of my developer machine, run the tests
in a docker container, being root in the container?
Note that I can run docker without sudo.
I don't want the ansible plabook to start creating files in my own
/usr/local/bin. Which is what I believe would happen if I run it as
root. I want it to:
From: https://fedoraproject.org/wiki/CI/Standard_Test_Interface
<https://fedoraproject.org/wiki/CI/Standard_Test_Interface>
* MUST execute the playbook as root
I want to execute the playbook as root inside a container.
Also there should be an env variable:
https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Inventory
<https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Inventory>
export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo
/usr/share/ansible/inventory)
Doesn't change a thing.
Miro, may I ask. How do you see system starts
TEST_SUBJECTS=docker:docker.io/library/fedora:26
<http://docker.io/library/fedora:26> with ordinary user credentials?
[tests (master)]$ whoami
churchyard
[tests (master)]$ docker run -ti fedora:rawhide /bin/bash
[root@397fa7f75863 /]# whoami
root
[root@397fa7f75863 /]# exit
exit
If there is no simple way to run the test in mock/container/VM, with a
simple command, I'm afraid the whole idea of how the CI is designed is
flawed, because the barrier to cross before I can even write and execute
a minimal smoke test is extremely high. If this is not possible trough
ansible, please provide a wrapper that does exactly this:
* spins up a mock/docker/VM/etc.
* copies/mounts/etc. the tests inside
* installs the selected rpm package inside
* executes the ansible playbook as root inside
* reports the tests result outside
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
11:18 a.m.
On Mon, Apr 23, 2018 at 5:58 PM, Miro Hrončok <mhroncok(a)redhat.com> wrote:
On 23.4.2018 17:45, Andrei Stepanov wrote:
>
> On Mon, Apr 23, 2018 at 5:31 PM, Miro Hrončok <mhroncok(a)redhat.com
> <mailto:mhroncok@redhat.com>> wrote:
>
> On 23.4.2018 12:55, Andrei Stepanov wrote:
>
> Miro, Hi!
>
>
> Hi Andrei, thanks for your answer.
>
> Please check how you run tests.
> From the snippet I see that you run as ordinary user.
> ansible-playbook must be run as root.
>
>
> I run the tests as a regular user, I want them to be executed in a
> container, being root inside. That should not require me to run it
> as root.
>
> More logs also would be fine.
>
>
> The logs are not helpful, because they indicate the problem: I'm not
> root. Yet I've attached it.
>
> So let me rephrase the question:
>
> How do I, as a regular user of my developer machine, run the tests
> in a docker container, being root in the container?
>
> Note that I can run docker without sudo.
>
> I don't want the ansible plabook to start creating files in my own
> /usr/local/bin. Which is what I believe would happen if I run it as
> root. I want it to:
>
>
>
> From: https://fedoraproject.org/wiki/CI/Standard_Test_Interface <
> https://fedoraproject.org/wiki/CI/Standard_Test_Interface>
>
> * MUST execute the playbook as root
>
I want to execute the playbook as root inside a container.
There are test-runner and test-environment (
https://fedoraproject.org/wiki/CI/Standard_Test_Interface)
Standard_Test_Interface expects that tests start at test-runner.
There could be some preparation steps on test-runner (like installing
packages, etc). To make this possible and STI defines unified requirements:
"MUST execute the playbook as root" on test-runner.
CI pipeline runs all playbooks with root credentials.
Also there should be an env variable:
> https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Inventory <
> https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Inventory>
>
> export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo
> /usr/share/ansible/inventory)
>
Doesn't change a thing.
Miro, may I ask. How do you see system starts TEST_SUBJECTS=docker:
> docker.io/library/fedora:26 <http://docker.io/library/fedora:26> with
> ordinary user credentials?
>
[tests (master)]$ whoami
churchyard
[tests (master)]$ docker run -ti fedora:rawhide /bin/bash
[root@397fa7f75863 /]# whoami
root
[root@397fa7f75863 /]# exit
exit
Could you please say: id churchyard ?
The point is: user that can start/stop/act on containers can do any
modification to all system. It is the same as you act from root account.
Short: "Giving them full root access to the host system."
Long:
https://www.projectatomic.io/blog/2015/08/why-we-dont-let-non-root-users-...
If there is no simple way to run the test in mock/container/VM, with
a
simple command, I'm afraid the whole idea of how the CI is designed is
flawed, because the barrier to cross before I can even write and execute a
minimal smoke test is extremely high. If this is not possible trough
ansible, please provide a wrapper that does exactly this:
* spins up a mock/docker/VM/etc.
* copies/mounts/etc. the tests inside
* installs the selected rpm package inside
* executes the ansible playbook as root inside
* reports the tests result outside
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
11:51 a.m.
On 23.4.2018 18:18, Andrei Stepanov wrote:
I want to execute the playbook as root inside a container.
There are test-runner and test-environment
(https://fedoraproject.org/wiki/CI/Standard_Test_Interface)
I understood that this is a specification. As a user, I'm not really
interested in a specification, I want to be able to create integration
tests with ease. I feel lost in all the MUSTs, etc.
Standard_Test_Interface expects that tests start at test-runner.
There could be some preparation steps on test-runner (like installing
packages, etc). To make this possible and STI defines unified
requirements: "MUST execute the playbook as root" on test-runner.
CI pipeline runs all playbooks with root credentials.
I understand that. Yet I struggle to understand the following:
How do I test that my tests are correct without running them on my own
machine under root? Please provide examples, preferably link to a how to
(or if it is not yet documented, we can do that together).
Could you please say: id churchyard ?
I'm in the docker group.
uid=1000(churchyard) gid=1000(churchyard)
groups=1000(churchyard),10(wheel),18(dialout),135(mock),1002(taskotron),1004(docker)
The point is: user that can start/stop/act on containers can do any
modification to all system. It is the same as you act from root account.
Short: "Giving them full root access to the host system."
Long:
https://www.projectatomic.io/blog/2015/08/why-we-dont-let-non-root-users-...
I understand that this is not secure. However I wrote the tests and I am
not running some arbitrary (possibly malicious) code. I just want to do
this to avoid the tests to create and modify files in my system.
-----------------------
Let me explain a bit about what's my "goal", so we are not burning time
on Y problem.
I wrote a script that runs some commands (pdflatex in particular). If
that script exits with 0, I consider it good. If it exits with >0, I
consider it bad. I can successfully run the script on my machine to test
if my pdflatex works as expected.
Now I want to put this into CI, so when a new version of texlive is
built, this script runs on a system with the newly built latex. If it
fails, somebody needs to be notified.
In order to do this, I went trough [1] and I created a bunch of
boilerplate in yaml to run the script (which is tedious, but acceptable,
I guess).
Now I want to verify that my yaml boilerplate works. I want to say:
run-this-standard-test-in-docker --image fedora:rawhide \
--nvr texlive-2017-3.fc29
Yet I struggle to find a way how.
Note that the following works for me as well:
run-this-standard-test-in-mock --mock fedora-rawhide-x86_64 \
--nvr texlive-2017-3.fc29
[1] https://fedoraproject.org/wiki/CI/Tests#Wrapping
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
12:39 a.m.
I concur with Miro, I don't want to run the test suite directly on my
machine and the less I want to run it with root privileges. I think the
best option would be to use mock, since we are using mock all the time
to build packages.
However, I have SystemTap tests which needs to build and load Kernel
module. While it is possible to load kernel modules in container, I am
not sure it is wise to do that. So Vagrant or something similar could be
used instead (or in parallel?).
V.
Dne 23.4.2018 v 18:51 Miro Hrončok napsal(a):
On 23.4.2018 18:18, Andrei Stepanov wrote:
> I want to execute the playbook as root inside a container.
>
>
> There are test-runner and test-environment
> (https://fedoraproject.org/wiki/CI/Standard_Test_Interface)
I understood that this is a specification. As a user, I'm not really
interested in a specification, I want to be able to create integration
tests with ease. I feel lost in all the MUSTs, etc.
> Standard_Test_Interface expects that tests start at test-runner.
>
> There could be some preparation steps on test-runner (like installing
> packages, etc). To make this possible and STI defines unified
> requirements: "MUST execute the playbook as root" on test-runner.
>
> CI pipeline runs all playbooks with root credentials.
I understand that. Yet I struggle to understand the following:
How do I test that my tests are correct without running them on my own
machine under root? Please provide examples, preferably link to a how
to (or if it is not yet documented, we can do that together).
> Could you please say: id churchyard ?
I'm in the docker group.
uid=1000(churchyard) gid=1000(churchyard)
groups=1000(churchyard),10(wheel),18(dialout),135(mock),1002(taskotron),1004(docker)
> The point is: user that can start/stop/act on containers can do any
> modification to all system. It is the same as you act from root account.
> Short: "Giving them full root access to the host system."
> Long:
>
https://www.projectatomic.io/blog/2015/08/why-we-dont-let-non-root-users-...
I understand that this is not secure. However I wrote the tests and I
am not running some arbitrary (possibly malicious) code. I just want
to do this to avoid the tests to create and modify files in my system.
-----------------------
Let me explain a bit about what's my "goal", so we are not burning
time on Y problem.
I wrote a script that runs some commands (pdflatex in particular). If
that script exits with 0, I consider it good. If it exits with >0, I
consider it bad. I can successfully run the script on my machine to
test if my pdflatex works as expected.
Now I want to put this into CI, so when a new version of texlive is
built, this script runs on a system with the newly built latex. If it
fails, somebody needs to be notified.
In order to do this, I went trough [1] and I created a bunch of
boilerplate in yaml to run the script (which is tedious, but
acceptable, I guess).
Now I want to verify that my yaml boilerplate works. I want to say:
run-this-standard-test-in-docker --image fedora:rawhide \
--nvr texlive-2017-3.fc29
Yet I struggle to find a way how.
Note that the following works for me as well:
run-this-standard-test-in-mock --mock fedora-rawhide-x86_64 \
--nvr texlive-2017-3.fc29
[1] https://fedoraproject.org/wiki/CI/Tests#Wrapping
1:20 a.m.
I have proposed a solution to this use case a while ago but did't get too
much support. You can see the issue and the PR there:
https://pagure.io/standard-test-roles/issue/106
https://pagure.io/standard-test-roles/pull-request/109
Please weight in if you think it solves your needs.
Thanks,
Fred
On Tue, Apr 24, 2018 at 7:39 AM, Vít Ondruch <vondruch(a)redhat.com> wrote:
I concur with Miro, I don't want to run the test suite directly
on my
machine and the less I want to run it with root privileges. I think the
best option would be to use mock, since we are using mock all the time
to build packages.
However, I have SystemTap tests which needs to build and load Kernel
module. While it is possible to load kernel modules in container, I am
not sure it is wise to do that. So Vagrant or something similar could be
used instead (or in parallel?).
V.
Dne 23.4.2018 v 18:51 Miro Hrončok napsal(a):
> On 23.4.2018 18:18, Andrei Stepanov wrote:
>> I want to execute the playbook as root inside a container.
>>
>>
>> There are test-runner and test-environment
>> (https://fedoraproject.org/wiki/CI/Standard_Test_Interface)
>
> I understood that this is a specification. As a user, I'm not really
> interested in a specification, I want to be able to create integration
> tests with ease. I feel lost in all the MUSTs, etc.
>
>> Standard_Test_Interface expects that tests start at test-runner.
>>
>> There could be some preparation steps on test-runner (like installing
>> packages, etc). To make this possible and STI defines unified
>> requirements: "MUST execute the playbook as root" on test-runner.
>>
>> CI pipeline runs all playbooks with root credentials.
>
> I understand that. Yet I struggle to understand the following:
>
> How do I test that my tests are correct without running them on my own
> machine under root? Please provide examples, preferably link to a how
> to (or if it is not yet documented, we can do that together).
>
>> Could you please say: id churchyard ?
>
> I'm in the docker group.
>
> uid=1000(churchyard) gid=1000(churchyard)
> groups=1000(churchyard),10(wheel),18(dialout),135(mock),
1002(taskotron),1004(docker)
>
>> The point is: user that can start/stop/act on containers can do any
>> modification to all system. It is the same as you act from root account.
>> Short: "Giving them full root access to the host system."
>> Long:
>> https://www.projectatomic.io/blog/2015/08/why-we-dont-let-
non-root-users-run-docker-in-centos-fedora-or-rhel/
>
> I understand that this is not secure. However I wrote the tests and I
> am not running some arbitrary (possibly malicious) code. I just want
> to do this to avoid the tests to create and modify files in my system.
>
> -----------------------
>
> Let me explain a bit about what's my "goal", so we are not burning
> time on Y problem.
>
> I wrote a script that runs some commands (pdflatex in particular). If
> that script exits with 0, I consider it good. If it exits with >0, I
> consider it bad. I can successfully run the script on my machine to
> test if my pdflatex works as expected.
>
> Now I want to put this into CI, so when a new version of texlive is
> built, this script runs on a system with the newly built latex. If it
> fails, somebody needs to be notified.
>
> In order to do this, I went trough [1] and I created a bunch of
> boilerplate in yaml to run the script (which is tedious, but
> acceptable, I guess).
>
> Now I want to verify that my yaml boilerplate works. I want to say:
>
> run-this-standard-test-in-docker --image fedora:rawhide \
> --nvr texlive-2017-3.fc29
>
> Yet I struggle to find a way how.
>
> Note that the following works for me as well:
>
>
> run-this-standard-test-in-mock --mock fedora-rawhide-x86_64 \
> --nvr texlive-2017-3.fc29
>
> [1] https://fedoraproject.org/wiki/CI/Tests#Wrapping
>
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
4:31 a.m.
On 22.4.2018 15:15, Miro Hrončok wrote:
How do I run the standard test interface tests with docker?
OK I get it that there is apparently no automated way to run CI tests in
Docker. I can live with that (I guess).
However, what is the manual way? I still cannot use this CI thing
because this is blocking me. How are others running the tests?
Thanks for tips.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
7:19 a.m.
On 3.5.2018 11:31, Miro Hrončok wrote:
On 22.4.2018 15:15, Miro Hrončok wrote:
> How do I run the standard test interface tests with docker?
OK I get it that there is apparently no automated way to run CI tests in
Docker. I can live with that (I guess).
However, what is the manual way? I still cannot use this CI thing
because this is blocking me. How are others running the tests?
I would still very much like to run the CI tests on my machine, in an
isolated environment (such as mock, docker, etc.). Without that
possibility, the entire benefit is worthless.
Is there nobody who would know how to run the tests?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
11:11 a.m.
On Mon, May 28, 2018 at 02:19:19PM +0200, Miro Hrončok wrote:
I would still very much like to run the CI tests on my machine, in
an
isolated environment (such as mock, docker, etc.). Without that possibility,
the entire benefit is worthless.
Is there nobody who would know how to run the tests?
This works for me:
TEST_SUBJECTS=~/cloud-images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2
TEST_ARTIFACTS=~/.cache/artifacts
ansible-playbook -i /usr/share/ansible/inventory/standard-inventory-qcow2 tests.yml
I use these images:
https://download.fedoraproject.org/pub/fedora/linux/releases/27/CloudImag...
https://download.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86...
https://download.fedoraproject.org/pub/fedora/linux/releases/26/CloudImag...
https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-180...
https://cloud.centos.org/centos/6/images/CentOS-6-x86_64-GenericCloud-180...
There is also a tool that I use in a different project that starts a
docker container that starts VMs:
https://github.com/linux-system-roles/test-harness/pull/19/commits/0c985c...
It could be adjusted for your use case I guess.
It is not docker but a VM, but it is a start.
Kind regards
Till
2174
days inactive
2210
days old
11 comments
5 participants
participants (5)
-
Andrei Stepanov -
Frederic Lepied -
Miro Hrončok -
Till Maas -
Vít Ondruch