New subject: [Bug 1102911] can't run iscsid in a docker container

29 May 2014


      https://bugzilla.redhat.com/show_bug.cgi?id=1102911
Bug ID: 1102911
           Summary: can't run iscsid in a docker container
           Product: Fedora
           Version: 20
         Component: iscsi-initiator-utils
          Assignee: cleech@redhat.com
          Reporter: jslagle@redhat.com
        QA Contact: extras-qa@fedoraproject.org
                CC: admiller@redhat.com, agrover@redhat.com,
                    cleech@redhat.com, dwalsh@redhat.com,
                    golang@lists.fedoraproject.org, hdegoede@redhat.com,
                    lsm5@redhat.com, mattdm@redhat.com,
                    mgoldman@redhat.com, s@shk.io, vbatts@redhat.com
        Depends On: 1100000
+++ This bug was initially created as a clone of Bug #1100000 +++
Description of problem:
Can't start iscsid in a docker container
Version-Release number of selected component (if applicable):
# rpm -q docker-io
docker-io-0.11.1-3.fc20.x86_64
How reproducible:
always
Steps to Reproduce:
1. use the published fedora image, docker pull fedora
2. start the container, docker run -t -i fedora /bin/bash
3. install iscsi-initiator-utils
4. try to start iscsid:
bash-4.2# iscsid -f
iscsid: can not bind NETLINK_ISCSI socket
strace also attached
--- Additional comment from James Slagle on 2014-05-21 14:45:57 EDT ---
To give a little more context into what I'm doing, I'm trying to run OpenStack
nova compute configured to use the nova-baremetal driver inside a container.
when nova-baremetal provisions a machine it acts as an iscsi initiator and logs
into a target that has been created on the machine that is being provisioned.
It then dd's the requested image onto the disk.
therefore, aiui, iscsid must be running inside the container where you are also
running iscsiadm.
This same thing has also been tried in lxc, with what I expect is the same
issue:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1226855
--- Additional comment from Daniel Walsh on 2014-05-28 12:29:35 EDT ---
Was SELinux involved?  If you put the machine into permissive mode does it
work?  Or try this as a privleged image.  Might be something that we are doing
to lock the system down.
--- Additional comment from Daniel Walsh on 2014-05-28 12:33:18 EDT ---
What are the permissions on /opt/hello
ls -ld /opt 
ls -ld /opt/hello
--- Additional comment from James Slagle on 2014-05-28 15:01:53 EDT ---
SELinux is already in permissive mode on the Docker host.
I did try in a privileged container, and I get something slightly different.
iscsid -f just hangs forever on the command line.
An strace shows (attached) it polling forever on a fd, i had to ctrl-c it in
both cases.
--- Additional comment from James Slagle on 2014-05-28 15:02:26 EDT ---
--- Additional comment from James Slagle on 2014-05-28 15:03:16 EDT ---
i think comment 3 was for another bug maybe?  anyway, if not:
bash-4.2# ls -ld /opt 
drwxr-xr-x. 2 root root 4096 Aug  7  2013 /opt
bash-4.2# ls -ld /opt/hello
ls: cannot access /opt/hello: No such file or directory
--- Additional comment from James Slagle on 2014-05-28 15:08:58 EDT ---
ah....actually, i suspect iscsid running forever in the foreground may indicate
it *is* working.  sorry, i wasn't thinking that -f was telling it to run in the
foreground.
i will see if i can actually connect to a target from the privileged container
and report back
--- Additional comment from James Slagle on 2014-05-29 09:34:32 EDT ---
I'm using a privileged container running sshd as the process (so that I can
login with a couple of different shells),
I had to add this to my Dockerfile for the container, otherwise iscsid won't
start:
VOLUME ["/var/lock/iscsi"]
i start the conatiner with:
docker run --privileged -ti --name initiator -p 8022:22 -d iscsi-initiator
then ssh in, and i start iscsid:
iscsid -d 8 -f
that appears to start fine
then ssh in another session, and i can discover the target (target is actually
on the container host):
[root@bcf697cb8673 ~]# iscsiadm -m discovery -t st -p 192.168.122.1
192.168.122.1:3260,1 iqn.2013-07.com.example.storage.ssd1
but when i try to login to the target, iscsid exits (or crashes, hard to tell):
[root@bcf697cb8673 ~]# iscsiadm -m node --targetname
iqn.2013-07.com.example.storage.ssd1 --portal 192.168.122.1 --login
Logging in to [iface: default, target: iqn.2013-07.com.example.storage.ssd1,
portal: 192.168.122.1,3260] (multiple)
iscsiadm: got read error (0/0), daemon died?
iscsiadm: Could not login to [iface: default, target:
iqn.2013-07.com.example.storage.ssd1, portal: 192.168.122.1,3260].
iscsiadm: initiator reported error (18 - could not communicate to iscsid)
iscsiadm: Could not log into all portals
from the ssh session where iscsid is running i see:
iscsid: mgmt_ipc_write_rsp: rsp to fd 5
iscsid: poll result 1
iscsid: mgmt_ipc_write_rsp: rsp to fd 5
iscsid: poll result 1
iscsid: in read_transports
iscsid: Adding new transport tcp
iscsid: Matched transport tcp
iscsid: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'handle'
iscsid: sysfs_attr_get_value: new uncached attribute
'/sys/class/iscsi_transport/tcp/handle'
iscsid: sysfs_attr_get_value: add to cache
'/sys/class/iscsi_transport/tcp/handle'
iscsid: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/handle'
with attribute value '18446744072107593760'
iscsid: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'caps'
iscsid: sysfs_attr_get_value: new uncached attribute
'/sys/class/iscsi_transport/tcp/caps'
iscsid: sysfs_attr_get_value: add to cache
'/sys/class/iscsi_transport/tcp/caps'
iscsid: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/caps' with
attribute value '0x39'
iscsid: Allocted session 0x7f38ceb4f9b0
iscsid: no authentication configured...
iscsid: resolved 192.168.122.1 to 192.168.122.1
iscsid: setting iface default, dev , set ip , hw , transport tcp.
iscsid: get ev context 0x7f38ceb5c470
iscsid: set TCP recv window size to 524288, actually got 425984
iscsid: set TCP send window size to 524288, actually got 425984
iscsid: connecting to 192.168.122.1:3260
iscsid: sched conn context 0x7f38ceb5c470 event 2, tmo 0
iscsid: thread 0x7f38ceb5c470 schedule: delay 0 state 3
iscsid: Setting login timer 0x7f38ceb578e0 timeout 15
iscsid: thread 0x7f38ceb578e0 schedule: delay 60 state 3
iscsid: exec thread 7f38ceb5c470 callback
iscsid: put ev context 0x7f38ceb5c470
iscsid: connected local port 37259 to 192.168.122.1:3260
iscsid: in kcreate_session
iscsid: in __kipc_call
iscsid: in kwritev
iscsid: sendmsg: bug? ctrl_fd 4
Maybe the lines with sysfs_attr_get_value are indicative of something that's
needed from /sys still?
These exact same discovery and login commands work fine running from a libvirt
vm connecting to the same target.
On my container host, i do have the correct iscsi kernel modules, and I also
see these in the container:
on the host:
[root@teletran-1 docker]# lsmod | grep iscsi
iscsi_tcp              18333  0 
libiscsi_tcp           24176  1 iscsi_tcp
libiscsi               54750  2 libiscsi_tcp,iscsi_tcp
scsi_transport_iscsi    97405  4 iscsi_tcp,libiscsi
on the container:
[root@bcf697cb8673 ~]# lsmod | grep iscsi
iscsi_tcp              18333  0 
libiscsi_tcp           24176  1 iscsi_tcp
libiscsi               54750  2 libiscsi_tcp,iscsi_tcp
scsi_transport_iscsi    97405  4 iscsi_tcp,libiscsi
i'll attach an strace of the iscsid process that's exiting, if that helps.
i can also attach an strace of an iscsid process that shows it working from a
libvirt VM, if you think that would be helpful to compare.
--- Additional comment from James Slagle on 2014-05-29 09:37:12 EDT ---
strace of iscsid generated with:
strace -f -o iscsid.strace iscsid -d 12 -f
the iscsid process exits when you try to login to an iscsi target from the
container.
--- Additional comment from James Slagle on 2014-05-29 10:08:33 EDT ---
note that the output i'm now seeing seems to match very closely what was
reported in https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1226855 when
this same thing was tried with lxc-tools instead of Docker.
--- Additional comment from Daniel Walsh on 2014-05-29 15:32:31 EDT ---
So this looks to be more specific to iscsid and namespacing then to docker.  I
think you should open the bug with them and see if they can help.
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1100000
[Bug 1100000] can't run iscsid in a docker container
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1102911] New: can't run iscsid in a docker container