9:38 a.m.
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Bug ID: 1096273
Summary: repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
Product: Red Hat Enterprise Linux 7
Version: 7.1
Component: docker
Severity: low
Assignee: lsm5(a)redhat.com
Reporter: ldoktor(a)redhat.com
QA Contact: virt-bugs(a)redhat.com
CC: admiller(a)redhat.com, golang(a)lists.fedoraproject.org,
lsm5(a)redhat.com, mattdm(a)redhat.com,
mgoldman(a)redhat.com, skottler(a)redhat.com,
vbatts(a)redhat.com
Depends On: 1087704
+++ This bug was initially created as a clone of Bug #1087704 +++
Description of problem:
I noticed weird reaction to SIGSTOP signal. When you start container with
--sig-proxy, ignore all signals in container and from OS use kill -SIGSTOP to
the `docker run` process, it's put into background. (so far correct behavior)
Than when you use kill -SIGCONT, it's resumed (still in background) and it's
continuing to send signal 21 to the container (100% CPU utilization).
This disappears once you attach the process with fg.
Version-Release number of selected component (if applicable):
docker-0.10.0-8.el7.x86_64
docker-io-0.9.1-1.fc21.x86_64
How reproducible:
always
Steps to Reproduce:
1. /usr/bin/docker -D run --tty=false --rm -i --name test_eoly
localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`; do trap
"echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1; done'
2. ps ax |grep docker
3. kill -SIGSTOP $PID
4. kill -SIGCONT $PID
Actual results:
[1]+ Pozastavena /usr/bin/docker -D run --tty=false --rm -i --name
test_eoly localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`;
do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1;
done'
[root@t530 ~]# Received 21, ignoring...
Received 21, ignoring...
Received 21, ignoring...
Received 21, ignoring...
Received 21, ignoring...
Received 21, ignoring...
... (100% CPU utilization, when you use fg it stops and you can use the
container again...)
Expected results:
[1]+ Pozastavena /usr/bin/docker -D run --tty=false --rm -i --name
test_eoly localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`;
do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1;
done'
[root@t530 ~]# Received 21, ignoring...
Additional info:
I'm not definitely sure if this is really required as it's not a common usage.
But as there is no way to detach the process apart from kill -SIGSTOP people
might use it and get to this problem...
--- Additional comment from Lukas Doktor on 2014-05-05 03:59:30 EDT ---
I retested this with the upstream dc9c28f/0.10.0 with the same results. The
daemon output is:
...
[/home/medic/Work/Projekty/Docker/root|fa3816b6] -job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN) =
OK (0)
2014/05/05 09:58:13 POST
/v1.10/containers/b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5/kill?signal=TTIN
[/home/medic/Work/Projekty/Docker/root|fa3816b6] +job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN)
[/home/medic/Work/Projekty/Docker/root|fa3816b6] -job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN) =
OK (0)
2014/05/05 09:58:13 POST
/v1.10/containers/b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5/kill?signal=TTIN
[/home/medic/Work/Projekty/Docker/root|fa3816b6] +job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN)
[/home/medic/Work/Projekty/Docker/root|fa3816b6] -job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN) =
OK (0)
2014/05/05 09:58:13 POST
/v1.10/containers/b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5/kill?signal=TTIN
[/home/medic/Work/Projekty/Docker/root|fa3816b6] +job
kill(b01a849cb45ebe94c3a61fa021a5464186345d5b159faee4ea9d5da39fb36de5, TTIN)
...
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1087704
[Bug 1087704] repetition of signal 21 with --sig-proxy and SIGSTOP of
docker
--
You are receiving this mail because:
You are on the CC list for the bug.
9:53 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #1 from Lukas Doktor <ldoktor(a)redhat.com> ---
There is a slight difference between fedora and RHEL behavior in this
situation.
[RHEL results]
Actual results:
[1]+ Pozastavena /usr/bin/docker -D run --tty=false --rm -i --name
test_eoly localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`;
do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1;
done'
[root@t530 eclipse_links 147]$ [debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
No new lines, no output.
Output of docker daemon:
[31257f3e] -job
kill(d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688, TTIN) =
OK (0)
2014/05/09 16:35:38 POST
/v1.11/containers/d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688/kill?signal=TTIN
[31257f3e] +job
kill(d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688, TTIN)
[31257f3e] -job
kill(d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688, TTIN) =
OK (0)
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 5ms
2014/05/09 16:35:38 POST
/v1.11/containers/d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688/kill?signal=TTIN
[31257f3e] +job
kill(d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688, TTIN)
[31257f3e] -job
kill(d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688, TTIN) =
OK (0)
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 10ms
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 20ms
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 40ms
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 80ms
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 160ms
2014/05/09 16:35:38 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 320ms
2014/05/09 16:35:39 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 640ms
2014/05/09 16:35:39 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 1s
2014/05/09 16:35:40 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 1s
2014/05/09 16:35:41 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 1s
2014/05/09 16:35:42 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 1s
...
docker stops printing the "Received signal..." message. I'm able to use fg
to
join, but the container is not responding. When I press ctrl+c in docker daemon
it prints:
^C2014/05/09 16:41:26 Received signal 'interrupt', starting shutdown of
docker...
^C2014/05/09 16:41:27 Received signal 'interrupt', starting shutdown of
docker...
2014/05/09 16:41:27 http: Accept error: accept unix /var/run/docker.sock: too
many open files; retrying in 1s
^C2014/05/09 16:41:27 Received signal 'interrupt', starting shutdown of
docker...
after another ctrl+c:
2014/05/09 16:41:27 Force shutdown of docker, interrupting cleanup
and the process finishes.
When I execute the daemon again, it's unable to remove the containers:
docker rm output:
Error: Cannot destroy container d09b358f4e33: Driver devicemapper failed to
remove root filesystem
d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688: Error running
removeDevice
6de66f25bd2a
Error: Cannot destroy container 7a57a8a67336: Driver devicemapper failed to
remove root filesystem
7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7: Error running
removeDevice
2014/05/09 16:44:29 Error: failed to remove one or more containers
Docker daemon output:
2014/05/09 16:44:07 DELETE /v1.11/containers/d09b358f4e33?force=1&v=1
[e6f019e8] +job container_delete(d09b358f4e33)
Cannot destroy container d09b358f4e33: Driver devicemapper failed to remove
root filesystem
d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688: Error running
removeDevice
[e6f019e8] -job container_delete(d09b358f4e33) = ERR (1)
[error] server.go:1014 Error: Cannot destroy container d09b358f4e33: Driver
devicemapper failed to remove root filesystem
d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688: Error running
removeDevice
[error] server.go:90 HTTP Error: statusCode=500 Cannot destroy container
d09b358f4e33: Driver devicemapper failed to remove root filesystem
d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688: Error running
removeDevice
2014/05/09 16:44:17 DELETE /v1.11/containers/6de66f25bd2a?force=1&v=1
[e6f019e8] +job container_delete(6de66f25bd2a)
[e6f019e8] -job container_delete(6de66f25bd2a) = OK (0)
2014/05/09 16:44:18 DELETE /v1.11/containers/7a57a8a67336?force=1&v=1
[e6f019e8] +job container_delete(7a57a8a67336)
Cannot destroy container 7a57a8a67336: Driver devicemapper failed to remove
root filesystem
7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7: Error running
removeDevice
[e6f019e8] -job container_delete(7a57a8a67336) = ERR (1)
[error] server.go:1014 Error: Cannot destroy container 7a57a8a67336: Driver
devicemapper failed to remove root filesystem
7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7: Error running
removeDevice
[error] server.go:90 HTTP Error: statusCode=500 Cannot destroy container
7a57a8a67336: Driver devicemapper failed to remove root filesystem
7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7: Error running
removeDevice
mount output:
...
/dev/mapper/docker-253:3-4065163-7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7
on
/home/medic/Work/Projekty/Docker/root/devicemapper/mnt/7a57a8a6733621bf0b043f4d121221f20853256764fd0762ec49f7f595e87ac7
type ext4 (rw,relatime,seclabel,discard,stripe=16,data=ordered)
/dev/mapper/docker-253:3-4065163-d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688
on
/home/medic/Work/Projekty/Docker/root/devicemapper/mnt/d09b358f4e33b1bb1e49d7fa6e3b7c52fe618a2d36a8424b0cd4697e009f7688
type ext4 (rw,relatime,seclabel,discard,stripe=16,data=ordered)
docker ps -a output:
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
When I try to create the container again it fails with:
[root@t530 eclipse_links 2]$ /usr/bin/docker -D run --tty=false --rm -i --name
test_eoly localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`;
do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1;
done'
[debug] hijack.go:96 [hijack] End of stdout
[debug] commands.go:1958 End of CmdRun(), Waiting for hijack to finish.
2014/05/09 16:46:06 Error: Cannot start container
399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5: Cannot find
child for /test_eoly
Docker daemon output:
2014/05/09 16:46:06 POST /v1.11/containers/create?name=test_eoly
[e6f019e8] +job create(test_eoly)
-----> setting mount label discard
[e6f019e8] -job create(test_eoly) = OK (0)
2014/05/09 16:46:06 POST
/v1.11/containers/399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5/attach?stderr=1&stdin=1&stdout=1&stream=1
[e6f019e8] +job
inspect(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5,
container)
[e6f019e8] -job
inspect(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5,
container) = OK (0)
[e6f019e8] +job
attach(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
2014/05/09 16:46:06 POST
/v1.11/containers/399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5/start
[e6f019e8] +job
start(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
-----> setting mount label discard
[e6f019e8] +job
allocate_interface(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
[e6f019e8] -job
allocate_interface(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
= OK (0)
[e6f019e8] +job
release_interface(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
[e6f019e8] -job
release_interface(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5)
= OK (0)
[e6f019e8] -job
attach(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5) = OK
(0)
Cannot start container
399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5: Cannot find
child for /test_eoly
[e6f019e8] -job
start(399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5) = ERR
(1)
[error] server.go:1014 Error: Cannot start container
399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5: Cannot find
child for /test_eoly
[error] server.go:90 HTTP Error: statusCode=500 Cannot start container
399677e64c6b8414d39bf316b99f667cb26c76b2d87266124998726acda391d5: Cannot find
child for /test_eoly
When I repeat the exact same command once again (without restarting the daemon)
it passes:
2014/05/09 16:48:33 POST /v1.11/containers/create?name=test_eoly
[e6f019e8] +job create(test_eoly)
-----> setting mount label discard
[e6f019e8] -job create(test_eoly) = OK (0)
2014/05/09 16:48:33 POST
/v1.11/containers/4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234/attach?stderr=1&stdin=1&stdout=1&stream=1
[e6f019e8] +job
inspect(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234,
container)
[e6f019e8] -job
inspect(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234,
container) = OK (0)
[e6f019e8] +job
attach(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234)
2014/05/09 16:48:33 POST
/v1.11/containers/4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234/start
[e6f019e8] +job
start(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234)
-----> setting mount label discard
[e6f019e8] +job
allocate_interface(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234)
[e6f019e8] -job
allocate_interface(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234)
= OK (0)
[e6f019e8] -job
start(4aff2c82307724735a2a1c3b8723022b01642142836f9fd57c244f494eb5e234) = OK
(0)
--
You are receiving this mail because:
You are on the CC list for the bug.
2:33 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Chris Evich <cevich(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |cevich(a)redhat.com
Version|7.1 |7.0
Severity|low |high
--
You are receiving this mail because:
You are on the CC list for the bug.
3:37 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Daniel Walsh <dwalsh(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |dwalsh(a)redhat.com
--- Comment #4 from Daniel Walsh <dwalsh(a)redhat.com> ---
I am not seeing this in the current 0.11 package
--
You are receiving this mail because:
You are on the CC list for the bug.
1:24 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #5 from Lukas Doktor <ldoktor(a)redhat.com> ---
I retested this with the latest docker-0.11.1-2.el7.x86_64 on RHEL7 with the
same results as in the bugzilla:
[root@localhost client]# docker -D run --tty=false --rm -i --name test_eoly
fedora bash -c 'for NUM in `seq 1 64`; do trap "echo Received $NUM,
ignoring..." $NUM; done; while :; do sleep 1; done'
[1]+ Pozastavena docker -D run --tty=false --rm -i --name
test_eoly fedora bash -c 'for NUM in `seq 1 64`; do trap "echo Received $NUM,
ignoring..." $NUM; done; while :; do sleep 1; done'
[root@localhost client]# [debug] stdcopy.go:111 framesize: 50
Received 21, ignoring...
Received 21, ignoring...
[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
[debug] stdcopy.go:111 framesize: 25
...
# I used fg+enter
fg
docker -D run --tty=false --rm -i --name test_eoly fedora bash -c 'for NUM in
`seq 1 64`; do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do
sleep 1; done'
[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
Docker daemon output:
...
[debug] server.go:981 Calling POST /containers/{name:.*}/kill
2014/05/20 07:16:53 POST
/v1.11/containers/90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3/kill?signal=TTIN
[a8b38a24] +job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN)
[a8b38a24] -job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN) =
OK (0)
[debug] server.go:981 Calling POST /containers/{name:.*}/kill
2014/05/20 07:16:53 POST
/v1.11/containers/90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3/kill?signal=TTIN
[a8b38a24] +job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN)
[a8b38a24] -job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN) =
OK (0)
[debug] server.go:981 Calling POST /containers/{name:.*}/kill
2014/05/20 07:16:53 POST
/v1.11/containers/90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3/kill?signal=TTIN
[a8b38a24] +job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN)
[a8b38a24] -job
kill(90e0ed8723e8b0a8ddfdc8376b7d26465894bb5c4f19cca7fdae1bdf130871f3, TTIN) =
OK (0)
--
You are receiving this mail because:
You are on the CC list for the bug.
2:13 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Daniel Walsh <dwalsh(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|lsm5(a)redhat.com |mheon(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
11:04 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #6 from Matthew Heon <mheon(a)redhat.com> ---
So, you're not actually forwarding the signal to the container with
--sig-proxy.
Doing a ps -ax | grep docker does not actually find the PID you use to forward
signals to the container using --sig-proxy. Docker furnishes the PID of the
container itself, via the docker inspect command (docker inspect $CONTAINER_ID
| grep Pid). The given PID will not match the one found via ps.
I believe you're stopping and restarting some internal Docker process
responsible for managing the container. This is not something that is intended
to happen, or something we should support - I imagine Apache would also start
throwing errors if we began sending signals to its forked processes.
(For reference, I can reproduce perfectly with your instructions. However, by
changing ps -ax to docker inspect, the issue no longer occurs)
This could likely use some documentation to make it clear that ps is not a
supported method of finding a Docker container's PID (at least, with any degree
of reliability). I'll investigate existing docs to see if they address this.
--
You are receiving this mail because:
You are on the CC list for the bug.
7:37 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Daniel Walsh <dwalsh(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED
--- Comment #7 from Daniel Walsh <dwalsh(a)redhat.com> ---
Should we close this bug, or do we need to update documentation?
--
You are receiving this mail because:
You are on the CC list for the bug.
7:49 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #8 from Matthew Heon <mheon(a)redhat.com> ---
I think this does need to be documented somewhere, though I'm not sure where
best to do it - it doesn't clearly fit into any of the manpages. Might be
something for a Docker user's guide, if we have one?
--
You are receiving this mail because:
You are on the CC list for the bug.
11:32 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #9 from Matthew Heon <mheon(a)redhat.com> ---
It appears I was confused about how to use SIGPROXY - apologies, your original
report was valid.
I can reproduce this on RHEL7 with -SIGSTOP and -SIGCONT, but not with -SIGSTOP
and fg.
This mainly has to do with the fact that SIGSTOP is not proxied through
--sig-proxy, I believe. SIGSTOP is unmaskable, so the Golang signal handler
can't catch it - the foreground "docker run" process pauses, the background
container does not.
Documentation should be updated to note that Signal 19 is not proxied.
Additionally, I'm attempting to diagnose the issue with Signal 21 being
repeated forever on resume.
--
You are receiving this mail because:
You are on the CC list for the bug.
12:11 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #10 from Daniel Walsh <dwalsh(a)redhat.com> ---
Is this fixed in docker-1.0.0-10.el7.x86_64?
--
You are receiving this mail because:
You are on the CC list for the bug.
12:20 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #11 from Matthew Heon <mheon(a)redhat.com> ---
No, it is not. I can only reproduce this intermittently - sometimes sending
SIGCONT to a container stopped with SIGSTOP does nothing at all (the container
does not even resume). I'll look into this more.
--
You are receiving this mail because:
You are on the CC list for the bug.
6:11 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #12 from Lukas Doktor <ldoktor(a)redhat.com> ---
That's weird. I can reproduce this always on booth, RHEL and Fedora. Today I
tried it with the latest docker-io-1.0.0-6.fc20.x86_64 and the results are:
1) docker -D run --tty=false --rm -i --name test_eoly
localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`; do trap
"echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1; done'
2) TERM2: ps -ax |grep docker
3) TERM2: kill -SIGSTOP 10197
generates output in TERM1:
[1]+ Pozastavena docker -D run --tty=false --rm -i --name
test_eoly localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`;
do trap "echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1;
done'
4) TERM2: kill -SIGCONT 10197
generates output in TERM1:
[medic@t530 ~ 147]$ fg[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
...
until I execute fg (in TERM1):
docker -D run --tty=false --rm -i --name test_eoly
localhost:5000/ldoktor/fedora:latest bash -c 'for NUM in `seq 1 64`; do trap
"echo Received $NUM, ignoring..." $NUM; done; while :; do sleep 1; done'
[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=TTIN:
dial unix /var/run/docker.sock: too many open files
[debug] stdcopy.go:111 framesize: 25
Received 21, ignoring...
and waits. When I pres ctrl+c it complains about too many open files:
^C[debug] commands.go:556 Error sending signal: Post
http:///var/run/docker.sock/v1.12/containers/673424bd1e6e95ae83641e20bdc18065d6374e147f23df7e7fbc45e7cd240aa9/kill?signal=INT:
dial unix /var/run/docker.sock: too many open files
The only way to kill it is `kill -9 ...`
For fedora tests I'm executing docker in xterm, for RHEL I ssh from Fedora
xterm to virtual RHEL7 kvm guest. Today I tried that with gnome-terminal just
to be sure it's not related to some xterm specific behavior and it works 100%
of time.
--
You are receiving this mail because:
You are on the CC list for the bug.
7:52 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #13 from Matthew Heon <mheon(a)redhat.com> ---
I was actually running a build of the latest upstream Docker when attempting to
reproduce, which appears to be the source of the inconsistency.
The good news is that this build doesn't express this bug fully - SIG21 is sent
only 3 times, and there's no warning of too many open files.
The bad news is that 100% CPU utilization still persists despite this.
I think this likely has to do with SIGSTOP not being proxied (SIGSTOP and
SIGKILL can't be caught, so the signal proxy can't forward them - they apply to
the docker client process, not the container. I'll write up a patch to clarify
this in the docs). It seems like something is timing out on the server during
this, and the client is attempting to reconnect.
Worth noting that, if you send both signals near-simultaneously (kill -SIGSTOP
$PID; kill -SIGCONT $PID) this does not happen - definitely seems to be a
timeout.
I'll investigate further.
--
You are receiving this mail because:
You are on the CC list for the bug.
7:55 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #14 from Daniel Walsh <dwalsh(a)redhat.com> ---
BTW
Docker pause and docker unpause use CGroups to "FREEZE" containers.
--
You are receiving this mail because:
You are on the CC list for the bug.
9:09 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
--- Comment #15 from Matthew Heon <mheon(a)redhat.com> ---
Appears to be an issue with Golang proper
(https://code.google.com/p/go/issues/detail?id=4391&q=signal&colsp...)
Unknown when a fix will be released - there hasn't been any movement on the bug
since February.
--
You are receiving this mail because:
You are on the CC list for the bug.
1:14 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Daniel Walsh <dwalsh(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Component|docker |golang
QA Contact|virt-bugs(a)redhat.com |qe-baseos-tools(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
3:53 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
John Skeoch <jskeoch(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|mheon(a)redhat.com |jrieden(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
7:17 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Bug 1096273 depends on bug 1087704, which changed state.
Bug 1087704 Summary: repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1087704
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |CLOSED
Resolution|--- |CANTFIX
--
You are receiving this mail because:
You are on the CC list for the bug.
12:30 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Min Wang <minwang(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jrieden(a)redhat.com,
| |minwang(a)redhat.com
Assignee|jrieden(a)redhat.com |jcajka(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
12:56 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Matt Newsome <mnewsome(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |mnewsome(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
2:57 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Martin Cermak <mcermak(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |mcermak(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
2:57 p.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Martin Cermak <mcermak(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |mcermak(a)redhat.com
--
You are receiving this mail because:
You are on the CC list for the bug.
6:42 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Jakub Čajka <jcajka(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |MODIFIED
Fixed In Version| |golang-1.6-1.el7
--
You are receiving this mail because:
You are on the CC list for the bug.
6:42 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
errata-xmlrpc <errata-xmlrpc(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|MODIFIED |ON_QA
--
You are receiving this mail because:
You are on the CC list for the bug.
4:46 a.m.
New subject: [Bug 1096273] repetition of signal 21 with --sig-proxy and SIGSTOP
of docker
https://bugzilla.redhat.com/show_bug.cgi?id=1096273
Jakub Čajka <jcajka(a)redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC|golang(a)lists.fedoraproject. |
|org |
--
You are receiving this mail because:
You are on the CC list for the bug.
2941
days inactive
3638
days old
golang@lists.fedoraproject.org
25 comments
1 participants
participants (1)
-
Red Hat Bugzilla