[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Pods hanging in pending state

I've seem this when docker is pulling a image which has a common layer for a long time, looks like docker client doesn't pull images concurrently since there's issues on the graph driver.

A simple workaround is restart docker daemon.

Diego Castro
The CloudFather
(11) 3230.5927
+54 (911) 2159.1779
gtalk: diego castro getupcloud com

2016-02-18 11:34 GMT-03:00 Bendik Paulsrud <bendik paulsrud gmail com>:
Hi again,

We have had a new occurrence of a hanging pod. I have not restarted the service and here are the logs from the time interval the pod was deployed.

"oc describe pod <pod>" gives me this:

"journalctl -u docker -l" gives me this:

"journalctl -u origin-node -l" gives me this:

"docker info" states there is data space available.

I tried pull the image (kubernetes/redis:v1) afterwards and that worked fine.

Bendik Paulsrud

On 16 February 2016 at 15:48, Andy Goldstein <agoldste redhat com> wrote:
If this happens again, please capture the logs prior to restarting the service. That's really the only way we can try to discern what's happening.


On Tue, Feb 16, 2016 at 7:30 AM, Bendik Paulsrud <bendik paulsrud gmail com> wrote:

The node has not out of disk, and "docker info" showed there were available space left.

Since I don't know when the pending state occured i restarted origin-node and pasted the logs from origin-node here:

The issue seems to stop after restarting both origin-node and docker.

Version of Docker is v1.8.2 and OpenShift Origin is v1.1.1.1.

Bendik Paulsrud

On 16 February 2016 at 13:06, Andy Goldstein <agoldste redhat com> wrote:
Are you out of disk space anywhere? Can you post the node service logs somewhere for us to review?


On Tuesday, February 16, 2016, Bendik Paulsrud <bendik paulsrud gmail com> wrote:

We have encountered a situation where pods keep hanging in Pending state on one node in the cluster. When checking the node(docker ps), there are only maintainer containers (k8s_POD) running, no application containers.

When describing the pod with "oc describe" no error message is displayed, only that the maintainer container is assigned to the node and has started. There are no error messages in the event log either.

Checking the origin-node log and docker log I can't see any error messages.

Have anyone encountered a similar situation and know why this occurs?

Bendik Paulsrud

users mailing list
users lists openshift redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]