[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Node report OK but every pod marked unready



It seems that error is related to docker storage on that vm

.t

Sent from my iPhone

On 20 Apr 2017, at 8.53, Andrew Lau <andrew andrewklau com> wrote:

Unfortunately I did not. I dumped the logs and just removed the node in order to quickly restore the current containers on another node.

At the exact time it failed I saw a lot of the following:

===
thin_pool_watcher.go:72] encountered error refreshing thin pool watcher: 
error performing thin_ls on metadata device /dev/mapper/docker_vg-docker--pool_tmeta: Error running command `thin_ls --no-headers -m -o DEV,
EXCLUSIVE_BYTES /dev/mapper/docker_vg-docker--pool_tmeta`: exit status 127

failed (failure): rpc error: code = 2 desc = shim error: context deadline exceeded#015

Error running exec in container: rpc error: code = 2 desc = shim error: context deadline exceeded
===



On Thu, 20 Apr 2017 at 15:41 Tero Ahonen <tahonen redhat com> wrote:
Hi

Did u try to ssh to that node and execute sudo docker run to some container?

.t

Sent from my iPhone

> On 20 Apr 2017, at 8.18, Andrew Lau <andrew andrewklau com> wrote:
>
> I'm trying to debug a weird scenario where a node has had every pod crash with the error:
> "rpc error: code = 2 desc = shim error: context deadline exceeded"
>
> The pods stayed in the state Ready 0/1
> The docker daemon was responding and the kublet and all it's services were running. The node was reporting with the OK status.
>
> No resource limits were hit with CPU almost idle and memory at 25% utilisation.
>
>
>
>
> _______________________________________________
> users mailing list
> users lists openshift redhat com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]