[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: prune images from registry - getsockopt: connection timed out





On 11.10.2017 19:03, Ben Parees wrote:


On Wed, Oct 11, 2017 at 10:44 AM, Maciej Zarczynski <m zarczynski adbglobal com> wrote:
Hi,

for a while we are running openshift origin standalone docker-registry
and it works pretty well, but when i try to prune images i am facing
following situation:

[root c37ee07bf04f /]# time oc adm prune images --keep-tag-revisions=10 --confirm

error: error communicating with registry 172.30.146.162:5000: [Get https://172.30.146.162:5000/healthz: dial tcp 172.30.146.162:5000: getsockopt: connection timed out, Get http://172.30.146.162:5000/healthz: dial tcp 172.30.146.162:5000: getsockopt: connection timed out]


it looks like the machine you're running prune from doesn't have access to the cluster network.  can you run the command from one of your cluster nodes?

i already tried and the error message was very similar but the problem was letsencrypt certificate which is not valid for registry svc ip.
Running on the node:  oc adm prune images --keep-tag-revisions=10 --confirm  --insecure-skip-tls-verify=true 
did the trick.
Thanks for help.

real    4m38.046s

user    0m9.118s

sys    0m0.529s

[root c37ee07bf04f /]# time oc adm prune images --keep-tag-revisions=10000 --confirm

error: error communicating with registry 172.30.146.162:5000: [Get https://172.30.146.162:5000/healthz: dial tcp 172.30.146.162:5000: getsockopt: connection timed out, Get http://172.30.146.162:5000/healthz: dial tcp 172.30.146.162:5000: getsockopt: connection timed out]

real    4m37.320s

user    0m9.141s

sys    0m0.564s

[root c37ee07bf04f /]# time oc adm prune images --keep-tag-revisions=10 | wc

Dry run enabled - no modifications will be made. Add --confirm to remove images

  27771   53320 2861369

real    0m23.583s

user    0m10.540s

sys    0m0.576s

[root c37ee07bf04f /]# time oc adm prune images --keep-tag-revisions=10000 | wc

Dry run enabled - no modifications will be made. Add --confirm to remove images

     47      53    3031

real    0m23.728s

user    0m9.060s

sys    0m0.465s

[root c37ee07bf04f /]# oc get svc -n default

NAME               CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE

docker-registry    172.30.146.162   <none>        5000/TCP                  138d

kubernetes         172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     138d

registry-console   172.30.142.214   <none>        9000/TCP                  138d

router             172.30.105.200   <none>        80/TCP,443/TCP,1936/TCP   138d

[root c37ee07bf04f /]# oc version

oc v3.6.0+c4dd4cf

kubernetes v1.6.1+5115d708d7

features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://intentionally.removed.com:8443

openshift v1.5.0+031cbe4

kubernetes v1.5.2+43a9be4


As you can see above, dry run works without problem and return a bunch
of elements for removeal but when flag --confirm is added problem appears.

At fist i thought that docker-registry pod i failing with healthchecks
but after after some investigation it is probably not the cause, at
least i don't see any events for docker-registry pod.

Is it possible to change some timeouts values for origin-master ? (I
suspect that the error is caused by origin-master which breaks
connection after ~ 277s from oc cli to registry)

I was also thinking about workaround: Use output from dry-run and pass
to some other tool (skopeo maybe?).

Have you ever faced such problem?


Best Regards,
Maciej Żarczyński


[https://www.adbglobal.com/wp-content/uploads/adb.png]
adbglobal.com<https://www.adbglobal.com>
[https://www.adbglobal.com/wp-content/uploads/linkedin_logo.png]<https://www.linkedin.com/company/adb/>         [https://www.adbglobal.com/wp-content/uploads/twitter_logo.png] <https://twitter.com/adb_global>        [https://www.adbglobal.com/wp-content/uploads/pinterest_logo.png] <https://pinterest.com/adbglobal/pins/>

_______________________________________________
users mailing list
users lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users



--
Ben Parees | OpenShift



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]