[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: prune images error - 503





On Wed, Oct 26, 2016 at 12:09 AM, Lionel Orellana <lionelve gmail com> wrote:
It might be trying to go through the proxy. 

If you have one configured it might, can you try without it?

The logs from the docker registry does not look like are connected with pruning.
Usually you'll see a direct messages on each side of the call, that's not happening
in your case.

 

From the master this works.

curl -v --noproxy 172.19.38.253 http://172.19.38.253:5000/healthz

On 26 October 2016 at 09:04, Lionel Orellana <lionelve gmail com> wrote:
Hi Maciej

Here's what I got from the prune logs.

I1026 08:54:32.200913   20095 prune.go:794] Using registry: 172.19.38.253:5000
I1026 08:54:32.200938   20095 prune.go:181] Trying https for 172.19.38.253:5000
I1026 08:54:32.200995   20095 round_trippers.go:296] GET https://172.19.38.253:5000/healthz
I1026 08:54:32.201010   20095 round_trippers.go:303] Request Headers:
I1026 08:54:32.201021   20095 round_trippers.go:306]     Authorization: Basic dW51c2VkOk53cmwtZ1FkM0x3Vm9CQU9hai1VdnZkTkljWG1NQ2QwRUN3UFVIN1c1LU0=
I1026 08:55:32.483709   20095 round_trippers.go:321] Response Status:  in 60282 milliseconds
I1026 08:55:32.483838   20095 prune.go:186] Error with https for 172.19.38.253:5000: Get https://172.19.38.253:5000/healthz: Service Unavailable
I1026 08:55:32.483878   20095 prune.go:181] Trying http for 172.19.38.253:5000
I1026 08:55:32.483939   20095 round_trippers.go:296] GET http://172.19.38.253:5000/healthz
I1026 08:55:32.483961   20095 round_trippers.go:303] Request Headers:
I1026 08:55:32.483974   20095 round_trippers.go:306]     Authorization: Basic dW51c2VkOk53cmwtZ1FkM0x3Vm9CQU9hai1VdnZkTkljWG1NQ2QwRUN3UFVIN1c1LU0=
I1026 08:56:35.078825   20095 round_trippers.go:321] Response Status: 503 Service Unavailable in 62594 milliseconds
I1026 08:56:35.078925   20095 prune.go:186] Error with http for 172.19.38.253:5000: unexpected status code 503
F1026 08:56:35.079066   20095 helpers.go:110] error: error communicating with registry: unexpected status code 503
-bash-4.2$ 


The address of the registry is fine. The registry is insecure. 

On the registry logs the following came up. Not sure if it's related to the prune but it happened at the very same time.

time="2016-10-25T21:56:37.829909567Z" level=error msg="error authorizing context: authorization header required" go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=7df9debf-45eb-48fa-ae07-57a09de9b3b4 http.request.method=GET http.request.remoteaddr="172.19.39.129:56350" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c 
172.19.39.129 - - [25/Oct/2016:21:56:37 +0000] "GET /openshift/token?account=serviceaccount&scope=repository%3Atransaction-authority-lite%2Ftransaction-auth-lite%3Apull HTTP/1.1" 200 1823 "" "docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64"

time="2016-10-25T21:56:37.841655976Z" level=info msg="response completed" go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=213f9613-c430-4386-bef2-cb67a7069f52 http.request.method=GET http.request.remoteaddr="172.19.39.129:56352" http.request.uri="/openshift/token?account=serviceaccount&scope=repository%3Atransaction-authority-lite%2Ftransaction-auth-lite%3Apull" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" http.response.contenttype="application/json" http.response.duration=6.642711ms http.response.status=200 http.response.written=1823 instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c 

time="2016-10-25T21:56:37.844393921Z" level=debug msg="authorizing request" go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=fdcf1a81-7b93-4981-a957-7417386cf7de http.request.method=GET http.request.remoteaddr="172.19.39.129:56354" http.request.uri="/v2/transaction-authority-lite/transaction-auth-lite/manifests/sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c vars.name="transaction-authority-lite/transaction-auth-lite" vars.reference="sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" 

time="2016-10-25T21:56:37.844554174Z" level=debug msg="Origin auth: checking for access to repository:transaction-authority-lite/transaction-auth-lite:pull" go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=fdcf1a81-7b93-4981-a957-7417386cf7de http.request.method=GET http.request.remoteaddr="172.19.39.129:56354" http.request.uri="/v2/transaction-authority-lite/transaction-auth-lite/manifests/sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c vars.name="transaction-authority-lite/transaction-auth-lite" vars.reference="sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" 

time="2016-10-25T21:56:37.852266154Z" level=debug msg=GetImageManifest go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=fdcf1a81-7b93-4981-a957-7417386cf7de http.request.method=GET http.request.remoteaddr="172.19.39.129:56354" http.request.uri="/v2/transaction-authority-lite/transaction-auth-lite/manifests/sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c vars.name="transaction-authority-lite/transaction-auth-lite" vars.reference="sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" 
172.19.39.129 - - [25/Oct/2016:21:56:37 +0000] "GET /v2/transaction-authority-lite/transaction-auth-lite/manifests/sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b HTTP/1.1" 200 40788 "" "docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64"

time="2016-10-25T21:56:37.930862363Z" level=info msg="response completed" go.version=go1.6.3 http.request.host="172.19.38.253:5000" http.request.id=fdcf1a81-7b93-4981-a957-7417386cf7de http.request.method=GET http.request.remoteaddr="172.19.39.129:56354" http.request.uri="/v2/transaction-authority-lite/transaction-auth-lite/manifests/sha256:591f586760735ddf513b8bad2f98aaa77d079050d6f0bd392eb4b93d6aa3e21b" http.request.useragent="docker/1.10.3 go/go1.6.2 git-commit/2a93377-unsupported kernel/3.10.0-327.28.3.el7.x86_64 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.v1+prettyjws" http.response.duration=87.849901ms http.response.status=200 http.response.written=40788 instance.id=9e99450d-8197-4cce-ab17-1cf256d32a7c 


Thanks

Lionel.

On 25 October 2016 at 20:53, Maciej Szulik <maszulik redhat com> wrote:
Upon pruning images this command reaches out to registry to actually
remove them from there. Apparently, it has has problems reaching out
to your registry. There are two flags you might want to check out
--certificate-authority - which points to a CA when self-signed cert is used
--registry-url - to specify which registry to talk to.

Additionally you can increase the verbosity (--loglevel=7) and see the communication
between oadm prune images and the server, including the registry. If still experiencing
problems, please lemme know with that logs attached, so I could help more.

Maciej

On Tue, Oct 25, 2016 at 9:29 AM, Lionel Orellana <lionelve gmail com> wrote:
Hi.

I'm trying to run 

oadm prune images --keep-tag-revisions=3 --keep-younger-than=24h --confirm 

but after a while I get 

error: error communicating with registry: unexpected status code 503

I am able to prune builds and deployments. prune images without --confirm also returns Ok. 

I've been building a ~6GB image repeatedly in the last couple of days. Seems like it's struggling to delete some of those? 

Thanks

Lionel 

_______________________________________________
users mailing list
users lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users






[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]