[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: How to use extra trusted CA certs when pulling images for a builder

On Wed, 13 Nov 2019 at 01:34, Ben Parees <bparees redhat com> wrote:

On Tue, Nov 12, 2019 at 3:45 AM Joel Pearson <japearson agiledigital com au> wrote:

On Tue, 12 Nov 2019 at 15:37, Ben Parees <bparees redhat com> wrote:

On Mon, Nov 11, 2019 at 11:26 PM Joel Pearson <japearson agiledigital com au> wrote:
I've now discovered that the cluster-samples-operator doesn't seem honour the proxy settings, and I see lots of errors in the cluster-samples-operator-xxxx pod logs

time="2019-11-12T04:15:49Z" level=warning msg="Image import for imagestream dotnet tag 2.1 generation 2 failed with detailed message Internal error occurred: Get https://I /v2/: x509: certificate signed by unknown authority"

Is there a way to get that operator to use the same user-ca-bundle?

image import should be using those CAs (it's really about the openshift-apiserver, not the samples operator) automatically (sounds like another potential bug, but i'll let Oleg weigh in on this one).  

However barring that, you can use the mechanism described here to setup additional CAs for importing from registries:

you can follow the more detailed instructions here:

I tried this approach but it didn't work for me.

I ran this command:

oc create configmap registry-cas -n openshift-config \
--from-file=registry.redhat.io..5000=/path/to/ca.crt \
--from-file=registry.redhat.io..443=/path/to/ca.crt \


oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-cas"}}}' --type=merge

And that still didn't work. First I deleted the cluster-samples-operator-xxxx pod, then I tried forcing the masters to restart by touching some machine config (I don't know a better way).
But it still didn't work.  Maybe the samples operator doesn't let you easily override the trusted CA certs?

Because no good bug report should not be rewarded with some educational background:

The samples operator is only responsible for creating the imagestream, it isn't actually doing the import (ie reaching out to the registry and pulling down the metadata and putting it in the imagestream).  That task is performed by the openshift-apiserver.  What should be happening when you update the image config resource with the name of the CA configmap is that the openshift-apiserver operator should observe the configuration change and provide the new CAs to the openshift-apiserver pods (which necessitates a restart of the openshift-apiserver pods).

Once the openshift-apiserver pods are restarted with the new CAs, you should be able to run "oc import-image" to retry the import.  (The samples operator is supposed to retry the failed imports periodically, but there is a different bug that is being fixed related to that, so until then you'll have to retry the import manually once you've corrected whatever caused the failure).

So again, there may be a bug here in terms of the openshift-apiserver picking up the CAs and we need to investigate it (as well as a separate bug if it is not picking up the proxy CAs), but I wanted you to understand the relevant components so your own debugging process can be more productive.

Thanks for the explanation Ben, it helped me figure out there the issues where. 

For other reasons (I had tried to customise name of the Azure resource group and something would occasionally (once a day?) change it back to the default name), I ended up burning the cluster to the ground, and I configured the "spec.trustedCA.name" reference during installation by customising the manifest files generated by "openshift-install create manifests --dir=ignition-files" and then the samples operators worked out of the box!

So it was just that the samples operator doesn't retry as you mentioned.

Also, I didn't need to setup the additional CAs for importing from registries in the end.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]