[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Second master-api fails to start



I'm installing OpenShift for the first time on internal servers that were provisioned and then configured by me. I had to host the repos internally on our Artifactory server as the internal cluster has no direct internet access. I followed all of the steps here: https://docs.openshift.org/latest/install_config/install/prerequisites.html and here: https://docs.openshift.org/latest/install_config/install/advanced_install.html. I'm installing it using openshift-ansible at commit ac4ef6bb65c24890592b6b35b11be82f4ddc6deb. These systems are RHEL7.

My configuration file looks like below with minor edits to remove internal information:
# Create an OSOv3 group that contains the master, nodes, etcd, and lb groups.
# The lb group lets Ansible configure HAProxy as the load balancing solution.
# Comment lb out if your load balancer is pre-configured.
[OSOv3:children]
masters
nodes
etcd
lb

# Set variables common for all OSOv3 hosts
[OSOv3:vars]
ansible_ssh_user=username
ansible_sudo=true
deployment_type=origin

# Uncomment the following to enable htpasswd authentication; defaults to
# DenyAllPasswordIdentityProvider.
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]

# Native high availability cluster method with optional load balancer.
# If no lb group is defined installer assumes that a load balancer has
# been preconfigured. For installation the value of
# openshift_master_cluster_hostname must resolve to the load balancer
# or to one or all of the masters defined in the inventory if no load
# balancer is present.
openshift_master_cluster_method=native
openshift_master_cluster_hostname=oshiftmst01.example.com
openshift_master_cluster_public_hostname=oshiftmst01.example.com

# host group for masters
[masters]
oshiftmst01.example.com
oshiftmst02.example.com
oshiftmst03.example.com

# host group for etcd
[etcd]
oshiftmst01.example.com
oshiftmst02.example.com
oshiftmst03.example.com

# Specify load balancer host
[lb]
oshiftinf01.example.com

# host group for nodes, includes region info
[nodes]
oshiftmst0[1:3].example.com openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
oshiftslv01.example.com
oshiftslv02.example.com
oshiftslv03.example.com
oshiftinf01.example.com

Everything is installed correctly on the mst01 host and it starts correctly. When I get to the mst02 host, it fails like this:
TASK [openshift_master : Start and enable master api] **************************
fatal: [oshiftmst02.example.com]: FAILED! => {"changed": false, "failed": true, "msg": "Job for origin-master-api.service failed because a timeout was exceeded. See \"systemctl status origin-master-api.service\" and \"journalctl -xe\" for details.\n"}

When I try to manually start it, I get this:
[username oshiftmst02 ~]$ sudo systemctl start origin-master-api.service
Job for origin-master-api.service failed because a timeout was exceeded. See "systemctl status origin-master-api.service" and "journalctl -xe" for details.

When I look closer:
Aug 09 15:48:47 oshiftmst02 systemd[1]: Starting Atomic OpenShift Master API...
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.573484   15969 start_api.go:102] Using a listen address override "0.0.0.0:8443"
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: W0809 15:48:47.576872   15969 start_master.go:270] assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: W0809 15:48:47.576910   15969 start_master.go:270] assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in t
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.587135   15969 plugins.go:71] No cloud provider specified.
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.588409   15969 genericapiserver.go:81] Adding storage destination for group
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.588443   15969 genericapiserver.go:81] Adding storage destination for group extensions
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.588472   15969 start_master.go:383] Starting master on 0.0.0.0:8443 (v1.2.1)
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.588482   15969 start_master.go:384] Public master address is https://oshiftmst01.example.com:8443
Aug 09 15:48:47 oshiftmst02 atomic-openshift-master-api[15969]: I0809 15:48:47.588511   15969 start_master.go:388] Using images from "openshift/origin-<component>:v1.2.1"
Aug 09 15:50:01 oshiftmst02 kernel: SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
Aug 09 15:50:01 oshiftmst02 systemd[1]: Created slice user-0.slice.
Aug 09 15:50:01 oshiftmst02 systemd[1]: Starting user-0.slice.
Aug 09 15:50:01 oshiftmst02 systemd[1]: Started Session 10668 of user root.
Aug 09 15:50:01 oshiftmst02 systemd[1]: Starting Session 10668 of user root.
Aug 09 15:50:01 oshiftmst02 systemd[1]: Removed slice user-0.slice.
Aug 09 15:50:01 oshiftmst02 systemd[1]: Stopping user-0.slice.
Aug 09 15:50:17 oshiftmst02 systemd[1]: origin-master-api.service start operation timed out. Terminating.
Aug 09 15:50:17 oshiftmst02 systemd[1]: Failed to start Atomic OpenShift Master API.
Aug 09 15:50:17 oshiftmst02 systemd[1]: Unit origin-master-api.service entered failed state.
Aug 09 15:50:17 oshiftmst02 systemd[1]: origin-master-api.service failed.

I thought that maybe the access wasn't open between masters, but curl works:
[username oshiftmst02 ~]$ curl --insecure https://oshiftmst01.example.com:8443
{
  "paths": [
    "/api",
    "/api/v1",
    "/apis",
    "/controllers",
    "/healthz",
    "/healthz/ping",
    "/healthz/ready",
    "/metrics",
    "/oapi",
    "/oapi/v1",
    "/swaggerapi/"
  ]
}

I'm assuming that I'm just not understanding the system well enough. Any help would be appreciated.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]