[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Aggregating container logs using Kibana





On Tue, Apr 5, 2016 at 11:50 AM, Lorenz Vanthillo <lorenz vanthillo outlook com> wrote:
This are all the steps I'm performing:

oc new-project logging

$ oc secrets new logging-deployer nothing=/dev/null

$ oc process logging-deployer-account-template -n openshift \ | oc create -f -

$ oc policy add-role-to-user edit --serviceaccount logging-deployer $ oc policy add-role-to-user daemonset-admin --serviceaccount logging-deployer $ oadm policy add-cluster-role-to-user oauth-editor \ system:serviceaccount:logging:logging-deployer

$ oadm policy add-scc-to-user \ privileged system:serviceaccount:logging:aggregated-logging-fluentd

$ oadm policy add-cluster-role-to-user cluster-reader \ system:serviceaccount:logging:aggregated-logging-fluentd

Than I execute the deployer template:

$ oc process logging-deployer-template -n openshift \ -v KIBANA_HOSTNAME=kibana.example.com,ES_CLUSTER_SIZE=1,PUBLIC_MASTER_URL=https://localhost:8443 \ | oc create -f -

This creates 3 logging-fluentd pods (I have 3 nodes, 1 unschedulable on master machine) and some empty services (the logs of the pods are telling me the permission error)
When I check oc edit scc privileged and oc edit scc hostmount-anyuid it's all fine.

$ oc label nodes --all logging-infra-fluentd=true

I've edited /master/master-config.yaml + restart
$ oc scale dc/logging-kibana --replicas=2


$ oc delete oauthclient/kibana-proxy $ oc process logging-support-template | oc create -f -

The last step creates also some pods. It's a bit weird for me that this step is only mentioned for troubleshooting or is
it an issue that I don't have those pods after executing the deployer-template?
The template 'logging-support-template' creates your ImageStreams (along with your routes and oauthclient) so it shouldn't be creating your pods.  There may have been a delay in scheduling your pods initially or the image stream tags could have been in the processes of being fetched.

What does the following output?
oc get is, svc, pods, daemonset, dc, routes, oauthclient -n logging

And do you still see the same permission denied errors in the Fluentd logs?
 


From: lorenz vanthillo outlook com
To: lmeyer redhat com
CC: users lists openshift redhat com
Subject: RE: Aggregating container logs using Kibana
Date: Tue, 5 Apr 2016 18:00:02 +0200


I still have the same issue:

I've deleted it from scc hostmount-anyuid and added it on scc privileged.
I've deleted all fluentd pods but still the same issue. Even after recreating the project.


From: lmeyer redhat com
Date: Tue, 5 Apr 2016 10:29:04 -0400
Subject: Re: Aggregating container logs using Kibana
To: lorenz vanthillo outlook com
CC: users lists openshift redhat com



On Tue, Apr 5, 2016 at 10:26 AM, Luke Meyer <lmeyer redhat com> wrote:

2016-04-05 10:55:13 +0000 [error]: unexpected error error_class=Errno::EACCES error=#<Errno::EACCES: Permission denied - /var/log/es-containers.log.pos>

This looks like https://github.com/openshift/origin-aggregated-logging/issues/89 - keeps fluentd from reading any logs on the node.

You should be able to resolve this by adding the fluentd service account to the privileged SCC, then having fluentd restart everywhere.

 oadm policy add-scc-to-user privileged system:serviceaccount:logging:aggregated-logging-fluentd

Oh; probably need to also remove them from the  hostmount-anyuid SCC.
 

_______________________________________________
users mailing list
users lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]