[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

aggregate logging: kibana error



Dear community,
I followed the guide https://docs.openshift.org/latest/install_config/aggregate_logging.html

NAME                          READY     STATUS    RESTARTS   AGE
logging-kibana-1-uwob1        1/2       Error     12         43m


$ oc logs logging-kibana-1-uwob1  -c kibana
{"name":"Kibana","hostname":"logging-kibana-1-uwob1","pid":7,"level":50,"err":{"message":"Request Timeout after 5000ms","name":"Error","stack":"Error: Request Timeout after 5000ms\n    at null.<anonymous> (/opt/app-root/src/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-04-20T07:16:15.760Z","v":0}
{"name":"Kibana","hostname":"logging-kibana-1-uwob1","pid":7,"level":60,"err":{"message":"Request Timeout after 5000ms","name":"Error","stack":"Error: Request Timeout after 5000ms\n    at null.<anonymous> (/opt/app-root/src/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-04-20T07:16:15.762Z","v":0}
[root MRNZ-TS8-OC-MASTER-01 glusterfs]# oc logs logging-kibana-1-uwob1  -c kibana
{"name":"Kibana","hostname":"logging-kibana-1-uwob1","pid":7,"level":50,"err":{"message":"Request Timeout after 5000ms","name":"Error","stack":"Error: Request Timeout after 5000ms\n    at null.<anonymous> (/opt/app-root/src/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-04-20T07:38:40.789Z","v":0}
{"name":"Kibana","hostname":"logging-kibana-1-uwob1","pid":7,"level":60,"err":{"message":"Request Timeout after 5000ms","name":"Error","stack":"Error: Request Timeout after 5000ms\n    at null.<anonymous> (/opt/app-root/src/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-04-20T07:38:40.790Z","v":0}


Elastic search pod is running, but the log shows:
[2016-04-20 06:57:03,910][ERROR][io.fabric8.elasticsearch.plugin.acl.DynamicACLFilter] [Baphomet] Exception encountered when seeding initial ACL
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:151)
        at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.checkGlobalBlock(TransportShardSingleOperationAction.java:103)
        at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.<init>(TransportShardSingleOperationAction.java:132)
        at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.<init>(TransportShardSingleOperationAction.java:116)

Fluent pod is running too, but the log shows:
2016-04-20 07:47:18 +0000 [error]: fluentd main process died unexpectedly. restarting.
2016-04-20 07:47:48 +0000 [error]: unexpected error error="getaddrinfo: Name or service not known"
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:878:in `initialize'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:878:in `open'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:878:in `block in connect'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/timeout.rb:52:in `timeout'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:877:in `connect'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:862:in `do_start'
  2016-04-20 07:47:48 +0000 [error]: /usr/share/ruby/net/http.rb:851:in `start'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in `transmit'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in `execute'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in `execute'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/rest-client-1.8.0/lib/restclient/resource.rb:51:in `get'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/kubeclient-1.1.2/lib/kubeclient/common.rb:310:in `block in api'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/kubeclient-1.1.2/lib/kubeclient/common.rb:51:in `handle_exception'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/kubeclient-1.1.2/lib/kubeclient/common.rb:309:in `api'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/kubeclient-1.1.2/lib/kubeclient/common.rb:304:in `api_valid?'
  2016-04-20 07:47:48 +0000 [error]: /opt/app-root/src/gems/fluent-plugin-kubernetes_metadata_filter-0.18.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:134:in `configure’
2016-04-20 07:50:42 +0000 [warn]: emit transaction failed: error_class=Fluent::ConfigError error="Exception encountered fetching metadata from Kubernetes API endpoint: getaddrinfo: Name or service not known" tag="kubernetes.var.log.containers.docker-registry-2-9tehj_default_registry-957340d6e4686b63bdcdccf18f1b0b4054d1faab9724097928f8102c3190f312.log"
  2016-04-20 07:50:42 +0000 [warn]: /opt/app-root/src/gems/fluent-plugin-kubernetes_metadata_filter-0.18.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:232:in `rescue in start_watch'
  2016-04-20 07:50:42 +0000 [warn]: /opt/app-root/src/gems/fluent-plugin-kubernetes_metadata_filter-0.18.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:228:in `start_watch'
  2016-04-20 07:50:42 +0000 [warn]: /opt/app-root/src/gems/fluent-plugin-kubernetes_metadata_filter-0.18.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:140:in `block in configure'
2016-04-20 07:51:11 +0000 [warn]: emit transaction failed: error_class=SocketError error="getaddrinfo: Name or service not known" tag="kubernetes.var.log.containers.router-1-94cv3_default_router-3e8bc8d6d4ef8a52f8f46a7e83b5a510a4f1c8d63658cc49b3ce506822e84811.log”



Any idea why kibana is not starting properly?

Greetings,
   Sebastian



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]