[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Logging seems to be working, but no logs are collected



Thanks. Those links are useful.

It looks to me like its a problem at the fluentd level. This is what I see on on of the fluentd pods:

sh-4.2# cat /var/log/es-containers.log.pos                                                                                                  
cat: /var/log/es-containers.log.pos: No such file or directory                                                                              
sh-4.2# cat /var/log/journal.pos                                                                                                            
s=52fdd277f90749b0a442c78739b1efa7;i=50d69;b=2a3f1736a1a1486d83f95db719fdc281;m=5465b53fd1;t=55cdac4738846;x=85596f3f5f5a27e4sh-4.2#                                                                                                                                   
sh-4.2# journalctl -c `cat /var/log/journal.pos`                                                                                            
No journal files were found.                                                                                                                
-- No entries --                                                                                                                            

Which might sort of explain why everything is running but no logs are being processed.

This is based on a centos7 image with only the necessary openshift packages installed and then openshift installed using ansible. The logging setup in the inventory file is this:

openshift_hosted_logging_deployer_version=v3.6.0
openshift_hosted_logging_deploy=true
openshift_hosted_logging_storage_kind=nfs
openshift_hosted_logging_storage_access_modes=['ReadWriteOnce']
openshift_hosted_logging_storage_nfs_directory=/exports
openshift_hosted_logging_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_logging_storage_volume_name=logging
openshift_hosted_logging_storage_volume_size=10Gi
openshift_hosted_logging_storage_labels={'storage': 'logging'}
 

Tim


On 31/10/2017 16:37, Jeff Cantrill wrote:
Please provide additional information, logs, etc or post the output of [1] someplace for review.  Additionally, consider reviewing [2].


On Tue, Oct 31, 2017 at 11:47 AM, Tim Dudgeon <tdudgeon ml gmail com> wrote:
Hi All,

I've deployed logging using the ansible installer (v3.6.0) for a fairly simple openshift setup and everything appears to running:

NAME                                       READY     STATUS RESTARTS   AGE
logging-curator-1-gvh73                    1/1       Running 24         3d
logging-es-data-master-xz0e7a0c-1-deploy   0/1       Error 0          3d
logging-es-data-master-xz0e7a0c-4-deploy   0/1       Error 0          3d
logging-es-data-master-xz0e7a0c-5-deploy   0/1       Error 0          3d
logging-es-data-master-xz0e7a0c-7-t4xpf    1/1       Running 0          3d
logging-fluentd-4rm2w                      1/1       Running 0          3d
logging-fluentd-8h944                      1/1       Running 0          3d
logging-fluentd-n00bn                      1/1       Running 0          3d
logging-fluentd-vt8hh                      1/1       Running 0          3d
logging-kibana-1-g7l4z                     2/2       Running 0          3d

(the failed pods were related to getting elasticsearch running, but that was resolved).

The problem is that I don't see any logs in Kibana. When I look in the fluentd pod logs I see lots of stuff like this:

2017-10-31 13:53:15 +0000 [warn]: no patterns matched tag="journal.system"
2017-10-31 13:58:02 +0000 [warn]: no patterns matched tag="kubernetes.journal.container"
2017-10-31 14:02:18 +0000 [warn]: no patterns matched tag="journal.system"
2017-10-31 14:07:15 +0000 [warn]: no patterns matched tag="journal.system"
2017-10-31 14:11:20 +0000 [warn]: no patterns matched tag="journal.system"
2017-10-31 14:15:16 +0000 [warn]: no patterns matched tag="journal.system"
2017-10-31 14:19:58 +0000 [warn]: no patterns matched tag="journal.system"

Is this the cause, and if so what is wrong?
If not how to debug this?

Tim



_______________________________________________
users mailing list
users lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users



--
--
Jeff Cantrill
Senior Software Engineer, Red Hat Engineering
OpenShift Integration Services
Red Hat, Inc.
Office: 703-748-4420 | 866-546-8970 ext. 8162420
jcantril redhat com
http://www.redhat.com



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]