System logs are empty (/var/log/messages; /var/log/secure; etc)

BANJOSA asked:

I found that rsyslog stopped writing on logs (messages; secure; cron;etc)

System information:
NAME=”Red Hat Enterprise Linux Server”
VERSION=”7.4 (Maipo)”
ID=”rhel”
ID_LIKE=”fedora”
VARIANT=”Server”
VARIANT_ID=”server”
VERSION_ID=”7.4″

already restarted rsyslog with no results. Other logs processed by rsyslogd are being writen without issues.

Also restarted systemd-journald.

rsyslog.conf (Omitting commented lines):

$ModLoad imuxsock
$ModLoad imjournal
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
local3.*                                                /var/log/sshd-second.log

Journald.conf (Omitting commented lines):

[Journal]

Already deleted /run/log/journal/* and restarted journald
Already deleted imjournal.state and restarted rsyslog

output of the command rsyslogd -N 1:

rsyslogd: version 8.24.0, config validation run (level 1), master config /etc/rsyslog.conf

rsyslogd: End of config validation run. Bye.

output of the command rsyslogd -N 6:

rsyslogd: version 8.24.0, config validation run (level 6), master config /etc/rsyslog.conf

rsyslogd: End of config validation run. Bye.

Already tried all the solutions described in https://unix.stackexchange.com/questions/124942/rsyslog-not-logging with no results

Output of lsof -p

COMMAND   PID USER   FD      TYPE             DEVICE SIZE/OFF     NODE NAME
rsyslogd 5820 root  cwd       DIR              202,2      236       64 /
rsyslogd 5820 root  rtd       DIR              202,2      236       64 /
rsyslogd 5820 root  txt       REG              202,2   663872   552421 /usr/sbin/rsyslogd
rsyslogd 5820 root  mem       REG               0,18  8388608    27215 /run/log/journal/ca23e130dda846d5b2a30e4ab9461e43/system.journal
rsyslogd 5820 root  mem       REG              202,2    29352 13067688 /usr/lib64/rsyslog/imudp.so
rsyslogd 5820 root  mem       REG              202,2    68192    34595 /usr/lib64/libbz2.so.1.0.6
rsyslogd 5820 root  mem       REG              202,2    99944    34673 /usr/lib64/libelf-0.168.so
rsyslogd 5820 root  mem       REG              202,2   402384    34477 /usr/lib64/libpcre.so.1.2.0
rsyslogd 5820 root  mem       REG              202,2    19888    34705 /usr/lib64/libattr.so.1.1.0
rsyslogd 5820 root  mem       REG              202,2   297328    45575 /usr/lib64/libdw-0.168.so
rsyslogd 5820 root  mem       REG              202,2   111080  1274500 /usr/lib64/libresolv-2.17.so
rsyslogd 5820 root  mem       REG              202,2    19384    34688 /usr/lib64/libgpg-error.so.0.10.0
rsyslogd 5820 root  mem       REG              202,2   535064    34703 /usr/lib64/libgcrypt.so.11.8.2
rsyslogd 5820 root  mem       REG              202,2   157400    34499 /usr/lib64/liblzma.so.5.2.2
rsyslogd 5820 root  mem       REG              202,2   155752    34476 /usr/lib64/libselinux.so.1
rsyslogd 5820 root  mem       REG              202,2  1139680    34473 /usr/lib64/libm-2.17.so
rsyslogd 5820 root  mem       REG              202,2    20032    34709 /usr/lib64/libcap.so.2.22
rsyslogd 5820 root  mem       REG              202,2    24928 13067682 /usr/lib64/rsyslog/imjournal.so
rsyslogd 5820 root  mem       REG              202,2    38032 13067689 /usr/lib64/rsyslog/imuxsock.so
rsyslogd 5820 root  mem       REG              202,2    24416 13067690 /usr/lib64/rsyslog/lmnet.so
rsyslogd 5820 root  mem       REG              202,2  2127336    23375 /usr/lib64/libc-2.17.so
rsyslogd 5820 root  mem       REG              202,2    88720  1233870 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
rsyslogd 5820 root  mem       REG              202,2    20040    35554 /usr/lib64/libuuid.so.1.3.0
rsyslogd 5820 root  mem       REG              202,2    40824   374355 /usr/lib64/libfastjson.so.4.0.0
rsyslogd 5820 root  mem       REG              202,2    15424   392270 /usr/lib64/libestr.so.0.0.0
rsyslogd 5820 root  mem       REG              202,2    44448    23398 /usr/lib64/librt-2.17.so
rsyslogd 5820 root  mem       REG              202,2    19776    34471 /usr/lib64/libdl-2.17.so
rsyslogd 5820 root  mem       REG              202,2   144792  1274481 /usr/lib64/libpthread-2.17.so
rsyslogd 5820 root  mem       REG              202,2    90632    34489 /usr/lib64/libz.so.1.2.7
rsyslogd 5820 root  mem       REG              202,2   164112    23368 /usr/lib64/ld-2.17.so
rsyslogd 5820 root  mem       REG              202,2   162560     3600 /usr/lib64/libsystemd.so.0.6.0
rsyslogd 5820 root    0r      CHR                1,3      0t0     1041 /dev/null
rsyslogd 5820 root    1w      CHR                1,3      0t0     1041 /dev/null
rsyslogd 5820 root    2w      CHR                1,3      0t0     1041 /dev/null
rsyslogd 5820 root    3u     IPv4              28378      0t0      UDP *:syslog
rsyslogd 5820 root    4u     IPv6              28379      0t0      UDP *:syslog
rsyslogd 5820 root    5r  a_inode                0,9        0     5987 inotify
rsyslogd 5820 root    6u     unix 0xffff8800da61a400      0t0    28380 socket
rsyslogd 5820 root    7r      REG               0,18  8388608    27215 /run/log/journal/ca23e130dda846d5b2a30e4ab9461e43/system.journal
rsyslogd 5820 root    8u  a_inode                0,9        0     5987 [eventpoll]
rsyslogd 5820 root    9w      REG              202,2   193240     8457 /var/log/haproxy.log

Does anyone have any clues?

My answer:


A bug fix for a problem where rsyslog would stop writing entries from the systemd journal was released in errata RHBA-2017:2306. Ensure that your system is up to date.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.