Rsyslog Server over TCP Port 514 Link to heading

Set up Link to heading

  • Virtual machines:
Log Managed ServerClient
OSCentOS 6.9CentOS 6.9
Hostnameserver.flemingcollege.caclient.flemingcollege.ca
IP address192.168.1.100/24192.168.1.101/24
  • Domain name: flemingcollege.ca

1. Configure server to receive logs from client. Link to heading

[root@server ~]$ vi /etc/rsyslog.conf
# Line 17, 18: uncomment 2 lines
$ModLoad imtcp
$InputTCPServerRun 514

# Line 19: add senders you permit to access
$AllowedSender TCP, 127.0.0.1, 192.168.1.0/24, *.flemingcollege.ca

[root@server ~]$ service rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]

2. Add iptables rule and configure SELinux to allow Rsyslog traffic. Link to heading

[root@server ~]$ iptables -I INPUT 5 -m state --state NEW -m tcp -p tcp --dport 514 -j ACCEPT
[root@server ~]$ service iptables save
[root@server ~]$ service iptables restart
[root@server ~]$ semanage port -a -t syslogd_port_t -p tcp 514

3. Configure client. Link to heading

[root@client ~]$ vi /etc/rsyslog.conf
# Line 72: uncomment all
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1   # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g     # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on   # save messages to disk on shutdown
$ActionQueueType LinkedList     # run asynchronously
$ActionResumeRetryCount -1      # infinite retries if host is down
# Add to the end of the file to send only authentication messages to server
authpriv.*    @@server.flemingcollege.ca:514
# Or
authpriv.*    @@192.168.1.100:514
# We can send more if we need, all the options are in the config file
kern.*     # Log all kernel messages                                                
mail.*     # Log all mail messages                                                
cron.*     # Log cron stuff
*.emerg    # Log emergency messages
*.info;mail.none;authpriv.none
# Log anything (except mail and authentication) of level info or higher.
local7.*   # Log boot messages


[root@client ~]$ service rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]

4. After configuration, we can test by connecting SSH to both machine and logs of authentication will be sent to the default directory /var/log/secure on server. Link to heading

[root@server ~]$ tail -10 /var/log/secure
Nov  8 07:17:26 localhost unix_chkpwd[2838]: password check failed for user (root)
Nov  8 07:17:29 localhost sshd[2836]: Failed password for root from 192.168.1.125 port 22116 ssh2
Nov  8 07:17:49 localhost sshd[2837]: Connection closed by 192.168.1.125
Nov 10 07:19:49 client sshd[2534]: Invalid user 3 from 192.168.1.125
Nov 10 07:19:49 client sshd[2535]: input_userauth_request: invalid user 3
Nov 10 07:19:50 client sshd[2534]: pam_unix(sshd:auth): check pass; user unknown
Nov 10 07:19:50 client sshd[2534]: pam_succeed_if(sshd:auth): error retrieving information about user 3
Nov 10 07:19:52 client sshd[2534]: Failed password for invalid user 3 from 192.168.1.125 port 22119 ssh2

Optional Link to heading

Option 1: Separate logs to different files. Link to heading

We can separate logs which are sent to the server to different files, sort by hostname or date.

[root@server ~]$ vi /etc/rsyslog.conf
# Add to define log file's names and directory
$template Secure_log,"/var/log/secure.d/%fromhost%_%$year%%$month%%$day%.secure"

# Add to save authentication messages to log files
authpriv.*     -?Secure_log

[root@server ~]$ service rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]

[root@server ~]$ ll /var/log/secure.d
total 12
-rw-------. 1 root root 4181 Nov  8 07:58 client_20171108.secure
-rw-------. 1 root root 1330 Nov  8 07:17 localhost_20171108.secure

Option 2: Save logs to databases. Link to heading

  1. Install MySQL and create password for root.
  2. Create database and user for Rsyslog to save logs.
[root@server ~]$ yum -y install rsyslog-mysql

# Create database for rsyslog
[root@server ~]$ cat /usr/share/doc/rsyslog-mysql-*/createDB.sql | mysql -u root -p
Enter password: # Enter password for root

[root@server ~]$ mysql -u root -p
Enter password: # Enter password for root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

# Create a user named "rsyslog" and grant all privileges to him to access Syslog database (change your own 'p@ssw0rd')
mysql> grant all privileges on Syslog.* to rsyslog@'localhost' identified by 'p@ssw0rd';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye
  1. Configure Rsyslog to save logs to database.
[root@server ~]$ vi /etc/rsyslog.conf
# Near line 20: add to save logs to database
$ModLoad ommysql

# Add to save authentication messages to log files
# How to write :ommysql:Hostname,Database,MysqlUser,MysqlPassword
authpriv.*     :ommysql:localhost,Syslog,rsyslog,password

[root@server ~]$ service rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]
  1. Now we can test by connecting SSH to both machine and logs of authentication will be sent to the Syslog database on server.
[root@server ~]$ mysql -u rsyslog -p Syslog
Enter password: # Enter password for rsyslog
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show tables;
+------------------------+
| Tables_in_Syslog       |
+------------------------+
| SystemEvents           |
| SystemEventsProperties |
+------------------------+
2 rows in set (0.00 sec)

mysql> select ReceivedAt,Facility,Priority,FromHost,Message from SystemEvents;
+---------------------+----------+----------+-----------+------------------------------------------------------------------------+
| ReceivedAt          | Facility | Priority | FromHost  | Message                                                                |
+---------------------+----------+----------+-----------+------------------------------------------------------------------------+
| 2017-11-08 07:17:26 |       10 |        5 | localhost |  password check failed for user (root)                                 |
| 2017-11-08 07:17:29 |       10 |        6 | localhost |  Failed password for root from 192.168.1.125 port 22116 ssh2           |
| 2017-11-08 07:17:49 |       10 |        6 | localhost |  Connection closed by 192.168.1.125                                    |
| 2017-11-08 07:19:41 |       10 |        6 | client    |  Invalid user 3 from 192.168.1.125                                     |
| 2017-11-08 07:19:41 |       10 |        6 | client    |  input_userauth_request: invalid user 3                                |
| 2017-11-08 07:19:42 |       10 |        4 | client    |  pam_unix(sshd:auth): check pass; user unknown                         |
| 2017-11-08 07:19:42 |       10 |        2 | client    |  pam_succeed_if(sshd:auth): error retrieving information about user 3  |
| 2017-11-08 07:19:44 |       10 |        6 | client    |  Failed password for invalid user 3 from 192.168.1.125 port 22119 ssh2 |
+---------------------+----------+----------+-----------+------------------------------------------------------------------------+
10 rows in set (0.00 sec)

Rsyslog Server for Cisco Devices Link to heading

Configuration Link to heading

  • Enable NTP in the Rsyslog server
[root@server ~]$ vi /etc/ntp.conf
server 127.127.1.0
fudge 127.127.1.0 stratum 10
  • Configure the router to sync time with the Rsyslog server
R1(config)# ntp authentication-key 1 md5
R1(config)# ntp server 192.168.1.100 key 1
R1(config)# ntp trusted-key 1
R1(config)# ntp authenticate
*Dec  5 22:07:36.198: NTP Core(INFO): keys initilized.
*Dec  5 22:07:36.198: NTP Core(NOTICE): proto: precision =  usec
*Dec  5 22:07:36.198: %NTP : Drift Read : FFFFFFFF.FFFFF470
*Dec  5 22:07:36.198: NTP Core(DEBUG): drift value read: -0.000000000
*Dec  5 22:07:36.198: NTP Core(NOTICE): ntpd  PPM
*Dec  5 22:07:36.198: NTP: Initialized interface Embedded-Service-Engine0/0
*Dec  5 22:07:36.198: NTP: Initialized interface GigabitEthernet0/0
*Dec  5 22:07:36.198: NTP: Initialized interface GigabitEthernet0/1
*Dec  5 22:07:36.198: NTP: Initialized interface GigabitEthernet0/2
*Dec  5 22:07:36.198: NTP: Initialized interface Backplane-GigabitEthernet0/3
*Dec  5 22:07:36.198: NTP: Initialized interface Serial0/0/0
*Dec  5 22:07:36.198: NTP: Initialized interface Serial0/0/1
*Dec  5 22:07:36.198: NTP: Initialized interface RG-AR-IF-INPUT1
*Dec  5 22:07:36.198: NTP: Initialized interface VoIP-Null0
*Dec  5 22:07:36.198: NTP Core(INFO): more memory added for keys.
*Dec  5 22:07:36.198: NTP Core(INFO): key (1) added.
*Dec  5 22:07:36.198: NTP Core(INFO): key (1) marked as trusted.
*Dec  5 22:07:37.198: NTP message sent to 192.168.1.100, from interface 'NULL' (0.0.0.0).
*Dec  5 22:07:37.198: NTP message received from 192.168.1.100 on interface 'GigabitEthernet0/0' (192.168.1.1).
*Dec  5 22:07:37.198: NTP Core(DEBUG): ntp_receive: message received
*Dec  5 22:07:37.198: NTP Core(DEBUG): ntp_receive: peer is 0x22606EE0, next action is 1.
*Dec  5 22:07:37.198: NTP Core(DEBUG): Peer becomes reachable, poll set to 6.
*Dec  5 22:07:37.198: NTP Core(INFO): 192.168.1.100 E014 84 reachable
*Dec  5 22:07:37.198: NTP Core(INFO): 192.168.1.100 F02D 8D popcorn popcorn
*Dec  5 22:07:38.322: NTP Core(INFO): 0.0.0.0 C01C 0C clock_step
...
*Dec  6 02:21:14.444: NTP Core(NOTICE): Clock is synchronized.

R1# show ntp association
  address         ref clock       st   when   poll reach  delay  offset   disp
*~192.168.1.100   127.127.1.0     11     13     64     1  1.234  -0.544 3937.6
 * sys.peer, # selected, + candidate, - outlyer, x falseticker, ~ configured

R1# show ntp status
Clock is synchronized, stratum 12, reference is 192.168.1.100
nominal freq is 250.0000 Hz, actual freq is 250.0000 Hz, precision is 2**20
ntp uptime is 12800 (1/100 of seconds), resolution is 4000
reference time is DDD1D29A.722F8839 (02:21:14.446 UTC Wed Dec 6 2017)
clock offset is -0.5441 msec, root delay is 1.23 msec
root dispersion is 7887.42 msec, peer dispersion is 3937.65 msec
loopfilter state is 'CTRL' (Normal Controlled Loop), drift is 0.000000000 s/s
system poll interval is 64, last update was 17 sec ago.
  • Create a log file which store all router’s logs in the Rsyslog server
[root@server ~]$ cd /var/log

[root@server ~]$ touch cisco

[root@server ~]$ vi /etc/syslog.conf
# Cisco routers use the local7 facility
local7.info /var/log/cisco

[root@server ~]$ /etc/init.d/syslog restart
  • Configure the router to send log messages to the Rsyslog server
R1(config)# logging ip_address_of_server
R1(config)# logging trap
R1(config)# logging on
R1(config)# service timestamps log datetime

Verification Link to heading

[root@server ~]$ cat /var/log/cisco.log
Nov 22 11:30:25 172.16.100.254 86: Nov 22 16:30:09: %SYS-5-CONFIG_I: Configured from console by console
Nov 22 11:30:25 172.16.100.254 87: Nov 22 16:30:10: %SYS-3-LOGGINGHOST_FAIL: Logging to host 172.16.100.100 port 514 failed
Nov 22 11:30:25 172.16.100.254 88: Nov 22 16:30:15: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 172.16.100.100 port 514 started - reconnection
Nov 22 11:34:29 172.16.100.254 90: Nov 22 16:34:13: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
Nov 22 11:34:29 172.16.100.254 91: Nov 22 16:34:14: %SYS-3-LOGGINGHOST_FAIL: Logging to host 172.16.100.100 port 514 failed
Nov 22 11:34:29 172.16.100.254 92: Nov 22 16:34:19: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 172.16.100.100 port 514 started - reconnection
Nov 22 11:35:21 172.16.100.254 93: Nov 22 16:35:20: %LINK-5-CHANGED: Interface GigabitEthernet0/1, changed state to administratively down
Nov 22 11:35:21 172.16.100.254 94: Nov 22 16:35:21: %SYS-5-CONFIG_I: Configured from console by console

Reference Link to heading