Monitoring, monitoring.

Earlier in my Cybersecurity journey, I never understood the importance of event logging on a system. I now know that log collection and analysis is everything. Every event and log captured can tell a story if forensically pieced together.

Logs are historical records of events that are created by operating system, software and anything that can create a process on a system. Log generation and analysis are extremely vital in these fields:

1. Troubleshooting systems, software and networks.
2. Security audits and compliance.
3. Understanding normal behavior of systems, software, networks and users. This helps to identify anomalies.
4. Forensic investigation.

I want to discuss some tools that can be used to improve log generation and analysis.

1. Sysmon

Sysmon is a tool used to enrich Windows event logs. It enables enhanced monitoring of events such as process creations, network connections, changes to the file system and registry. Sysmon was initially only available for Windows but now it can be installed on Linux/Unix systems(SysmonForLinux). Sysmon is automatically loaded on startup.

Setup

1. Download Sysmon on the endpoint to monitor: https://download.sysinternals.com/files/Sysmon.zip
2. Download Sysmon configuration file:

Invoke-WebRequest -Uri https://raw.githubusercontent.com/olafhartong/sysmon-modular/master/sysmonconfig.xml -OutFile sysmonconfig.xml

This configuration file allows Sysmon to monitor a lot of events such as commands executed, network connections, processes created, changes to filesystem and registry.

3. Install Sysmon to use configuration file:

sysmon -accepteula -i path\to\sysmonconfig.xml

After successful installation, Sysmon will begin to generate several events. This events can be seen by opening Event Viewer, Then clicking Application and Services Logs> Microsoft> Windows> Sysmon > Operational.

Fig 1: View of Sysmon Logs

This events have IDs which are used to quickly recognize the particular event created. See more on Sysmon event IDs here: https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon#events

Common Sysmon event IDs are

• Event ID 1: Process creation
• Event ID 3: Network connection
• Event ID 5: Process terminated
• Event ID 11: FileCreate
• Event ID 12: RegistryEvent (Object create and delete)

2. SysmonForLinux

SysmonForLinux is basically Sysmon but for Linux. It is used for enhanced monitoring of events such as process creations, network connections, changes to the file system.

Setup

1. Install SysmonForLinux

Centos 8

a. Register Microsoft key and feed

sudo rpm -Uvh https://packages.microsoft.com/config/centos/8/packages-microsoft-prod.rpm

b. Install SysmonForLinux

sudo dnf install sysmonforlinux

Ubuntu 20
a. Register Microsoft key and feed

wget -q https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb

b. Install SysmonForLinux

sudo apt-get update
sudo apt-get install sysmonforlinux

2. After installation is complete, download the SysmonForLinux configuration file.

3. Use the SysmonForLinux configuration file:

./sysmon -c <NAME_OF_SYSMON_LINUX_CONFIG>.xml

4. Enable sysmon to run at startup:

systemctl enable sysmon

You can find Sysmon logs in /var/log/syslog. Below is an example of a Sysmon Log.

pr 10 10:12:51 slave sysmon: <Event><System><Provider Name="Linux-Sysmon" Guid="{ff032593-a8d3-4f13-b0d6-01fc615a0f97}"/><EventID>1</EventID><Version>5</Version><Level>4</Level><Task>1</Task><Opcode>0</Opcode><Keywords>0x8000000000000000</Keywords><TimeCreated SystemTime="2022-04-10T06:12:51.059265000Z"/><EventRecordID>15332</EventRecordID><Correlation/><Execution ProcessID="821" ThreadID="821"/><Channel>Linux-Sysmon/Operational</Channel><Computer>slave</Computer><Security UserId="0"/></System><EventData><Data Name="RuleName">-</Data><Data Name="UtcTime">2022-04-10 06:12:51.059</Data><Data Name="ProcessGuid">{5fa610dc-7563-6252-212d-822ce7550000}</Data><Data Name="ProcessId">4099</Data><Data Name="Image">/usr/bin/tail</Data><Data Name="FileVersion">-</Data><Data Name="Description">-</Data><Data Name="Product">-</Data><Data Name="Company">-</Data><Data Name="OriginalFileName">-</Data><Data Name="CommandLine">tail -f /var/log/syslog</Data><Data Name="CurrentDirectory">/home/st9</Data><Data Name="User">root</Data><Data Name="LogonGuid">{5fa610dc-0000-0000-0000-000000000000}</Data><Data Name="LogonId">0</Data><Data Name="TerminalSessionId">3</Data><Data Name="IntegrityLevel">no level</Data><Data Name="Hashes">-</Data><Data Name="ParentProcessGuid">{5fa610dc-750c-6252-5dee-4c8796550000}</Data><Data Name="ParentProcessId">3988</Data><Data Name="ParentImage">/usr/bin/bash</Data><Data Name="ParentCommandLine">bash</Data><Data Name="ParentUser">root</Data></EventData></Event>

3. Linux Auditing System aka Auditd

This is a tool native to Linux/Unix systems that can be used to monitor actions on a system. To use Auditd, rules have to be created to audit a particular action.

Auditd can monitor things like:

a. File and directory access

b. System calls

c. Pre-configured auditable events within the kernel. Red Hat maintains a list of these types of events.

Setup

1. Check if Auditd is already installed:

sudo yum list audit audit-libs

2. Install Auditd if not already installed:

sudo yum install audit

Understanding Auditd

For file and directory access a watch rule can be used. This rule can track whether a file or directory is triggered by certain types of access, including read, write, execute, and attribute changes.

Basic rule syntax:

auditctl -w path_to_file -p permissions -k key_name

For example, to track write and access in a directory:

auditctl -w /home/vagrant/do_not_modify -p wa -k directory-modified

For persistence after reboot, use this instead:

echo "-w /home/vagrant/do_not_modify -p wa -k directory-modified" >> /etc/audit/rules.d/audit.rules

To apply changes:

service auditd reload

To list Auditd rules:

auditctl -l

For testing purposes, make a file inside of the audited directory. Then search Auditd logs search with the key/tag:

ausearch -i -k directory-modified

Logs generated by Auditd can be found in /var/log/audit/audit.log file. Auditd can be used for auditing other things like system calls, binary executions etc.

The logs generated by these tools can be collected by a SIEM for processing. Wazuh is a good example of an opensource unified SIEM and XDR that can be used to process logs generated at endpoints. I will write about more tools that can be used for generating logs of interest in my next posts.