Parsing Syslog Format Logs
Parse RFC 3164 syslog messages to extract timestamp, hostname, process name, PID, and message content from system-level logs.
Detailed Explanation
Syslog Format (RFC 3164)
Syslog is the standard logging protocol for Unix/Linux systems. RFC 3164 defines the BSD syslog format, which is still the most commonly encountered format in system logs, journal files, and network device outputs.
Format Structure
<priority>timestamp hostname application[pid]: message
- Priority is optional and enclosed in angle brackets (e.g.,
<34>) - Timestamp follows the
Mmm dd HH:MM:SSformat (e.g.,Jan 15 10:30:00) - Hostname identifies the originating system
- Application is the process name, optionally followed by
[pid]
Example Log Lines
Jan 15 10:30:00 webserver01 sshd[12345]: Failed password for root from 10.0.0.5 port 22
Jan 15 10:30:01 webserver01 nginx[6789]: upstream timed out (110: Connection timed out)
Jan 15 10:30:02 appserver02 myapp[4321]: INFO Starting health check routine
Jan 15 10:30:03 dbserver01 kernel: [UFW BLOCK] IN=eth0 OUT= MAC=... SRC=203.0.113.5 DST=10.0.0.1
Fields Extracted
| Field | Description |
|---|---|
| Timestamp | Jan 15 10:30:00 — note: no year in RFC 3164 |
| Hostname | webserver01 — the originating host |
| Process | sshd — the application name |
| PID | 12345 — process ID (if present) |
| Message | The full message content |
Severity Detection
Syslog messages do not always include an explicit severity keyword. The parser scans the message content for common keywords (ERROR, WARN, INFO, DEBUG, FAIL) to assign a severity level. Messages without recognizable keywords are classified as UNKNOWN.
Note on RFC 5424
RFC 5424 (the newer syslog standard) uses a more structured format with ISO 8601 timestamps and structured data elements. The parser primarily targets RFC 3164 but can handle many RFC 5424 messages through the auto-detect mode.
Use Case
Analyzing Linux system logs from /var/log/syslog or /var/log/messages, investigating SSH authentication failures, monitoring kernel messages and firewall logs, parsing logs from network devices that output in syslog format, and debugging system service issues.