Systemd Service Logging Configuration
Configure logging for systemd services using StandardOutput, StandardError, and the systemd journal. Learn journalctl filtering, log rotation, and forwarding to external systems.
Detailed Explanation
Centralized Logging with systemd Journal
Systemd provides a built-in logging system called the journal. Services can route their output to the journal for centralized, structured, and queryable log management.
Basic Logging Configuration
[Service]
StandardOutput=journal
StandardError=journal
SyslogIdentifier=myapp
StandardOutput=journal: Routes stdout to the journalStandardError=journal: Routes stderr to the journalSyslogIdentifier=myapp: Sets the log identifier (defaults to the service name)
StandardOutput/StandardError Options
| Value | Destination |
|---|---|
journal |
systemd journal (recommended) |
syslog |
syslog socket (forwarded to journal too) |
kmsg |
Kernel message buffer |
null |
Discarded (/dev/null) |
inherit |
Inherited from PID 1 |
file:/path |
Direct to a specific file |
append:/path |
Append to a specific file |
Querying Logs with journalctl
# All logs for a service
journalctl -u myapp.service
# Follow live (like tail -f)
journalctl -u myapp.service -f
# Logs since boot
journalctl -u myapp.service -b
# Logs from today
journalctl -u myapp.service --since today
# Last 100 lines
journalctl -u myapp.service -n 100
# Only errors and above
journalctl -u myapp.service -p err
# Time range
journalctl -u myapp.service --since "2024-01-01 00:00" --until "2024-01-02 00:00"
# JSON output for parsing
journalctl -u myapp.service -o json-pretty
Log Priority Levels
| Level | Keyword | Use Case |
|---|---|---|
| 0 | emerg | System unusable |
| 1 | alert | Immediate action needed |
| 2 | crit | Critical errors |
| 3 | err | Errors |
| 4 | warning | Warnings |
| 5 | notice | Normal but significant |
| 6 | info | Informational |
| 7 | debug | Debug messages |
Filter by priority: journalctl -u myapp -p warning shows warnings and above.
Journal Storage and Rotation
Configure in /etc/systemd/journald.conf:
[Journal]
SystemMaxUse=500M # Maximum disk usage
SystemKeepFree=1G # Keep at least 1G free
MaxRetentionSec=1month # Keep logs for 1 month
MaxFileSec=1week # Rotate files weekly
Direct File Logging
For applications that need file-based logs:
StandardOutput=append:/var/log/myapp/output.log
StandardError=append:/var/log/myapp/error.log
Combine with logrotate for rotation:
/var/log/myapp/*.log {
daily
rotate 14
compress
missingok
notifempty
postrotate
systemctl kill -s HUP myapp.service
endscript
}
Structured Logging
Applications can send structured data to the journal using the native protocol. This enables rich queries beyond simple text matching.
Use Case
Setting up comprehensive logging for a production service that needs centralized log collection, time-based log queries, severity filtering, and integration with log management tools like Loki or Elasticsearch.