Journalctl — Debugging and Log Analysis

Systemd journal captures logs from services, kernel, and other sources in an indexed binary format. Journalctl offers powerful filtering that is significantly better than grep in /var/log, because it allows combining filters by service, priority, time, and other metadata in a single query. Structured logs also preserve context — you know exactly which process and PID generated each message.

Basics¶

journalctl -u nginx                    # logs for a specific service
journalctl -fu nginx                   # follow mode — real-time monitoring
journalctl -n 50 -u myapp              # last 50 lines

Filtering¶

journalctl --since '1 hour ago'        # logs from the last hour
journalctl --since today               # since midnight
journalctl -b                          # current boot
journalctl -p err                      # errors and above only
journalctl -k                          # kernel messages (dmesg alternative)
journalctl _UID=1000                   # logs from a specific user
journalctl _TRANSPORT=audit            # audit logs

Filters can be combined — for example, journalctl -u nginx -p warning --since '2 hours ago' shows warnings and errors from nginx over the last two hours. Priority levels correspond to syslog: emerg, alert, crit, err, warning, notice, info, debug.

Formats and grep¶

journalctl -u nginx -o json-pretty     # structured JSON output
journalctl -u nginx -o cat             # plain text without metadata
journalctl -u myapp --grep 'error'     # full-text search
journalctl -u myapp -o verbose         # all metadata fields

JSON output is useful for machine processing — it can be piped to jq for analysis or sent to external log management systems.

Management¶

journalctl --disk-usage                # how much space logs consume
journalctl --vacuum-size=500M          # limit to 500 MB
journalctl --vacuum-time=30d           # delete logs older than 30 days

For persistent settings, edit /etc/systemd/journald.conf — the SystemMaxUse and MaxRetentionSec parameters control maximum log size and age. Restart systemd-journald after changes.

Journal is your friend¶

Powerful log filtering directly on the server covers most diagnostic scenarios. For centralized logging from multiple servers, consider Elasticsearch/OpenSearch with Fluentd or Grafana Loki with Promtail.