Linux Basics – An Intro to Log Files

Linux log files are a wonderful source of information. They can be used for debugging and troubleshooting almost any running application. If you are bored of watching TV, try staring at your Linux log files for a while. It can get addictive.

Linux logs are mostly stored in plain text. Most log files on standard Linux are found in the /var/log directory. Some software may move the log locations around (such as cPanel).

The /var/log directory

Some log files are stored in files (e.g /var/log/messages). Log files may or may not have the extension “.log“. Some applications have their own log folder under /var/log. Applications such as Apache/httpd and Exim both create their own folders as they create multiple log files.

Reading Log Files

We normally never open log files with a text editor as some editors may lock the file and this could crash the application. There are also very few good reasons to edit a log file.

What we do need to do occasionally is just to wipe out some log files that are taking too much space. Only do this if you really cannot make space somewhere else. Never delete a log file as that may also crash an application using the log file. Rather just execute ” >/var/log/messages ” to blank that log file and reclaim the space.

Tools to extract info from log files are the following:

• cat – this flushes the whole log file content to your log in session. On a big log file, this can be a problem. The ‘cat‘ command is usually used on conjunction with the ‘grep‘ command as the following example:   cat /var/log/syslog |egrep '^May 20 00:09:16' (find all log entries in syslog that occurred at 9 minutes past midnight on May 20th)
• head – this will read lines from the start of the log file
• tail – this will read lines from the end of the log file (newest entries). The ‘tail‘ command can be run in ‘follow’ mode with ‘tail -f‘ which will show you log entries in real time as they are made.
• Most log files contain at least the following: Date, [Hostname], Application/Service and Message.
• Some log files trace events with a unique ID entry if many lines of log are created by a single event such as the arrival of an email. Using the ‘grep‘ command for the unique ID can then show you the whole process from start to finish.

Key System Logs

The following log files are the main ones to look out for within Linux.

• Authorisation Log
• Daemon Log
• Debug Log
• Kernel Log
• System Log

The authorisation log (auth.log) tracks use of the authorisation systems which control user access.

The daemon log (daemon.log) tracks services that run in the background which perform important tasks. Daemons tend to have no graphical output.

The debug log provides debug output for applications.

The kernel log provides details about the Linux kernel.

The system log contains the most information about your system and if your application doesn’t have its own log the entries will probably be in this log file.

Rotating Logs

Log files rotate periodically so that they don’t get too big. The logrotate utility is responsible for rotating log files. You can tell when a log has been rotated because it will be followed by a number such as auth.log.1, auth.log.2. It is possible to change the frequency of log rotation by editing the file /etc/ logrotate.conf

I hope this concise guide will help to point you in the right direction the next time you need to diagnose an issue or merely waste some time log-gazing…

If you missed the previous one, check out our article about the man and mkdir commands.

See you in our next Linux Basis tutorial where we discuss head, tail, cat commands and more.

Happy Hosting!