Thursday, November 26th, 2009

How to remove cruft from your apache log

Keeping an eye on apache logs is an essential part of a webmaster’s job. Checking the logs help you detect performance problems and security issues. There are many log analyser software available to do this job effectively and create graphs, charts and other visualisations. Still, there are times when you need to check data directly from raw logs. But not all entries are useful. Access to a web page creates log entries not just for that page but for the JavaScript files, CSS files and images included in that page. This makes difficult to find information easily from raw logs. Separating such entries from main access log would help you to analyse data easily.

There are two ways to remove cruft from main log.

  1. Conditional logging.
  2. Separate (sub)domains for media and feeds.