Search This Blog

2015-12-31

Apache Log Analysis

The below will install and configure GoAccess
GoAccess is an httpd-access log parser that will give you page hit statistics and other useful analytic tools.

Equipment

GoAccess OS: RHEL 7.0
Server OS: FreeBSD 5.4 STABLE
Server Application: Apache/1.3.34 (Unix)

Prerequisites

Install GoAccess

yum -y install goaccess

Ensure you have access to httpd-access.log

Either live files under: /var/log/httpd-access.log
OR
Copy the files from a live server: scp /var/log/httpd-access.log $servername:/path/

Configure GoAccess

Open up /etc/goaccess.conf and change the three parameters: time-format, date-format, and log-format to match what I have here
grep -E "log-format|time-format|date-format" /etc/goaccess.conf | grep -v "^#"
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h %l %u %^[%d:%t %^] "%r" %s %b "%R" "%u"

The original httpd.conf combined LogFormat that I have looks like

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

The important part in this config is to note that we do some ignoring of fields that apache adds.
We ignore the brackets before and after the timestamp:

%^[
%^]

We can see that we also:
Remove escapes \"%r\"
Change \"%{Referer}i\" to "%R"
Change \"%{User-Agent}i\" to "%u"

Analyzing Some Logs

Either ensure you are in /var/log/ or copy the files to a directory on your server.

GoAccess supports piping of log files to analyze so we can just run it like this:
bzcat httpd-access.log.* | goaccess
Alternatively, generate an html report just by redirecting output to an html file like this:
bzcat httpd-access.log.* | goaccess > report.html

http://webmasters.stackexchange.com/questions/4852/what-is-the-best-apache-logs-analyzer
https://en.wikipedia.org/wiki/List_of_web_analytics_software
http://www.debianhelp.co.uk/analog.htm
http://stackoverflow.com/questions/9234699/understanding-apache-access-log
http://xmodulo.com/interactive-apache-web-server-log-analyzer-linux.html
http://i-heart-geek.blogspot.ca/2011/10/top-command-line-tips-apache-access-log.html

No comments:

Post a Comment