adi

Web Developer

fl

Analytics for HTTP logs in Forwarded Log Format

Install

On a Linux or BSD machine, in a terminal, run: 

mkdir -p ~/src
cd ~/src
wget -qO - https://adi.onl/fl/fl@1.0.0.tgz | tar -xzvf -
cd fl
make install PREFIX=$HOME

Uninstall

cd ~/src/fl
make uninstall PREFIX=$HOME

Usage

The package provides 8 scripts flp, flu, flv, flr and flvp, flvu , flvv and flvr.

The fl prefix stands for Forwarded Log and the suffixes stand for pages, unique, views and referrers.

The middle v in the second set stands for virtual host. The second set of 4 scripts is for the Forwarded Log with virtual hosts format. It's identical to the normal Forwarded Log except the name of the virtual host is prepended to each line.

Use the appropriate script for the statistic and for the log format:

Pages

$ flp /var/www/access.log
/docs.html 641
/ 3227

Use sort -nrk 2 to sort on second column: 

$ clp /var/www/access.log | sort -nrk 2
/ 3227
/docs.html 641

Uniques and Visits

Uniques and visits scripts output the actual unique and visit requests by default. Use wc -l to count: 

$ flu /var/www/access.log | wc -l
    2000

Referrers

$ flr /var/www/access.log
none 865
https://example.com 1222

Use sort -nrk 2 to sort on second column: 

$ flr /var/www/access.log | sort -nrk 2
https://example.com 1222
none 865

We recommend using cblfb for filtering bots. 

$ flr /var/www/access.log | cblfb
none 821
https://example.com 1212

Also, it's a good idea to enable log rotation.

The scripts operate on GET requests to paths with no extension or ending in .html that returned a 200 HTTP response status code. 

https://example.com/docs
https://example.com/docs/
https://examples.com/docs.html

Files

Download

Changelog

1.0.0 - 2022-03-04

Added

Changed

Removed

[0.0.1]

Added