ioguix' README

06 August 2012 1 comment

Using pgBagder and logsaw for scheduled reports


While waiting for next version of pgBadger, here is a tip to create scheduled pgBadger report. For this demo, I’ll suppose:

You will need pgbadger and logsaw. Both tools are under BSD license and pure perl script with no dependencies.

“logsaw” is a tool aimed to parse log files, looking for some regexp-matching lines, printing them to standard output and remembering where it stop last time. At start, it searches for the last line it parsed during its last call, and starts working from there. Yes, for those familiar with “tail_n_mail”, it does the same thing, but without the mail and report processing part. Moreover, it supports rotation and compression of log files (not sure about tail_n_mail). Thanks to this tool, we’ll be able to create new reports from where the last one finished!

We need to create a simple configuration file for logsaw:

$ cat <<EOF > ~postgres/.logsaw

There’s three more optional parameters in this configuration file you might want to know:

That’s it for logsaw. See its README files for more details, options and sample configuration file.

Now, the command line to create the report using pgbadger:

$ logsaw | pgbadger -g -o /path/to/report-$(date +%Y%m%d).html -

You might want to add the -f option to pgbadger if it doesn’t guess the log format itself (stderr or syslog or csv).

About creating reports on a weekly basis, let’s say every sunday at 2:10am, using crontab:

10 2 1-7 * 7  logsaw | pgbadger -g -o /path/to/report-$(date +\%Y\%m\%d).html -

Here you go, enjoy :)

That’s why I like UNIX style and spirit commands: simple single-task but powerful and complementary tools.

Wait or help for more nice features in pgBadger !

Cheers !

PS: There’s another tool to deal with log files and reports you might be interested in: “logwatch”

Comments Leave a comment by email or Disqus

Note: I only publish your name/pseudo, mail subject and content. I will NOT publish your email address.

gilles reply
Fri, 31 Aug 2012 23:51:19 +0200

About creating reports with just pgbadger 2.x on a weekly basis, like in your example using crontab:

10 2 1-7 * 7 pgbadger -o /path/to/report-$(date +%Y%m%d).html -l /path/to/pgbadger-incremental.last `find /var/log/postgresql/postgresql.* -mtime -7`