@GeorgeBailey That would depend on your definition of "effective"... They work, they do what they say they should. They also take up a lot of resources.

@GeorgeBailey The downside to stuffing the log directly into a database is that when the database is down or unreachable, your website will stop. That's why I'd recommend having the website log to syslog/a file, and then having another program parsing the log and putting it into the database. (I've actually written things to do that for proprietary log formats, and yes, having the logs in a database was very useful indeed.)

@JennyD, Resource consumption was something I knew could be an issue. I've been optimizing as I go, and hopefully any changes needed at production will be straight forward.

@JennyD, I solved the db down problem by having them go into temporary files before they are inserted into the database. The webserver will automatically pick up and retry any previous fails before importing the next batch.

The temporary files represent a single batch which is at 5 second or 5 minute intervals depending on client needs.

It doesn't lend itself so well to live monitoring. I am used to being able to tail -f | grep to see live streaming info, so in a later version I may be including some improvements.

My temporary files are actual SQL statements at the moment, so it really lends itself to parsing for the db. :-) Just not so much for manual viewing.