LogHog Python Client

This is a Python client for the LogHog log management server (https://github.com/activefrequency/loghogd). It implements a drop-in LogHandler for the Python logging framework.

Python's built-in logging using the FileHandler family does not prevent multiple processes from writing to the same log file (often over each other). This is a typical issue with Django running under apache2.

LogHog solves this by having all the processes log to a central server. The LogHog server takes care of writing the messages one at a time, rotating and compressing files, deleting old log files, etc.

You can think of LogHog as a very secure and user friendly syslog + logrotate in one package.

When to use LogHog

There are many situations when LogHog will come in handy. Here are some examples:

• Your application runs on multiple servers and you want the data in one place
• You have a multi-process application and you want every process to write to a single log file
• You want to offload your logging to a different server
• Your application writes logs

In fact there is almost no reason not to use LogHog. It is fast, simple, secure, and it stays out of your way.

Quickstart

Step 1: Install the LogHog server (loghogd). If you are using Ubuntu, run the following:

sudo add-apt-repository ppa:activefrequency/ppa
sudo apt-get update
sudo apt-get install loghogd

Note: if you are running an older (e.g.: 10.04) release of Ubuntu, you may need to first install the python-software-properties package to get the add-apt-repository command.

If you are using Debian, run the following:

echo 'deb http://ppa.launchpad.net/activefrequency/ppa/ubuntu lucid main' | sudo tee -a /etc/apt/sources.list.d/99-loghogd.list
echo 'deb-src http://ppa.launchpad.net/activefrequency/ppa/ubuntu lucid main' | sudo tee -a /etc/apt/sources.list.d/99-loghogd.list

gpg --keyserver hkp://keyserver.ubuntu.com/ --recv-keys F96CE604
gpg -a --export F96CE604 | sudo apt-key add -

sudo apt-get update
sudo apt-get install loghogd

Step 2: List your application in the LogHog logging facilities. Put the following in /etc/loghogd/facilities.conf:

[my-first-app]
rotate = 0 0 * * *
backup_count = 14

And reload loghog:

sudo /etc/init.d/loghogd reload

Step 3: Install LogHog Python Client (this codebase):

pip install loghog

Step 4: Enable logging in your application. Add the following to your app startup:

import logging
from loghog import LoghogHandler

logger = logging.getLogger()

handler = LoghogHandler('my-first-app')

handler.setFormatter(logging.Formatter('%(levelname)s - %(message)s'))
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

logger.info('Hello world!')

Start your app and look at /var/log/loghogd/my-first-app/ to see your application's log."

For more extensive configuration, see the examples directory.

Step 4: Django: If you want to use LogHog with your Django application, add the following code to your settings.py:

LOGHOG_APP_NAME = 'my-first-app'

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'simple': {
            'format': '%(levelname)s - %(message)s'
        },
    },
    'handlers': {
        'root':{
            'level':'DEBUG',
            'class':'loghog.LoghogHandler',
            'app_name': LOGHOG_APP_NAME,
            'formatter': 'simple',
        },
    },
    'loggers': {
        'root': {
            'handlers':['root', ],
            'propagate': False,
            'level': 'INFO',
        },
        'django': {
            'handlers':['root', ],
            'propagate': False,
            'level': 'INFO',
        },
        'django.request': {
            'handlers': ['root', ],
            'level': 'INFO',
            'propagate': False,
        },
    }
}

Now, all default logging from within Django will be sent to the LogHog server. If you want to send log messages from within your code, you can:

import logging
logger = logging.getLogger('django')

def view(request):
    # ...
    logger.info('Hello world')
    # ...
    return response

Configuration

LogHog can do a lot of things for you. It can rotate files based on a schedule or file size, encrypt traffic between your application and the server using SSL/TLS, sign messages you send to the server, etc. Take a look a examples/kitchen-sink.py for an example of the usage of all of these.

Here are some highlights:

# Specify a remote host
handler = LoghogHandler('my-app', host='example.com', port=5566)

# Use UDP
handler = LoghogHandler('my-app', stream=False)

# Use zlib compression to save on bandwidth
handler = LoghogHandler('my-app', compression=True)

# Sign messages using HMAC
handler = LoghogHandler('my-app', secret='12345')

# Use a different hostname than reported by the hostname(1) command
handler = LoghogHandler('my-app', hostname='foo')

# Tell LoghogHandler that you want it to print connection errors to stdout
# NOTE: this does nothing if stream=False
handler = LoghogHandler('my-app', print_debug=True)

# Use SSL
ssl_info = {
    'pemfile': '/PATH/TO/CLIENT.pem',
    'cacert': '/etc/loghogd/certs/loghogd-ca.cert',
}

handler = LoghogHandler('my-app', ssl_info=ssl_info)

For configuring the LogHog server, see https://github.com/activefrequency/loghogd.

License

This code is released under the Apache 2 license. See LICENSE for more details.

Contributing

Feel free to fork this code and submit pull requests. If you are new to GitHub, feel free to send patches via email to [email protected].

Credits

Credit goes to Active Frequency, LLC (http://activefrequency.com/) for sponsoring this project.