Philipp's Computing Blog

Success is about speed and efficiency

wsgi-request-logger - Logging HTTP Requests With Any WSGI Web Application like Flask, Bottle or Django

I like WSGI (micro-)frameworks like Bottle, Flask or Django and used them for a couple of web applications so far. One thing that I wasn't able to do properly was request logging. It either wasn't in a format I liked, was dependend on the server implementation of WSGI or simply non-existent.
So I forked wsgilog written by L. C. Rees and created my own WSGI middleware, wsgi-request-logger, to do the logging right. By making use of Python's Logger Facilities, you can easily log to STDOUT, time rotated log files, email, syslog, etc. It can imitate Apache's combined log format to allow you to use any of the many tools for Apache log file analysis. The format depends on the specific configuration but in most cases, the so called combined log format is being used. Here is an example of how the log output looks (the IPs are fake):

162.107.87.88 - - [09/Jun/2013:01:36:58 +0200] "GET /feed/ HTTP/1.1" 200 15414 "-" "Apple-PubSub/65.32"
249.80.66.78 - - [09/Jun/2013:01:37:20 +0200] "GET /2009/02/format-usb-stick/ HTTP/1.1" 200 15008 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
105.210.193.30 - - [09/Jun/2013:01:37:27 +0200] "GET /2011/10/samsung-galaxy-s-plus-gt-i9001/ HTTP/1.0" 200 65842 "http://blog.philippklaus.de/2011/10/samsung-galaxy-s-plus-gt-i9001/" "Opera/9.80 (Windows NT 6.1; WOW64; U; Edition Next; Edition Yx; ru) Presto/2.11.310 Version/12.50"

Installation

The wsgi logging middleware can be installed with pip:

pip install wsgi-request-logger

Usage

Here is a rather academic snippet that shows you to use this middleware to wrap a bare WSGI application and log to the file access.log:

from requestlogger import WSGILogger, ApacheFormatter
from logging.handlers import TimedRotatingFileHandler

def application(environ, start_response):
    response_body = 'The request method was %s' % environ['REQUEST_METHOD']
    response_body = response_body.encode('utf-8')
    response_headers = [('Content-Type', 'text/plain'),
                        ('Content-Length', str(len(response_body)))]
    start_response('200 OK', response_headers)
    return [response_body]

handlers = [ TimedRotatingFileHandler('access.log', 'd', 7) , ]
loggingapp = WSGILogger(application, handlers, ApacheFormatter())

if __name__ == '__main__':
    from wsgiref.simple_server import make_server
    http = make_server('', 8080, loggingapp)
    http.serve_forever()

Now this is an example on how to use it with Bottle:

from bottle import Bottle, run, template
from requestlogger import WSGILogger, ApacheFormatter
from logging.handlers import TimedRotatingFileHandler

app = Bottle()

@app.route('/hello')
def hello():
    return "Hello World!"

handlers = [ TimedRotatingFileHandler('access.log', 'd', 7) , ]
app = WSGILogger(app, handlers, ApacheFormatter())

if __name__ == '__main__':
    run(app, host='localhost', port=8080)

You can provide WSGILogger with a single or multiple logging handlers to write the log entries to a file, to your stdin etc.

Resources

  • Check out PyPI's listing of wsgi-request-logger
  • The source code for this Python module is hosted on Github.
  • This software is based on WSGI middleware logging plugin (it only logs exceptions though!): wsgilog. It also enables wrapped WSGI apps to use its logging facilities.
  • Developers' Resources:
  • My first effort for request logging was bottlelog, a plugin for Bottle that would do request logging. Its design had a lot of flaws – it seems like you can't get request logging right from within a Bottle plugin. But its still available on Github.