daiquiri – Python logging setup helper

Build Status Latest Version

The daiquiri library provides an easy way to configure logging. It also provides some custom formatters and handlers.

Installation

pip install daiquiri

If you want to enable support of JSON output, you must also install the json flavor:

pip install daiquiri[json]

If you want to enable systemd support, you must install the systemd flavor:

pip install daiquiri[systemd]

Usage

The basic usage of daiquiri is to call the daiquiri.setup function that will setup logging with the options passed as keyword arguments. If no arguments are passed, the default will log to stderr. If stderr is a terminal, the output will use colors.

import logging

import daiquiri

daiquiri.setup(level=logging.INFO)

logger = daiquiri.getLogger(__name__)
logger.info("It works and log to stderr by default with color!")

You can specify different outputs with different formatters. The daiquiri.output module provides a collection of Output classes that you can use to your liking to configure the logging output. Any number of output can be configured.

import logging
import sys

import daiquiri

# Log both to stdout and as JSON in a file called /dev/null. (Requires
# `python-json-logger`)
daiquiri.setup(level=logging.INFO, outputs=(
    daiquiri.output.Stream(sys.stdout),
    daiquiri.output.File("/dev/null",
                         formatter=daiquiri.formatter.JSON_FORMATTER),
    ))

logger = daiquiri.getLogger(__name__, subsystem="example")
logger.info("It works and log to stdout and /dev/null with JSON")

If the default output configurations suit your needs, then for convenience you may pass the name of an output as a string rather than needing to import the class and produce an instance.

import logging

import daiquiri

daiquiri.setup(level=logging.INFO, outputs=('stdout', 'stderr'))

logger = daiquiri.getLogger(__name__)
logger.info("It works and logs to both stdout and stderr!")

At the moment the names ‘stderr’, ‘stdout’, ‘syslog’, and ‘journal’ are available, assuming the underlying handler is available.

Picking format

You can configure the format of any output by passing a formatter as the formatter argument to the contructor. Two default formatters are available: daiquiri.formatter.TEXT_FORMATTER which prints log messages as text, and the daiquiri.formatter.JSON_FORMATTER which prints log messages as parsable JSON (requires python-json-logger).

You can provide any class of type logging.Formatter as a formatter.

import logging

import daiquiri
import daiquiri.formatter

daiquiri.setup(level=logging.INFO, outputs=(
    daiquiri.output.Stream(formatter=daiquiri.formatter.ColorFormatter(
        fmt="%(asctime)s [PID %(process)d] [%(levelname)s] "
        "%(name)s -> %(message)s")),
    ))

logger = daiquiri.getLogger(__name__)
logger.info("It works with a custom format!")

Python warning support

The Python warnings module is sometimes used by applications and libraries to emit warnings. By default, they are printed on stderr. Daiquiri overrides this by default and log warnings to the py.warnings logger.

This can be disabled by passing the capture_warnings=False argument to daiquiri.setup.

Extra usage

While it’s not mandatory to use daiquiri.getLogger to get a logger instead of logging.getLogger, it is recommended as daiquiri provides an enhanced version of the logger object. It allows any keyword argument to be passed to the logging method and that will be available as part of the record.

import logging

import daiquiri
import daiquiri.formatter

daiquiri.setup(level=logging.INFO, outputs=(
    daiquiri.output.Stream(formatter=daiquiri.formatter.ColorFormatter(
        fmt=(daiquiri.formatter.DEFAULT_FORMAT +
             " [%(subsystem)s is %(mood)s]"))),
    ))

logger = daiquiri.getLogger(__name__, subsystem="example")
logger.info("It works and log to stderr by default with color!",
            mood="happy")

Syslog support

The daiquiri.output.Syslog output provides syslog output.

Systemd journal support

The daiquiri.output.Journal output provides systemd journal support. All the extra arguments passed to the logger will be shipped as extra keys to the journal.

File support

The daiquiri.output.File output class provides support to log into a file.

daiquiri.output.RotatingFile class logs to a file that rotates when a maximum file size has been reached.

daiquiri.output.TimedRotatingFile will rotate the log file on a fixed interval.

import datetime
import logging

import daiquiri

daiquiri.setup(
    level=logging.DEBUG,
    outputs=(
        daiquiri.output.File('errors.log', level=logging.ERROR),
        daiquiri.output.TimedRotatingFile(
            'everything.log',
            level=logging.DEBUG,
            interval=datetime.timedelta(hours=1))
    )
)

logger = daiquiri.getLogger(__name__)

logger.info('only to rotating file logger')
logger.error('both log files, including errors only')

API

class daiquiri.KeywordArgumentAdapter(logger, extra)

Logger adapter to add keyword arguments to log record’s extra data

Keywords passed to the log call are added to the “extra” dictionary passed to the underlying logger so they are emitted with the log message and available to the format string.

Special keywords:

extra
An existing dictionary of extra values to be passed to the logger. If present, the dictionary is copied and extended.
setLevel(level)

Set the specified level on the underlying logger.

daiquiri.getLogger(name=None, **kwargs)

Build a logger with the given name.

Parameters:name (string) – The name for the logger. This is usually the module name, __name__.
daiquiri.parse_and_set_default_log_levels(default_log_levels, separator='=')

Set default log levels for some loggers.

Parameters:default_log_levels – List of strings with format

<logger_name><separator><log_level>

daiquiri.set_default_log_levels(loggers_and_log_levels)

Set default log levels for some loggers.

Parameters:loggers_and_log_levels – List of tuple (logger name, level).
daiquiri.setup(level=30, outputs=[<daiquiri.output.Stream object>], program_name=None, capture_warnings=True)

Setup Python logging.

This will setup basic handlers for Python logging.

Parameters:
  • level – Root log level.
  • outputs – Iterable of outputs to log to.
  • program_name – The name of the program. Auto-detected if not set.
  • capture_warnings – Capture warnings from the `warnings’ module.

Output

class daiquiri.output.File(filename=None, directory=None, suffix='.log', program_name=None, formatter=<daiquiri.formatter.ColorExtrasFormatter object>, level=None)

Ouput to a file.

class daiquiri.output.Output(handler, formatter=<daiquiri.formatter.ColorExtrasFormatter object>, level=None)

Generic log output.

add_to_logger(logger)

Add this output to a logger.

class daiquiri.output.RotatingFile(filename=None, directory=None, suffix='.log', program_name=None, formatter=<daiquiri.formatter.ColorExtrasFormatter object>, level=None, max_size_bytes=0, backup_count=0)

Output to a file, rotating after a certain size.

do_rollover()

Manually forces a log file rotation.

class daiquiri.output.Stream(stream=<open file '<stderr>', mode 'w'>, formatter=<daiquiri.formatter.ColorExtrasFormatter object>, level=None)

Generic stream output.

class daiquiri.output.TimedRotatingFile(filename=None, directory=None, suffix='.log', program_name=None, formatter=<daiquiri.formatter.ColorExtrasFormatter object>, level=None, interval=datetime.timedelta(1), backup_count=0)

Rotating log file output, triggered by a fixed interval.

do_rollover()

Manually forces a log file rotation.

Handlers

class daiquiri.handlers.JournalHandler(program_name)

Journald based handler. Only available on platforms using systemd.

class daiquiri.handlers.SyslogHandler(program_name, facility=None)

Syslog based handler. Only available on UNIX-like platforms.

class daiquiri.handlers.TTYDetectorStreamHandler(stream=None)

Stream handler that adds a hint in the record if the stream is a TTY.

Formatter

class daiquiri.formatter.ColorExtrasFormatter(keywords=None, extras_template='[{0}: {1}]', extras_separator=' ', extras_prefix=' ', extras_suffix='', *args, **kwargs)

Combines functionality of ColorFormatter and ExtrasFormatter.

class daiquiri.formatter.ColorFormatter(fmt=None, datefmt=None)

Colorizes log output

class daiquiri.formatter.ExtrasFormatter(keywords=None, extras_template='[{0}: {1}]', extras_separator=' ', extras_prefix=' ', extras_suffix='', *args, **kwargs)

Formats extra keywords into %(extras)s placeholder.

Any keywords passed to a logging call will be formatted into a “extras” string and included in a logging message. Example:

logger.info(‘my message’, extra=’keyword’)
will cause an “extras” string of:
[extra: keyword]

to be inserted into the format in place of %(extras)s.

The optional keywords argument must be passed into the init function to enable this functionality. Without it normal daiquiri formatting will be applied. Any keywords included in the keywords parameter will not be included in the “extras” string.

Special keywords:

keywords
A list of strings containing keywords to filter out of the “extras” string.
extras_template
A format string to use instead of ‘[{0}: {1}]’
extras_separator
What string to “join” multiple “extras” with.
extras_prefix and extras_suffix
Strings which will be prepended and appended to the “extras” string respectively. These will only be prepended if the “extras” string is not empty.