The following documentation is based on the source code of version 10.0 of the coloredlogs package.
The most useful entry points into the documentation are install(), ColoredFormatter and enable_system_logging().
Colored terminal output for Python’s logging module.
The easiest way to get started is by importing coloredlogs and calling coloredlogs.install() (similar to logging.basicConfig()):
>>> import coloredlogs, logging
>>> coloredlogs.install(level='DEBUG')
>>> logger = logging.getLogger('some.module.name')
>>> logger.info("this is an informational message")
2015-10-22 19:13:52 peter-macbook some.module.name[28036] INFO this is an informational message
The install() function creates a ColoredFormatter that injects ANSI escape sequences into the log output.
The following environment variables can be used to configure the coloredlogs module without writing any code:
Environment variable | Default value | Type of value |
---|---|---|
$COLOREDLOGS_AUTO_INSTALL | ‘false’ | a boolean that controls whether auto_install() is called |
$COLOREDLOGS_LOG_LEVEL | ‘INFO’ | a log level name |
$COLOREDLOGS_LOG_FORMAT | DEFAULT_LOG_FORMAT | a log format string |
$COLOREDLOGS_DATE_FORMAT | DEFAULT_DATE_FORMAT | a date/time format string |
$COLOREDLOGS_LEVEL_STYLES | DEFAULT_LEVEL_STYLES | see parse_encoded_styles() |
$COLOREDLOGS_FIELD_STYLES | DEFAULT_FIELD_STYLES | see parse_encoded_styles() |
Here we’ll take a look at some examples of how you can customize coloredlogs using environment variables.
Here’s a screen shot of the default configuration for easy comparison with the screen shots of the following customizations (this is the same screen shot that is shown in the introduction):
The screen shot above was taken from urxvt which doesn’t support faint text colors, otherwise the color of green used for debug messages would have differed slightly from the color of green used for spam messages.
Apart from the faint style of the spam level, the default configuration of coloredlogs sticks to the eight color palette defined by the original ANSI standard, in order to provide a somewhat consistent experience across terminals and terminal emulators.
Of course you are free to customize the default configuration, in this case you can use any text style or color that you know is supported by your terminal. You can use the humanfriendly --demo command to try out the supported text styles and colors:
The simplest customization is to change the log format, for example:
$ export COLOREDLOGS_LOG_FORMAT='[%(hostname)s] %(asctime)s %(message)s'
$ coloredlogs --demo
[peter-mbp] 2018-01-14 15:45:44 message with level spam (5)
[peter-mbp] 2018-01-14 15:45:45 message with level debug (10)
[peter-mbp] 2018-01-14 15:45:46 message with level verbose (15)
[peter-mbp] 2018-01-14 15:45:47 message with level info (20)
[peter-mbp] 2018-01-14 15:45:48 message with level notice (25)
[peter-mbp] 2018-01-14 15:45:49 message with level warning (30)
[peter-mbp] 2018-01-14 15:45:50 message with level success (35)
[peter-mbp] 2018-01-14 15:45:51 message with level error (40)
[peter-mbp] 2018-01-14 15:45:52 message with level critical (50)
Here’s what that looks like in a terminal (I always work in terminals with a black background and white text):
You can also change the date/time format, for example you can remove the date part and leave only the time:
$ export COLOREDLOGS_LOG_FORMAT='%(asctime)s - %(message)s'
$ export COLOREDLOGS_DATE_FORMAT='%H:%M:%S'
$ coloredlogs --demo
15:45:34 - message with level spam (5)
15:45:35 - message with level debug (10)
15:45:36 - message with level verbose (15)
15:45:37 - message with level info (20)
15:45:38 - message with level notice (25)
15:45:39 - message with level warning (30)
15:45:40 - message with level success (35)
15:45:41 - message with level error (40)
15:45:42 - message with level critical (50)
Here’s what it looks like in a terminal:
Finally you can customize the colors and text styles that are used:
$ export COLOREDLOGS_LOG_FORMAT='%(asctime)s - %(message)s'
$ export COLOREDLOGS_DATE_FORMAT='%H:%M:%S'
$ export COLOREDLOGS_FIELD_STYLES='' # no styles
$ export COLOREDLOGS_LEVEL_STYLES='spam=22;debug=28;verbose=34;notice=220;warning=202;success=118,bold;error=124;critical=background=red'
$ coloredlogs --demo
15:45:23 - message with level spam (5)
15:45:24 - message with level debug (10)
15:45:25 - message with level verbose (15)
15:45:26 - message with level info (20)
15:45:27 - message with level notice (25)
15:45:28 - message with level warning (30)
15:45:29 - message with level success (35)
15:45:30 - message with level error (40)
15:45:31 - message with level critical (50)
Here’s an explanation of the features used here:
Of course none of this can be seen in the shell transcript quoted above, but take a look at the following screen shot:
With regards to the handling of log levels, the coloredlogs package differs from Python’s logging module in two aspects:
While the logging module uses the default logging level logging.WARNING, the coloredlogs package has always used logging.INFO as its default log level.
When logging to the terminal or system log is initialized by install() or enable_system_logging() the effective level [1] of the selected logger [2] is compared against the requested level [3] and if the effective level is more restrictive than the requested level, the logger’s level will be set to the requested level (this happens in adjust_level()). The reason for this is to work around a combination of design choices in Python’s logging module that can easily confuse people who aren’t already intimately familiar with it:
Effectively all user defined loggers inherit the default log level logging.WARNING from the root logger, which isn’t very intuitive for those who aren’t already familiar with the hierarchical nature of the logging module.
By avoiding this potentially confusing behavior (see #14, #18, #21, #23 and #24), while at the same time allowing the caller to specify a logger object, my goal and hope is to provide sane defaults that can easily be changed when the need arises.
[1] | Refer to logging.Logger.getEffectiveLevel() for details. |
[2] | The logger that is passed as an argument by the caller or the root logger which is selected as a default when no logger is provided. |
[3] | The log level that is passed as an argument by the caller or the default log level logging.INFO when no level is provided. |
The default log level for coloredlogs (logging.INFO).
The default log format for ColoredFormatter objects (a string).
The default date/time format for ColoredFormatter objects (a string).
A list of filenames that indicate a chroot and contain the name of the chroot.
Whether bold fonts can be used in default styles (a boolean).
This is disabled on Windows because in my (admittedly limited) experience the ANSI escape sequence for bold font is simply not translated by Colorama, instead it’s printed to the terminal without any translation.
Mapping of log format names to default font styles.
Mapping of log level names to default font styles.
The default logging format style (a single character).
A dictionary that maps the style characters %, { and $ (see the documentation of the python3:logging.Formatter class in Python 3.2+) to strings containing regular expression patterns that can be used to parse format strings in the corresponding style:
These regular expressions are used by FormatStringParser to introspect and manipulate logging format strings.
Automatically call install() when $COLOREDLOGS_AUTO_INSTALL is set.
The coloredlogs package includes a path configuration file that automatically imports the coloredlogs module and calls auto_install() when the environment variable $COLOREDLOGS_AUTO_INSTALL is set.
This function uses coerce_boolean() to check whether the value of $COLOREDLOGS_AUTO_INSTALL should be considered True.
Enable colored terminal output for Python’s logging module.
Parameters: |
|
---|
The coloredlogs.install() function is similar to logging.basicConfig(), both functions take a lot of optional keyword arguments but try to do the right thing by default:
Validate a logging format style.
Parameters: | value – The logging format style to validate (any value). |
---|---|
Returns: | The logging format character (a string of one character). |
Raises: | ValueError when the given style isn’t supported. |
On Python 3.2+ this function accepts the logging format styles %, { and $ while on older versions only % is accepted (because older Python versions don’t support alternative logging format styles).
Increase the verbosity of the root handler by one defined level.
Understands custom logging levels like defined by my verboselogs module.
Decrease the verbosity of the root handler by one defined level.
Understands custom logging levels like defined by my verboselogs module.
Check whether the log level of the root handler is set to a verbose level.
Returns: | True if the root handler is verbose, False if not. |
---|
Get the logging level of the root handler.
Returns: | The logging level of the root handler (an integer) or DEFAULT_LOG_LEVEL (if no root handler exists). |
---|
Set the logging level of the root handler.
Parameters: | level – The logging level to filter on (an integer or string). |
---|
If no root handler exists yet this automatically calls install().
Increase a logger’s verbosity up to the requested level.
Parameters: |
|
---|
This function is used by functions like install(), increase_verbosity() and enable_system_logging() to adjust a logger’s level so that log messages up to the requested log level are propagated to the configured output handler(s).
It uses logging.Logger.getEffectiveLevel() to check whether logger propagates or swallows log messages of the requested level and sets the logger’s level to the requested level if it would otherwise swallow log messages.
Effectively this function will “widen the scope of logging” when asked to do so but it will never “narrow the scope of logging”. This is because I am convinced that filtering of log messages should (primarily) be decided by handlers.
Find the defined logging levels.
Returns: | A dictionary with level names as keys and integers as values. |
---|
Here’s what the result looks like by default (when no custom levels or level names have been defined):
>>> find_defined_levels()
{'NOTSET': 0,
'DEBUG': 10,
'INFO': 20,
'WARN': 30,
'WARNING': 30,
'ERROR': 40,
'FATAL': 50,
'CRITICAL': 50}
Coerce a logging level name to a number.
Parameters: | value – A logging level (integer or string). |
---|---|
Returns: | The number of the log level (an integer). |
This function translates log level names into their numeric values. The logging module does this for us on Python 2.7 and 3.4 but fails to do so on Python 2.6 which coloredlogs still supports.
Find log level names which are aliases of each other.
Returns: | A dictionary that maps aliases to their canonical name. |
---|
Note
Canonical names are chosen to be the alias with the longest string length so that e.g. WARN is an alias for WARNING instead of the other way around.
Here’s what the result looks like by default (when no custom levels or level names have been defined):
>>> from coloredlogs import find_level_aliases
>>> find_level_aliases()
{'WARN': 'WARNING', 'FATAL': 'CRITICAL'}
Parse text styles encoded in a string into a nested data structure.
Parameters: | text – The encoded styles (a string). |
---|---|
Returns: | A dictionary in the structure of the DEFAULT_FIELD_STYLES and DEFAULT_LEVEL_STYLES dictionaries. |
Here’s an example of how this function works:
>>> from coloredlogs import parse_encoded_styles
>>> from pprint import pprint
>>> encoded_styles = 'debug=green;warning=yellow;error=red;critical=red,bold'
>>> pprint(parse_encoded_styles(encoded_styles))
{'debug': {'color': 'green'},
'warning': {'color': 'yellow'},
'error': {'color': 'red'},
'critical': {'bold': True, 'color': 'red'}}
Find the host name to include in log messages.
Parameters: | use_chroot – Use the name of the chroot when inside a chroot? (boolean, defaults to True) |
---|---|
Returns: | A suitable host name (a string). |
Looks for CHROOT_FILES that have a nonempty first line (taken to be the chroot name). If none are found then socket.gethostname() is used as a fall back.
Select a suitable program name to embed in log messages.
Returns: | One of the following strings (in decreasing order of preference):
|
---|
Prepare to replace a handler.
Parameters: |
|
---|---|
Returns: | A tuple of two values:
|
Find a (specific type of) handler in the propagation tree of a logger.
Parameters: |
|
---|---|
Returns: | A tuple of two values:
|
This function finds a logging handler (of the given type) attached to a logger or one of its parents (see walk_propagation_tree()). It uses the undocumented handlers attribute to find handlers attached to a logger, however it won’t raise an exception if the attribute isn’t available. The advantages of this approach are:
Identify stream handlers writing to the given streams(s).
Parameters: |
|
---|---|
Returns: | True if the handler is a StreamHandler logging to the given stream(s), False otherwise. |
This function can be used as a callback for find_handler().
Walk through the propagation hierarchy of the given logger.
Parameters: | logger – The logger whose hierarchy to walk (a Logger object). |
---|---|
Returns: | A generator of Logger objects. |
Note
This uses the undocumented logging.Logger.parent attribute to find higher level loggers, however it won’t raise an exception if the attribute isn’t available.
Log Formatter that supports %f for millisecond formatting.
This class extends Formatter to enable the use of %f for millisecond formatting in date/time strings, to allow for the type of flexibility requested in issue #45.
Format the date/time of a log record.
Parameters: |
|
---|---|
Returns: | The formatted date/time (a string). |
This method overrides formatTime() to set datefmt to DEFAULT_DATE_FORMAT when the caller hasn’t specified a date format.
When datefmt contains the token %f it will be replaced by the value of %(msecs)03d (refer to issue #45 for use cases).
Log Formatter that uses ANSI escape sequences to create colored logs.
ColoredFormatter inherits from BasicFormatter to enable the use of %f for millisecond formatting in date/time strings.
Note
If you want to use ColoredFormatter on Windows then you may need to call colorama.init(). This is done for you when you call coloredlogs.install().
Initialize a ColoredFormatter object.
Parameters: |
|
---|---|
Raises: | Refer to check_style(). |
This initializer uses colorize_format() to inject ANSI escape sequences in the log format string before it is passed to the initializer of the base class.
Rewrite a logging format string to inject ANSI escape sequences.
Parameters: |
|
---|---|
Returns: | The logging format string with ANSI escape sequences. |
This method takes a logging format string like the ones you give to logging.Formatter and processes it as follows:
As an example consider the default log format (DEFAULT_LOG_FORMAT):
%(asctime)s %(hostname)s %(name)s[%(process)d] %(levelname)s %(message)s
The default field styles (DEFAULT_FIELD_STYLES) define a style for the name field but not for the process field, however because both fields are part of the same whitespace delimited token they’ll be highlighted together in the style defined for the name field.
Apply level-specific styling to log records.
Parameters: | record – A LogRecord object. |
---|---|
Returns: | The result of logging.Formatter.format(). |
This method injects ANSI escape sequences that are specific to the level of each log record (because such logic cannot be expressed in the syntax of a log format string). It works by making a copy of the log record, changing the msg field inside the copy and passing the copy into the format() method of the base class.
An empty class used to copy LogRecord objects without reinitializing them.
Log filter to enable the %(hostname)s format.
Python’s logging module doesn’t expose the system’s host name while I consider this to be a valuable addition. Fortunately it’s very easy to expose additional fields in format strings: filter() simply sets the hostname attribute of each LogRecord object it receives and this is enough to enable the use of the %(hostname)s expression in format strings.
You can install this log filter as follows:
>>> import coloredlogs, logging
>>> handler = logging.StreamHandler()
>>> handler.addFilter(coloredlogs.HostNameFilter())
>>> handler.setFormatter(logging.Formatter('[%(hostname)s] %(message)s'))
>>> logger = logging.getLogger()
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.INFO)
>>> logger.info("Does it work?")
[peter-macbook] Does it work?
Of course coloredlogs.install() does all of this for you :-).
Install the HostNameFilter on a log handler (only if needed).
Parameters: |
|
---|
If fmt is given the filter will only be installed if fmt uses the hostname field. If fmt is not given the filter is installed unconditionally.
Initialize a HostNameFilter object.
Parameters: | use_chroot – Refer to find_hostname(). |
---|
Log filter to enable the %(programname)s format.
Python’s logging module doesn’t expose the name of the currently running program while I consider this to be a useful addition. Fortunately it’s very easy to expose additional fields in format strings: filter() simply sets the programname attribute of each LogRecord object it receives and this is enough to enable the use of the %(programname)s expression in format strings.
Refer to HostNameFilter for an example of how to manually install these log filters.
Install the ProgramNameFilter (only if needed).
Parameters: |
|
---|
If fmt is given the filter will only be installed if fmt uses the programname field. If fmt is not given the filter is installed unconditionally.
Initialize a ProgramNameFilter object.
Parameters: | programname – The program name to use (defaults to the result of find_program_name()). |
---|
A StreamHandler that gets the value of sys.stderr for each log message.
The StandardErrorHandler class enables monkey patching of sys.stderr. It’s basically the same as the logging._StderrHandler class present in Python 3 but it will available regardless of Python version. This handler is used by coloredlogs.install() to improve compatibility with the Python standard library.
Initialize a StandardErrorHandler object.
Shallow logging format string parser.
This class enables introspection and manipulation of logging format strings in the three styles supported by the logging module starting from Python 3.2 (%, { and $).
Initialize a FormatStringParser object.
Parameters: | style – One of the characters %, { or $ (defaults to DEFAULT_FORMAT_STYLE). |
---|---|
Raises: | Refer to check_style(). |
Get the field names referenced by a format string.
Parameters: | format_string – The logging format string. |
---|---|
Returns: | A list of strings with field names. |
Get the field names referenced by a format string.
Parameters: | format_string – The logging format string. |
---|---|
Returns: | A list of strings with field names. |
Group the results of get_pairs() separated by whitespace.
Parameters: | format_string – The logging format string. |
---|---|
Returns: | A list of lists of FormatStringToken objects. |
Tokenize a logging format string and extract field names from tokens.
Parameters: | format_string – The logging format string. |
---|---|
Returns: | A generator of FormatStringToken objects. |
A named tuple for the results of FormatStringParser.get_pairs().
The field name referenced in text (a string). If text doesn’t contain a formatting directive this will be None.
The text extracted from the logging format string (a string).
Responsible for normalizing field and level names.
Initialize a NameNormalizer object.
Normalize a field or level name.
Parameters: | name – The field or level name (a string). |
---|---|
Returns: | The normalized name (a string). |
Transforms all strings to lowercase and resolves level name aliases (refer to find_level_aliases()) to their canonical name:
>>> from coloredlogs import NameNormalizer
>>> from humanfriendly import format_table
>>> nn = NameNormalizer()
>>> sample_names = ['DEBUG', 'INFO', 'WARN', 'WARNING', 'ERROR', 'FATAL', 'CRITICAL']
>>> print(format_table([(n, nn.normalize_name(n)) for n in sample_names]))
-----------------------
| DEBUG | debug |
| INFO | info |
| WARN | warning |
| WARNING | warning |
| ERROR | error |
| FATAL | critical |
| CRITICAL | critical |
-----------------------
Normalize the keys of a dictionary using normalize_name().
Parameters: | value – The dictionary to normalize. |
---|---|
Returns: | A dictionary with normalized keys. |
Get a value from a dictionary after normalizing the key.
Parameters: |
|
---|---|
Returns: | The value of the normalized key (if any). |
Convert text with ANSI escape sequences to HTML.
Capture the output of an external command as if it runs in an interactive terminal.
Parameters: |
|
---|---|
Returns: | The output of the command. |
This function runs an external command under script (emulating an interactive terminal) to capture the output of the command as if it was running in an interactive terminal (including ANSI escape sequences).
Convert text with ANSI escape sequences to HTML.
Parameters: |
|
---|---|
Returns: | The text converted to HTML (a string). |
Encode whitespace so that web browsers properly render it.
Parameters: |
|
---|---|
Returns: | The text converted to HTML (a string). |
The purpose of this function is to encode whitespace in such a way that web browsers render the same whitespace regardless of whether ‘preformatted’ styling is used (by wrapping the text in a <pre>...</pre> element).
Note
While the string manipulation performed by this function is specifically intended not to corrupt the HTML generated by convert() it definitely does have the potential to corrupt HTML from other sources. You have been warned :-).
Replace runs of multiple spaces with non-breaking spaces.
Parameters: | match – A regular expression match object. |
---|---|
Returns: | The replacement string. |
This function is used by func:encode_whitespace() as a callback for replacement using a regular expression pattern.
Encode characters with a special meaning as HTML.
Parameters: | text – The plain text (a string). |
---|---|
Returns: | The text converted to HTML (a string). |
Convert a CSS color in hexadecimal notation into its R, G, B components.
Parameters: | value – A CSS color in hexadecimal notation (a string like ‘#000000’). |
---|---|
Returns: | A tuple with three integers (with values between 0 and 255) corresponding to the R, G and B components of the color. |
Raises: | ValueError on values that can’t be parsed. |
Choose a suitable color for the inverse text style.
Parameters: |
|
---|---|
Returns: | A CSS color in hexadecimal notation (a string). |
In inverse mode the color that is normally used for the text is instead used for the background, however this can render the text unreadable. The purpose of select_text_color() is to make an effort to select a suitable text color. Based on http://stackoverflow.com/a/3943023/112731.
Easy to use integration between coloredlogs and the UNIX cron daemon.
By using ColoredCronMailer as a context manager in the command line interface of your Python program you make it trivially easy for users of your program to opt in to HTML output under cron: The only thing the user needs to do is set CONTENT_TYPE="text/html" in their crontab!
Under the hood this requires quite a bit of magic and I must admit that I developed this code simply because I was curious whether it could even be done :-). It requires my capturer package which you can install using pip install 'coloredlogs[cron]'. The [cron] extra will pull in the capturer 2.4 or newer which is required to capture the output while silencing it - otherwise you’d get duplicate output in the emails sent by cron.
Initialize output capturing when running under cron with the correct configuration.
Stop capturing output and convert the output to HTML (when applicable).
Tell __exit__() to swallow all output (things will be silent).
This can be useful when a Python program is written in such a way that it has already produced output by the time it becomes apparent that nothing useful can be done (say in a cron job that runs every few minutes :-p). By calling silence() the output can be swallowed retroactively, avoiding useless emails from cron.
Easy to use UNIX system logging for Python’s logging module.
Admittedly system logging has little to do with colored terminal output, however:
So there :-P. Go take a look at enable_system_logging().
The pathname of the log device on Mac OS X (a string).
The pathname of the log device on Linux and most other UNIX systems (a string).
The default format for log messages sent to the system log (a string).
The %(programname)s format requires ProgramNameFilter but enable_system_logging() takes care of this for you.
The name[pid]: construct (specifically the colon) in the format allows rsyslogd to extract the $programname from each log message, which in turn allows configuration files in /etc/rsyslog.d/*.conf to filter these log messages to a separate log file (if the need arises).
Context manager to enable system logging.
Initialize a SystemLogging object.
Parameters: |
|
---|
Redirect logging messages to the system log (e.g. /var/log/syslog).
Parameters: |
|
---|---|
Returns: | A SysLogHandler object or None. If an existing handler is found and reconfigure is False the existing handler object is returned. If the connection to the system logging daemon fails None is returned. |
Note
When the logger’s effective level is too restrictive it is relaxed (refer to notes about log levels for details).
Create a SysLogHandler.
Parameters: |
|
---|---|
Returns: | A SysLogHandler object or None (if the system logging daemon is unavailable). |
The process of connecting to the system logging daemon goes as follows:
If SysLogHandler supports the socktype option (it does since Python 2.7) the following two socket types are tried (in decreasing preference):
- SOCK_RAW avoids truncation of log messages but may not be supported.
- SOCK_STREAM (TCP) supports longer messages than the default (which is UDP).
If socket types are not supported Python’s (2.6) defaults are used to connect to the selected address.
Find the most suitable destination for system log messages.
Returns: | The pathname of a log device (a string) or an address/port tuple as supported by SysLogHandler. |
---|
On Mac OS X this prefers LOG_DEVICE_MACOSX, after that LOG_DEVICE_UNIX is checked for existence. If both of these device files don’t exist the default used by SysLogHandler is returned.
Identify system logging handlers.
Parameters: | handler – The Handler class to check. |
---|---|
Returns: | True if the handler is a SysLogHandler, False otherwise. |
This function can be used as a callback for find_handler().