There is little worse as a developer than trying to figure out why an application is not working if you don’t know what is going on inside it. Sometimes you can’t even tell whether the system is working as designed at all.
When applications are running in production, they become black boxes that need to be traced and monitored. One of the simplest, yet most important ways to do so is by logging. Logging allows us—at the time we develop our software—to instruct the program to emit information while the system is running that will be useful for us and our sysadmins.
The same way we document code for future developers, we should direct new software to generate adequate logs for developers and sysadmins. Logs are a critical part of the system documentation about an application’s runtime status. When instrumenting your software with logs, think it like writing documentation for developers and sysadmins who will maintain the system in the future.
Some purists argue that a disciplined developer who uses logging and testing should hardly need an interactive debugger. If we cannot reason about our application during development with verbose logging, it will be even harder to do it when our code is running in production.
This article looks at Python’s logging module, its design, and ways to adapt it for more complex use cases. This is not intended as documentation for developers, rather as a guide to show how the Python logging module is built and to encourage the curious to delve deeper.
Why use the logging module?
A developer might argue, why aren’t simple print statements sufficient? The loggingmodule offers multiple benefits, including:
- Multi-threading support
- Categorization via different levels of logging
- Flexibility and configurability
- Separation of the how from the what
This last point, the actual separation of the what we log from the how we log enables collaboration between different parts of the software. As an example, it allows the developer of a framework or library to add logs and let the sysadmin or person in charge of the runtime configuration decide what should be logged at a later point.