Python logging to html

How to Get Started with Logging in Flask

Logging is a crucial component in the software life cycle. It allows you to take a peek inside your application and understand what is happening, which helps you address the problems as they appear. Flask is one of the most popular web frameworks for Python and logging in Flask is based on the standard Python logging module. In this article, you will learn how to create a functional and effective logging system for your Flask application.

🔭 Want to centralize and monitor your Flask application logs?

Head over to Logtail and start ingesting your logs in 5 minutes.

Prerequisites

Before proceeding with this article, ensure that you have a recent version of Python 3 installed on your machine. To best learn the concepts discussed here, you should also create a new Flask project so that you may try out all the code snippets and examples.

Create a new working directory and change into it with the command below:

mkdir flask-logging && cd flask-logging 

Install the latest version of Flask with the following command.

Getting started with logging in Flask

To get started, you need to create a new Flask application first. Go to the root directory of your project and create an app.py file.

from flask import Flask app = Flask(__name__) @app.route("/") def hello(): return "Hello, World!" @app.route("/info") def info(): return "Hello, World! (info)" @app.route("/warning") def warning(): return "A warning message. (warning)" 

In this example, a new instance of the Flask application ( app ) is created and three new routes are defined. When these routes are accessed, different functions will be invoked, and different strings will be returned.

Next, you can add logging calls to the info() and warning() functions so that when they are invoked, a message will be logged to the console.

. . . @app.route("/info") def info(): 
app.logger.info("Hello, World!")
 return "Hello, World! (info)" @app.route("/warning") def warning(): 
app.logger.warning("A warning message.")
 return "A warning message. (warning)" 

The highlighted lines above show how to access the standard Python logging module via app.logger . In this example, the info() method logs Hello, World! at the INFO level, and the warning() method logs «A warning message» at the WARNING level. By default, both messages are logged to the console.

To test this logger, start the dev server using the following command:

 * Debug mode: off WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Running on http://127.0.0.1:5000 Press CTRL+C to quit 

Keep the Flask dev server running and open up a new terminal window. Run the following command to test the /warning route:

curl http://127.0.0.1:5000/warning 

The following text should be returned:

A warning message. (warning) 

And then, go back to the dev server window, and a log message should appear:

[2022-10-17 12:43:33,907] WARNING in app: A warning message. 

As you can see, the output contains a lot more information than just the log message itself. The warning() method will automatically include the timestamp ( [2022-09-24 17:18:06,304] ), the log level ( WARNING ), and the program that logged this message ( app ).

However, if you visit the /info route, you will observe that the «Hello World!» message isn’t logged as expected. That’s because Flask ignores messages with log level lower than WARNING by default, but we’ll show how you can customize this behavior shortly.

One more thing to note is that every time you make changes to your Flask application, such as adding more loggers or modifying related configurations, you need to stop the dev server (by pressing CTRL+C ), and then restart it for the changes to take effect.

Understanding log levels

Log levels are used to indicate how urgent a log record is, and the logging module used under the hood by Flask offers six different log levels, each associated with an integer value: CRITICAL (50), ERROR (40), WARNING (30), INFO (20) and DEBUG (10). You can learn more about log levels and how they are typically used by reading this article.

Each of these log level has a corresponding method, which allows you to send log entry with that log level. For instance:

. . . @app.route("/") def hello(): app.logger.debug("A debug message") app.logger.info("An info message") app.logger.warning("A warning message") app.logger.error("An error message") app.logger.critical("A critical message") return "Hello, World!" 

However, when you run this code, only messages with log level higher than INFO will be logged. That is because you haven’t configured this logger yet, which means Flask will use the default configurations leading to the dropping of the DEBUG and INFO messages.

Remember to restart the server before making a request to the / route:

[2022-07-18 11:47:39,589] WARNING in app: A warning message [2022-07-18 11:47:39,590] ERROR in app: An error message [2022-07-18 11:47:39,590] CRITICAL in app: A critical message 

In the next section, we will discuss how to override the default Flask logging configurations so that you can customize its behavior according to your needs.

Configuring your logging system

Flask recommends that you use the logging.config.dictConfig() method to overwrite the default configurations. Here is an example:

from flask import Flask 
from logging.config import dictConfig
 
dictConfig(
 
  
"version": 1,
 
"formatters":  
"default":  
"format": "[%(asctime)s] %(levelname)s in %(module)s: %(message)s",
 
>
 
>,
 
"handlers":  
"console":  
"class": "logging.StreamHandler",
 
"stream": "ext://sys.stdout",
 
"formatter": "default",
 
>
 
>,
 
"root": ,
 
>
 
)
 app = Flask(__name__) . . . 

Let’s take a closer look at this configuration. First of all, the version key represents the schema version and, at the time this article is written, the only valid option is 1 . Having this key allows the schema format to evolve in the future while maintaining backward compatibility.

Next, the formatters key is where you specify formatting patterns for your log records. In this example, only a default formatter is defined. To define a format, you need to use LogRecord attributes , which always start with a % symbol.

For example, %(asctime)s indicates the timestamp in ASCII encoding, s indicates this attribute corresponds to a string. %(levelname)s is the log level, %(module)s is the name of the module that pushed the message, and finally, %(message)s is the message itself.

Inside the handlers key, you can create different handlers for your loggers. Handlers are used to push log records to various destinations. In this case, a console handler is defined, which uses the logging.StreamHandler library to push messages to the standard output. Also, notice that this handler is using the default formatter you just defined.

Finally, the root key is where you specify configurations for the root logger, which is the default logger unless otherwise specified. «level»: «DEBUG» means this root logger will log any messages higher than or equal to DEBUG , and «handlers»: [«console»] indicates this logger is using the console handler you just saw.

One last thing you should notice in this example is that the configurations are defined before the application ( app ) is initialized. It is recommended to configure logging behavior as soon as possible. If the app.logger is accessed before logging is configured, it will create a default handler instead, which could be in conflict with your configuration.

Formatting your log records

Let’s take a closer look at how to format log records in Flask. In the previous section, we introduced some LogRecord attributes and discussed how you can use them to create custom log messages:

. . . dictConfig( < "version": 1, 
"formatters":  
"default":  
"format": "[%(asctime)s] %(levelname)s | %(module)s >>> %(message)s",
 
>
 
>,
 . . . > ) . . . @app.route("/") def hello(): 
app.logger.info("An info message")
 return "Hello, World!" 

This configuration produces a log record that is formatted like this:

[2022-10-17 13:13:25,484] INFO | app >>> An info message 

Some of the attributes support further customization. For example, you can customize how the timestamp is displayed by adding a datefmt key in the configurations:

. . . dictConfig( < "version": 1, "formatters": < "default": < "format": "[%(asctime)s] %(levelname)s | %(module)s >>> %(message)s", 
"datefmt": "%B %d, %Y %H:%M:%S %Z",
 > >, . . . > ) . . . 

This yields a timestamp in the following format:

[October 17, 2022 13:22:40 Eastern Daylight Time] INFO | app >>> An info message 

You can read this article to learn more about customizing timestamps in Python. Besides %(asctime)s %(levelname)s , %(module)s , and %(message)s , there are several other LogRecord attributes available. You can find all of them in the linked documentation.

Logging to files

Logging to the console is great for development, but you will need a more persistent medium to store log records in production so that you may reference them in the future. A great way to start persisting your logs is to send them to local files on the server. Here’s how to set it up:

. . . dictConfig( < "version": 1, . . . "handlers": < "console": < "class": "logging.StreamHandler", "stream": "ext://sys.stdout", "formatter": "default", >, 
"file":  
"class": "logging.FileHandler",
 
"filename": "flask.log",
 
"formatter": "default",
 
>,
 >, 
"root": ,
 > ) . . . @app.route("/") def hello(): app.logger.debug("A debug message") return "Hello, World!" 

A new file handler is added to the handlers object and it uses the logging.FileHandler class. It also defines a filename which specifies the path to the file where the logs are stored. In the root object, the file handler is also registered so that logs are sent to the console and the configured file.

Once you restart your server, make a request to the /hello and observe that a flask.log file is generated at the root directory of your project. You can view its contents the following command:

. . . [October 17, 2022 13:29:12 Eastern Daylight Time] DEBUG | app >>> A debug message 

Rotating your log files

The FileHandler discussed above does not support log rotation so if you desire to rotate your log files, you can use either RotatingFileHandler or TimedRotatingFileHandler . They take the same parameters as FileHandler with some extra options.

For example, RotatingFileHandler takes two more parameters:

• maxBytes determines the maximum size of each log file. When the size limit is about to be exceeded, the file will be closed, and another file will be automatically created.
• backupCount specifies the number of files that will be retained on the disk, and the older files will be deleted. The retained files will be appended with a number extension .1 , .2 , and so on.

. . . dictConfig( < "version": 1, . . . "handlers": < 
"size-rotate":  
"class": "logging.handlers.RotatingFileHandler",
 
"filename": "flask.log",
 
"maxBytes": 1000000,
 
"backupCount": 5,
 
"formatter": "default",
 
>,
 >, 
"root": ,
 > ) . . . 

Notice that we are using logging.handlers.RotatingFileHandler and not logging.RotatingFileHandler . In this example, this logging system will retain six files, from flask.log , flask.log.1 up to flask.log.5 , and each one has a maximum size of 1MB.

On the other hand, TimedRotatingFileHandler splits the log files based on time. Here’s how to use it:

. . . dictConfig( < "version": 1, . . . "handlers": < 
"time-rotate":  
"class": "logging.handlers.TimedRotatingFileHandler",
 
"filename": "flask.log",
 
"when": "D",
 
"interval": 10,
 
"backupCount": 5,
 
"formatter": "default",
 
>,
 >, "root": < "level": "DEBUG", 
"handlers": ["time-rotate"],
 >, > ) . . . 

The interval specifies the time interval, and when specifies the unit, which could be any of the following:

• «S» : seconds
• «M» : minutes
• «H» : hours
• «D» : days
• «W0» — «W6» : weekdays, «W0» indicates Sunday. You can also specify an atTime option, which determines at what time the rollover happens. The interval option is not used in this case.
• «midnight» : creates a new file at midnight. You can also specify an atTime option, which determines at what time the rollover happens.

When we are using the TimedRotatingFileHandler , the old file will be appended a timestamp extension in the format %Y-%m-%d_%H-%M-%S ( time-rotate.log.2022-07-19_13-02-13 ).

If you need more flexibility when it comes to log rotation, you’re better off using a utility like logrotate instead as Python’s file rotation handlers are not designed for heavy production workloads.

🔭 Want to centralize and monitor your Flask logs?

Head over to Logtail and start ingesting your logs in 5 minutes.

Logging HTTP requests

Since Flask is a web framework, your application will likely be handling many HTTP requests, and logging information about them will help you understand what is happening inside your application. To demonstrate relevant concepts, we’ll setup a demo application where users can search for a location and get its current time, and then we will create a logging system for it (see the logging branch for the final implementation).

Start by cloning the repository to your machine using the following command:

git clone https://github.com/betterstack-community/flask-world-clock.git 

Change into the project directory:

You can check the structure of this project using the tree command:

Источник