Python Logging: How to Log to Multiple Locations
Today I decided to figure out how to make Python log to a file and the console simultaneously. Most of the time, I just want to log to a file, but occasionally I want to be able to see stuff on the console too to help with debugging. I found this ancient example in the Python documentation and ended up using it to mock up the following script:
import logging #---------------------------------------------------------------------- def log(path, multipleLocs=False): """ Log to multiple locations if multipleLocs is True """ fmt_str = '%(asctime)s - %(name)s - %(message)s' formatter = logging.Formatter(fmt_str) logging.basicConfig(filename=path, level=logging.INFO, format=fmt_str) if multipleLocs: console = logging.StreamHandler() console.setLevel(logging.INFO) console.setFormatter(formatter) logging.getLogger("").addHandler(console) logging.info("This is an informational message") try: 1 / 0 except ZeroDivisionError: logging.exception("You can't do that!") logging.critical("THIS IS A SHOW STOPPER. ") if __name__ == "__main__": log("sample.log") # log only to file log("sample2.log", multipleLocs=True) # log to file AND console! As you can see, when you pass True to the second argument, the script will create an instance of StreamHandler() which you can then configure and add to the current logger via the following call:
logging.getLogger("").addHandler(console) This works great on Linux, but on Windows 7 the sample2.log wasn’t getting created, so I had to modify the if statement as follows:
if multipleLocs: console = logging.StreamHandler() console.setLevel(logging.INFO) console.setFormatter(formatter) fhandler = logging.FileHandler(path) fhandler.setFormatter(formatter) logging.getLogger("").addHandler(console) logging.getLogger("").addHandler(fhandler) Now I should note that this causes a rather odd bug in that Python is somehow keeping track of the file names I write to across calls to my log function such that when I tell it to write to sample2.log, it write to it PLUS the original sample.log. To get around this, we have to create a logger instance with a unique name each time we call the script. Here’s an updated example that works correctly:
import logging import os #---------------------------------------------------------------------- def log(path, multipleLocs=False): """ Log to multiple locations if multipleLocs is True """ fname = os.path.splitext(path)[0] logger = logging.getLogger("Test_logger_%s" % fname) logger.setLevel(logging.INFO) fh = logging.FileHandler(path) formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s') fh.setFormatter(formatter) logger.addHandler(fh) if multipleLocs: console = logging.StreamHandler() console.setLevel(logging.INFO) console.setFormatter(formatter) logger.addHandler(console) logger.info("This is an informational message") try: 1 / 0 except ZeroDivisionError: logger.exception("You can't do that!") logger.critical("THIS IS A SHOW STOPPER. ") if __name__ == "__main__": log("sample.log") # log only to file log("sample2.log", multipleLocs=True) # log to file AND console! You will note that this time we base the logger name on the file name of the log. The logging module is pretty slick and lots of fun to play around with. I hope you found that as interesting as I did.