Python logging write traceback to file
These all still produce UnicodeEncodeError: Add at the top of settings.py Add the Byte Order Mark (BOM) character \ufeff to the formatter as suggested by the python 3 logging cookbook Use %r instead of %s in the log formatter Use the python unicode string on the formatter and then as suggested in this thread Question: I’m using the python logging module with the «native» configuration file support (config.fileconfig) as describe in the documentation here : http://docs.python.org/library/logging.html (see the logging.conf file) I was wondering if it’s possible to supply a tabulated data format in the configuration file: The sample configuration file is the following: I though that using the \t in the format would be enough So I think what’s happening is that when Django tries to print the SQL insert statement it hits a non-printable Unicode character and throws an error.
Python: traceback.print_stack(): How to colorize and reformat output
I want to see the full trace of the code till a particular point
. import traceback traceback.print_stack() . File ".venv/lib/python3.7/site-packages/django/db/models/query.py", line 144, in __iter__ return compiler.results_iter(tuple_expected=True, chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File ".venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1052, in results_iter results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size) File ".venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql cursor.execute(sql, params) File ".venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 110, in execute extra= File "/usr/lib64/python3.7/logging/__init__.py", line 1371, in debug self._log(DEBUG, msg, args, **kwargs) File "/usr/lib64/python3.7/logging/__init__.py", line 1519, in _log self.handle(record) File "/usr/lib64/python3.7/logging/__init__.py", line 1528, in handle if (not self.disabled) and self.filter(record): File "/usr/lib64/python3.7/logging/__init__.py", line 762, in filter result = f.filter(record) File "basic_django/settings.py", line 402, in filter traceback.print_stack() How to make this output more colorful using pygments.
Generally to colorize a json string in python i do
from pygments import highlight from pygments.lexers import JsonLexer from pygments.formatters import TerminalTrueColorFormatter json_str = '< "name":"John" >' print(highlight(json_str, JsonLexer(), TerminalTrueColorFormatter())) Similarly how to do that with traceback.print_stack()
Answer I Used based on Alexander Huszagh
1) we have to use Python3TracebackLexer
2) we have to use traceback.format_stack() which gives a list and then concatenate them as a string using ».join(traceback.format_stack()) .
import traceback import pygments from pygments.lexers import Python3TracebackLexer from pygments.formatters import TerminalTrueColorFormatter traceback_color = pygments.highlight(''.join(traceback.format_stack()),Python3TracebackLexer(),TerminalTrueColorFormatter(style='trac')) # trac or rainbow_dash i prefer print(traceback_color) Pygments lists the available lexers. You can do this with Python3TracebackLexer.
from pygments import highlight from pygments.lexers import Python3TracebackLexer from pygments.formatters import TerminalTrueColorFormatter err_str = ''' File ".venv/lib/python3.7/site-packages/django/db/models/query.py", line 144, in __iter__ return compiler.results_iter(tuple_expected=True, chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File ".venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1052, in results_iter results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size) File ".venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql cursor.execute(sql, params) File ".venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 110, in execute extra= File "/usr/lib64/python3.7/logging/__init__.py", line 1371, in debug self._log(DEBUG, msg, args, **kwargs) File "/usr/lib64/python3.7/logging/__init__.py", line 1519, in _log self.handle(record) File "/usr/lib64/python3.7/logging/__init__.py", line 1528, in handle if (not self.disabled) and self.filter(record): File "/usr/lib64/python3.7/logging/__init__.py", line 762, in filter result = f.filter(record) File "basic_django/settings.py", line 402, in filter traceback.print_stack() ''' print(highlight(err_str, Python3TracebackLexer(), TerminalTrueColorFormatter())) In order to get err_str , replace print_stack with format_stack as follows than do:
def colorize_traceback(err_str): return highlight(err_str, Python3TracebackLexer(), TerminalTrueColorFormatter()) try: . # Some logic except Exception: # Or a more narrow exception # tb.print_stack() print(colorize_traceback(tb.format_stack())) Alternatively, use the rich library.
With just two lines of code, it will prettify your tracebacks. and then some!
from rich.traceback import install install() How does it look afterwards? Take a gander:
And the beauty of it? It supports Pygment themes!
How to set the same logging format for all handlers?, The intended way is to attach the formatter to each handler as you’re creating them. Since you’re supposed to set up logging destinations in
Django UnicodeEncodeError when logging sql with unicode from BinaryField
I have a Django model with a BinaryField. When I save an instance of the model to the database Django prints out an error message like this:
UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 216: character maps to
The last line of the traceback indicates that the error is occurring in django.db.backends.utils.py — I’ve added a comment to the offending line:
class CursorDebugWrapper(CursorWrapper): # XXX callproc isn't instrumented at this time. def execute(self, sql, params=None): start = time() try: return super(CursorDebugWrapper, self).execute(sql, params) finally: stop = time() duration = stop - start sql = self.db.ops.last_executed_query(self.cursor, sql, params) self.db.queries_log.append(< 'sql': sql, 'time': "%.3f" % duration, >) ##### Error is reported from the logger.debug statement logger.debug( '(%.3f) %s; args=%s', duration, sql, params, extra= ) So I think what’s happening is that when Django tries to print the SQL insert statement it hits a non-printable Unicode character and throws an error. I don’t want to disable Django debug logging while I’m developing (it is disabled in production, of course). Any way to work around this issue?
Setting the encoding of the log handler to utf-8 seems to work. If there’s a better way please advise.
Things I tried that didn’t work. These all still produce UnicodeEncodeError:
- Add from __future__ import unicode_literals at the top of settings.py
- Add the Byte Order Mark (BOM) character \ufeff to the formatter as suggested by the python 3 logging cookbook
- Use %r instead of %s in the log formatter
- Use the python unicode string on the formatter ‘format’: u’%(asctime)-s %(levelname)s [%(name)s]: %(message)s’, and then logging._defaultFormatter = logging.Formatter(u»%(message)s») as suggested in this thread
Report Server Service Trace Log, The timestamp is based on Coordinated Universal Time (UTC). The file is in EN-US format. By default, trace logs are limited to 32 megabytes and
Using tabulation in Python logging format
I’m using the python logging module with the «native» configuration file support (config.fileconfig) as describe in the documentation here :
http://docs.python.org/library/logging.html (see the logging.conf file)
I was wondering if it’s possible to supply a tabulated data format in the configuration file:
The sample configuration file is the following:
[formatter_simpleFormatter] format=%(asctime)s - %(name)s - %(levelname)s - %(message)s I though that using the \t in the format would be enough but it doesn’t:
format=%(asctime)s\t%(name)s\t%(levelname)s\t%(message)s\t I tried a couple of things without success. I suppose it’s really easy to do but I don’t find it!
Sorry for coming late to the party , but the info could be useful for others also .
I also wanted a tabulated looking log, especially the «levelname» field
my format was looking like this
format = %(asctime)s - %(levelname)s - %(name)s - %(message)s which made my logs look something like this
2014-10-01 17:42:54,261 - INFO - internal. 2014-10-01 17:43:09,700 - DEBUG - internal. 2014-10-01 17:44:02,994 - WARNING - internal. 2014-10-01 17:44:31,686 - CRTITICAL - internal. my solution was to change the format like this
format = %(asctime)s - %(levelname)-8s - %(name)s - %(message)s which turned my logs in something like this
2014-10-01 17:42:54,261 - INFO - internal. 2014-10-01 17:43:09,700 - DEBUG - internal. 2014-10-01 17:44:02,994 - WARNING - internal. 2014-10-01 17:44:31,686 - CRITICAL - internal. The «8» is the length of the longest string that is expected there, in this case, «CRITICAL». The «-» tells to right-pad the string
. the string doesn’t get truncated
Have you tried entering a literal tab character in the config file instead of \t ? This works for me.
How do I write Flask’s excellent debug log message to a file in, Whenever a view raises an exception, this method will be called and passed the exception as argument. Python logging provides exception method that is used