- Catching an exception while using a Python ‘with’ statement
- 6 Answers 6
- Catching an exception while using a Python ‘with’ statement
- Differentiating between the possible origins of exceptions raised from a compound with statement
- Alternative approach using the equivalent form mentioned in PEP 343
- Usually a simpler approach will do just fine
- python exception handling inside with block
- 1 Answer 1
Catching an exception while using a Python ‘with’ statement
Enclosing with in a try/except statement doesn’t work either, and an exception is not raised. What can I do in order to process failure inside with statement in a Pythonic way?
What do you mean «enclosing ‘with’ in a try/except statement doesn’t work else: exception is not raised»? A with statement doesn’t magically break a surrounding try. except statement.
Interestingly, Java’s try-with-resources statement does support exactly this use case you want. docs.oracle.com/javase/tutorial/essential/exceptions/…
Perhaps this wasn’t possible in older versions of Python, but for 3.8, it seems putting with in the try statement does allow it to handle except FileNotFoundError
6 Answers 6
from __future__ import with_statement try: with open( "a.txt" ) as f : print f.readlines() except EnvironmentError: # parent of IOError, OSError *and* WindowsError where available print 'oops' If you want different handling for errors from the open call vs the working code you could do:
try: f = open('foo.txt') except IOError: print('error') else: with f: print f.readlines() As noted in stackoverflow.com/questions/5205811/…, the try block here is really too broad. No distinction is made between exceptions while creating the context manager and those in the body of the with statement, so it may not be a valid solution for all use cases.
@rbaleksandar If I recall correctly, my comment was strictly referring to the first example in the answer, where the entire with statement is inside the try/except block (so even if you have inner try/expect blocks, any exceptions they let escape will still hit the outer one). Douglas subsequently added the second example to address cases where that distinction matters.
@MikeCollins Exiting the ‘with’ will close the open file even when the file is open before the ‘with’.
In the 1st example, use OSError for Python 3.3+. As shown here: Changed in version 3.3: EnvironmentError, IOError, WindowsError, socket.error, select.error and mmap.error have been merged into OSError, and the constructor may return a subclass.
The best «Pythonic» way to do this, exploiting the with statement, is listed as Example #6 in PEP 343, which gives the background of the statement.
@contextmanager def opened_w_error(filename, mode="r"): try: f = open(filename, mode) except IOError, err: yield None, err else: try: yield f, None finally: f.close() with opened_w_error("/etc/passwd", "a") as (f, err): if err: print "IOError:", err else: f.write("guido::0:0::/:/bin/sh\n") @PaulSeeb Why wouldn’t you define it and save yourself from doing it each time you need to? It’s defined at your application level, and it’s as magic as any other contextmanager. I think someone using the with statement would understand it clearly (the name of the function might also be more expressive if you don’t like it). The «with» statement itself has been engineered to work this way, to define a «secure» block of code and delegate checking functions to context managers (to make the code more clear).
All this trouble just for not writing the finally block in the user code. I’m beginning to think we all suffer for a long hype symptom on the with statement.
The best way to handle exceptions in python is to write a function that catches and returns them? Seriously? The pythonic way to handle exceptions is to use a try. except statement.
This is brilliant, thank you! It’s the only answer that doesn’t violate the rule against having more than 1 line in a try statement and that also allows me to keep the with statement instead of opening and closing the connection separately.
Catching an exception while using a Python ‘with’ statement
The with statement has been available without the __future__ import since Python 2.6. You can get it as early as Python 2.5 (but at this point it’s time to upgrade!) with:
from __future__ import with_statement Here’s the closest thing to correct that you have. You’re almost there, but with doesn’t have an except clause:
with open("a.txt") as f: print(f.readlines()) except: # A context manager's __exit__ method, if it returns False will reraise the error when it finishes. If it returns True , it will suppress it. The open builtin's __exit__ doesn't return True , so you just need to nest it in a try, except block:
try: with open("a.txt") as f: print(f.readlines()) except Exception as error: print('oops') And standard boilerplate: don't use a bare except: which catches BaseException and every other possible exception and warning. Be at least as specific as Exception , and for this error, perhaps catch IOError . Only catch errors you're prepared to handle.
>>> try: . with open("a.txt") as f: . print(f.readlines()) . except IOError as error: . print('oops') . oops You cannot use f in the except statements to somehow "correct" the behavior since you're out of the context where it was created. (In my case I'm catching a warning and solve the related issue in the except block). To achieve this, I think only reimplementing a context manager like in @user765144 's answer does the job.
Differentiating between the possible origins of exceptions raised from a compound with statement
Differentiating between exceptions that occur in a with statement is tricky because they can originate in different places. Exceptions can be raised from either of the following places (or functions called therein):
- ContextManager.__init__
- ContextManager.__enter__
- the body of the with
- ContextManager.__exit__
For more details see the documentation about Context Manager Types.
If we want to distinguish between these different cases, just wrapping the with into a try .. except is not sufficient. Consider the following example (using ValueError as an example but of course it could be substituted with any other exception type):
try: with ContextManager(): BLOCK except ValueError as err: print(err) Here the except will catch exceptions originating in all of the four different places and thus does not allow to distinguish between them. If we move the instantiation of the context manager object outside the with , we can distinguish between __init__ and BLOCK / __enter__ / __exit__ :
try: mgr = ContextManager() except ValueError as err: print('__init__ raised:', err) else: try: with mgr: try: BLOCK except TypeError: # catching another type (which we want to handle here) pass except ValueError as err: # At this point we still cannot distinguish between exceptions raised from # __enter__, BLOCK, __exit__ (also BLOCK since we didn't catch ValueError in the body) pass Effectively this just helped with the __init__ part but we can add an extra sentinel variable to check whether the body of the with started to execute (i.e. differentiating between __enter__ and the others):
try: mgr = ContextManager() # __init__ could raise except ValueError as err: print('__init__ raised:', err) else: try: entered_body = False with mgr: entered_body = True # __enter__ did not raise at this point try: BLOCK except TypeError: # catching another type (which we want to handle here) pass except ValueError as err: if not entered_body: print('__enter__ raised:', err) else: # At this point we know the exception came either from BLOCK or from __exit__ pass The tricky part is to differentiate between exceptions originating from BLOCK and __exit__ because an exception that escapes the body of the with will be passed to __exit__ which can decide how to handle it (see the docs). If however __exit__ raises itself, the original exception will be replaced by the new one. To deal with these cases we can add a general except clause in the body of the with to store any potential exception that would have otherwise escaped unnoticed and compare it with the one caught in the outermost except later on - if they are the same this means the origin was BLOCK or otherwise it was __exit__ (in case __exit__ suppresses the exception by returning a true value the outermost except will simply not be executed).
try: mgr = ContextManager() # __init__ could raise except ValueError as err: print('__init__ raised:', err) else: entered_body = exc_escaped_from_body = False try: with mgr: entered_body = True # __enter__ did not raise at this point try: BLOCK except TypeError: # catching another type (which we want to handle here) pass except Exception as err: # this exception would normally escape without notice # we store this exception to check in the outer `except` clause # whether it is the same (otherwise it comes from __exit__) exc_escaped_from_body = err raise # re-raise since we didn't intend to handle it, just needed to store it except ValueError as err: if not entered_body: print('__enter__ raised:', err) elif err is exc_escaped_from_body: print('BLOCK raised:', err) else: print('__exit__ raised:', err) Alternative approach using the equivalent form mentioned in PEP 343
PEP 343 -- The "with" Statement specifies an equivalent "non-with" version of the with statement. Here we can readily wrap the various parts with try . except and thus differentiate between the different potential error sources:
import sys try: mgr = ContextManager() except ValueError as err: print('__init__ raised:', err) else: try: value = type(mgr).__enter__(mgr) except ValueError as err: print('__enter__ raised:', err) else: exit = type(mgr).__exit__ exc = True try: try: BLOCK except TypeError: pass except: exc = False try: exit_val = exit(mgr, *sys.exc_info()) except ValueError as err: print('__exit__ raised:', err) else: if not exit_val: raise except ValueError as err: print('BLOCK raised:', err) finally: if exc: try: exit(mgr, None, None, None) except ValueError as err: print('__exit__ raised:', err) Usually a simpler approach will do just fine
The need for such special exception handling should be quite rare and normally wrapping the whole with in a try . except block will be sufficient. Especially if the various error sources are indicated by different (custom) exception types (the context managers need to be designed accordingly) we can readily distinguish between them. For example:
try: with ContextManager(): BLOCK except InitError: # raised from __init__ . except AcquireResourceError: # raised from __enter__ . except ValueError: # raised from BLOCK . except ReleaseResourceError: # raised from __exit__ . python exception handling inside with block
Is the following code doing anything wrong with with statement and exception handling of python3? If no, what is the correct way of writing my expected output?
from contextlib import contextmanager @contextmanager def test(): print("Hello") yield print("goodbye") try: with test(): print("inside test") raise KeyError except KeyError: print("KeyError") else: print("else") finally: print("finally") Hello inside test KeyError finally Hello inside test goodbye KeyError finally which I believe other people write similarly in the hope that the file would be closed when an exception was raised during the processing of the file. My python3 version is:
Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> print(sys.version) 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] 1 Answer 1
The exception from within the block governed by the with statement is propagated to your generator context manager through generator.throw() as shown in PEP 343: "Generator Decorator", which raises the exception at the point where the generator was paused. In other words you should wrap the yield in a try/except or try/finally:
@contextmanager def test(): print("Hello") try: # The block of the with statement executes when the generator yields yield finally: print("goodbye") To quote the official documentation on the subject:
. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a try…except…finally statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the with statement that the exception has been handled, and execution will resume with the statement immediately following the with statement.