Python set non blocking

Blocking and Non-Blocking Socket I/O

In client server applications, when a client makes a request to a server, server processes the request and sends back a response. For this, both the client and the server first needs to establish a connection with one another through sockets (TCP or UDP). In the last few tutorials, we also saw, how a client can send data in form of request to the server and the server can operate on it, and then send a response back to the client.

Blocking Socket I/O

By default, TCP sockets are placed in a blocking mode. This means that the control is not returned to your program until some specific operation is complete.

For example, if you call the connect() method, the connection blocks your program until the operation is complete. On many occasions, we don’t want to keep our program waiting forever.

Taking another example, when we write a web browser client that connects to a web server, we should consider a stop functionality that can cancel an active connection process in the middle of its operation. This can be achieved by placing the socket in the non-blocking mode.

Non-Blocking Socket I/O

We can call setblocking(1) to set up blocking or setblocking(0) to unset blocking. Let’s understand it with the help of an example. First of all, let’s consider a Blocking Socket:

#!usr/bin/python import socket sock = socket.socket() host = socket.gethostname() sock.connect((host, 12345)) sock.setblocking(1) # Or simply omit this line as by default TCP sockets # are in blocking mode data = "Hello Python\n" *10*1024*1024 # Huge amount of data to be sent assert sock.send(data) # Send data till true

#!usr/bin/python #block_server.py import socket s = socket.socket() host = socket.gethostname() port = 12345 s.bind((host,port)) s.listen(5) while True: conn, addr = s.accept() # accept the connection data = conn.recv(1024) while data: # till data is coming print data data = conn.recv(1024) print "All Data Received" # Will execute when all data is received conn.close() break

Now, run block_server.py first and then block_client.py . You’ll notice that the server keeps on printing Hello Python. This will go on and on till all the data is sent. In the above code the line All Data Received will not be printed for long time, because as the client has to send a large number of string, which will take time, and until then the socket input-output will get blocked.

What’s going on here? The send() method will try to transmit all the data to the server, while the write buffer will get filled up. The kernel will put the process to sleep until the data in the buffer is transferred to the destination and the buffer is empty again. When the buffer becomes empty, the kernel will wake the process up again to get the next chunk of data that is to be transferred. In short, your code will block and it will not let anything else proceed.

Now consider a Non-Blocking Socket

#!usr/bin/python # non_blocking_client.py import socket sock = socket.socket() host = socket.gethostname() sock.connect((host, 12345)) sock.setblocking(0) # Now setting to non-blocking mode data = "Hello Python\n" *10*1024*1024 # Huge amount of data to be sent assert sock.send(data) # Send data till true

Now, if we run the non_blocking_client.py , you’ll notice that the program will run for a small time, it will print the last line «All Data Received» and soon terminate.

What’s going on here? Here the client did not send all the data. When we make a socket non-blocking by calling setblocking(0) , it will never wait for the operation to complete. So when we call the send() method, it will put as much data in the buffer as possible and return.

Источник

White Python

By default a socket is configured so that sending or receiving data blocks, stopping program execution until the socket is ready (setblocking(1)). Calls to send() wait for buffer space to be available for the outgoing data, and calls to recv() wait for the other program to send data that can be read. This form of I/O operation is easy to understand, but can lead to inefficient operation and even deadlocks, if both programs end up waiting for the other to send or receive data.

There are a few ways to work around this situation. One is to use a separate thread for communicating with each socket. This can introduce other complexities, though, with communication between the threads.

Another option is to change the socket to not block at all, and return immediately if it is not ready to handle the operation. Use the setblocking() method to change the blocking flag for a socket. The default value is 1, which means to block. Passing a value of 0 turns off blocking. If the socket is has blocking turned off and it is not ready for the operation, then socket.error is raised.

A compromise solution is to set a timeout value for socket operations. Use settimeout() to change the timeout of a socket to a floating point value representing the number of seconds to block before deciding the socket is not ready for the operation. When the timeout expires, a timeout exception is raised.

In the case of a non blocking socket that has no data available, recv will throw the socket.error exception and the value of the exception will have the errno of either EAGAIN or EWOULDBLOCK.

 import sys import socket import fcntl, os import errno from time import sleep s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect(('127.0.0.1',9999)) fcntl.fcntl(s, fcntl.F_SETFL, os.O_NONBLOCK) while True: try: msg = s.recv(4096) except socket.error, e: err = e.args[0] if err == errno.EAGAIN or err == errno.EWOULDBLOCK: sleep(1) print 'No data available' continue else: # a "real" error occurred print e sys.exit(1) else: # got a message, do something :)

The situation is a little different in the case where you’ve enabled non-blocking behavior via a time out with s.settimeout(n)

 import sys import socket from time import sleep s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect(('127.0.0.1',9999)) s.settimeout(2) while True: try: msg = s.recv(4096) except socket.timeout, e: err = e.args[0] # this next if/else is a bit redundant, but illustrates how the # timeout exception is setup if err == 'timed out': sleep(1) print 'recv timed out, retry later' continue else: print e sys.exit(1) except socket.error, e: # Something else happened, handle error, exit, etc. print e sys.exit(1) else: if len(msg) == 0: print 'orderly shutdown on server end' sys.exit(0) else: # got a message do something :) 

 import select mysocket.setblocking(0) ready = select.select([mysocket], [], [], timeout_in_seconds) if ready[0]: data = mysocket.recv(4096) 

Источник