How to send and receive live audio using socket programming in Python
Hi friends! In a previous tutorial we used opencv to obtain video frames of webcam and send them over wifi to server/client. Below is the video about basics of socket programming.
Today, we will move one step further, instead of transmitting video frames over wifi from one computer to another, we will use pyshine to send audio frames. The default audio frame will be of 1024 data samples for each audio channel from the microphone device of a computer. The mechanism is almost similar to that of video transmitting and receiving. So without any delay, let’s install the essentials.
Install pyshine version 0.0.6 in Windows OS as:
Both server and client computers should be on the same wifi router. The required IP address will be for Wifi LAN (inet)
Windows OS users
From the cmd window run this command:
The required IP address will be shown against IPv4 Address
Here is the server side code. First, please change the IP address: ‘192.168.1.105’ to yours, otherwise your server will not start.
server.py
import socket, cv2, pickle,struct,time import pyshine as ps mode = 'send' name = 'SERVER TRANSMITTING AUDIO' audio,context= ps.audioCapture(mode=mode) #ps.showPlot(context,name) # Socket Create server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM) host_ip = '192.168.1.105' port = 4982 backlog = 5 socket_address = (host_ip,port) print('STARTING SERVER AT',socket_address,'. ') server_socket.bind(socket_address) server_socket.listen(backlog) while True: client_socket,addr = server_socket.accept() print('GOT CONNECTION FROM:',addr) if client_socket: while(True): frame = audio.get() a = pickle.dumps(frame) message = struct.pack("Q",len(a))+a client_socket.sendall(message) else: break client_socket.close() Here is code for the client. Please give the IP address of your server accordingly.
client.py
import socket,cv2, pickle,struct import pyshine as ps mode = 'get' name = 'CLIENT RECEIVING AUDIO' audio,context = ps.audioCapture(mode=mode) ps.showPlot(context,name) # create socket client_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM) host_ip = '192.168.1.105' port = 4982 socket_address = (host_ip,port) client_socket.connect(socket_address) print("CLIENT CONNECTED TO",socket_address) data = b"" payload_size = struct.calcsize("Q") while True: while len(data) payload_size: packet = client_socket.recv(4*1024) # 4K if not packet: break data+=packet packed_msg_size = data[:payload_size] data = data[payload_size:] msg_size = struct.unpack("Q",packed_msg_size)[0] while len(data) msg_size: data += client_socket.recv(4*1024) frame_data = data[:msg_size] data = data[msg_size:] frame = pickle.loads(frame_data) audio.put(frame) client_socket.close() To run the code, on server side:
Once the server starts listening
Thats it! If you have questions, suggestions please do comment. Have a nice day!
Welcome to Audiostream’s documentation!¶
Audiostream is a Python extension that provide an easy-to-use API for streaming bytes to the speaker, or read an audio input stream. It use SDL + SDL_Mixer for streaming the audio out, and use platform-api for reading audio input.
This extension works on Android and iOS as well.
Examples¶
Example to generate a wave form using sin() method, and stream to the speaker:
from time import sleep from audiostream import get_output from audiostream.sources.wave import SineSource # get a output stream where we can play samples stream = get_output(channels=2, rate=22050, buffersize=1024) # create one wave sin() at 440Hz, attach it to our speaker, and play sinsource = SineSource(stream, 440) sinsource.start() # you can change the frequency of the source during the playtime for x in xrange(10): sinsource.frequency = 440 + x sleep(.5) # ok we are done, stop everything. sinsource.stop()
Example to read microphone bytes:
from audiostream import get_input # declare a callback where we'll receive the data def callback_mic(data): print 'i got', len(data) # get the microphone (or from another source if available) mic = get_input(callback=callback_mic) mic.start() sleep(5) mic.stop()
To be able to record microphone on Android, you need to have the RECORD_AUDIO permission
Core API¶
Initialize the engine and get an output device. This method can be used only once, usually at the start of your application.
- rate (integer) – Rate of the audio, default to 44100
- channels (integer) – Number of channels, minimum 1, default to 2
- encoding (integer) – Encoding of the audio stream, can be 8 or 16, default to 16
- buffersize (integer) – Size of the output buffer. Tiny buffer will use consume more CPU, but will be more reactive.
from audiostream import get_output stream = get_output(channels=2, rate=22050, buffersize=1024)
audiostream. get_input ( callback : callable, source : string, rate : int, channels : int, encoding : int, buffersize : int ) → `AudioInput` instance¶
Return an AudioInput instance. All the data received from the input will be stored in a queue. You need to AudioInput.poll() the queue regulary in order to trigger the callback.
Please note that the callback will be called in the same thread as the one that call AudioInput.poll() .
- callback (callable) – Callback to call when bytes are available on the input, called from the audio thread.
- source (string) – Source device to read, default to ‘default. Depending of the platform, you might read other input source. Check the get_input_sources() function.
- channels (integer) – Number of channels, minimum 1, default to 2
- encoding (integer) – Encoding of the audio stream, can be 8 or 16, default to 16
- buffersize (integer) – Size of the input buffer. If
from audiostream import get_input def mic_callback(buf): print 'got', len(buf) # get the default audio input (mic on most cases) mic = get_input(callback=mic_callback) mic.start() while not quit: mic.poll() # do something here, like sleep(2) mic.stop()
This function currently work only on Android and iOS.
Return a list of available input sources. This list is platform-dependent. You might need some additionnals permissions in order to access to the sources.
- android: ‘camcorder’, ‘default’, ‘mic’, ‘voice_call’, ‘voice_communication’, ‘voice_downlink’, ‘voice_recognition’, ‘voice_uplink’
- ios: ‘default’
This function currently work only on Android and iOS.
Abstract class for handling an audio input. Normally, the default audio source is the microphone. It will be recorded with a rate of 44100hz, mono, with 16bit PCM. Theses defaults are the most used and guaranted to work on Android and iOS. Any others combination might fail.
Start the input to gather data from the source
Stop the input to gather data from the source
Read the internal queue and dispatch the data through the callback
Callback to call when bytes are available on the input, called from the audio thread. The callback must have one parameter for receiving the data.
(readonly) Encoding of the audio, can be 8 or 16, default to 16
(readonly) Source device to read, default to ‘default. Depending of the platform, you might read other input source. Check the get_input_sources() function.
(readonly) Number of channels, minimum 1, default to 2
class audiostream. AudioOutput ( object ) ¶
Abstract class for handling audio output stream, and handle the mixing of multiple sample. One sample is an instance of AudioSample abstract class. You can implement your own sample that generate bytes, and thoses bytes will be mixed in the final output stream.
We also expose multiple AudioSample implementation, such as:
- audiostream.sources.thread.ThreadSource : base for implementing a generator that run in a thread
- audiostream.sources.wave.SineSource : generate a sine wave
- audiostream.sources.puredata.PatchSource : sample generator that use a Puredata patch (require pylibpd)
Add a sample to manage in the internal mixer. This method is usually called in the AudioSample.start()
remove_sample ( sample : AudioSample ) ¶
| Parameters: | sample ( AudioSample ) – sample managed by the mixer |
|---|
Remove a sample from the internal mixer. This method is usually called in the AudioSample.stop()
class audiostream. AudioSample ¶
AudioSample is a class for generating bytes that will be consumed by AudioOutput’. The data goes first on a RingBuffer, and the buffer is consumed by the speaker, according to the :class:`AudioOutput initialization.
from audiostream import get_output, AudioSample stream = get_output(channels=1, buffersize=1024, rate=22050) sample = AudioSample() stream.add_sample(sample) sample.play() while True: # audio stuff, this is not accurate. sample.write("\\x00\\x00\\x00\\x00\\xff\\xff\\xff\\xff")
If you don’t write enough data (underrun), the library will fill with \x00 . If you write too much (overrun), the write method will block, until the data is consumed.
You should use audiostream.sources.ThreadSource instead.
write ( chunk : bytes ) ¶
| Parameters: | chunk (bytes) – Data chunk to write |
|---|
Write a data chunk into the ring buffer. It will be consumed later by the speaker.
Play the sample using the internal ring buffer
Sample generators¶
Sample generator using thread, does nothing by default. It can be used to implement your own generator.
__init__ ( stream : AudioOutput ) ¶
| Parameters: | stream ( AudioOutput ) – The AudioOutput instance to use |
|---|
get_bytes ( ) → bytes¶
Must return a bytes string with the data to store in the ring buffer.
class audiostream.sources.wave. SineSource ( ThreadSource ) ¶
Sample generator that use the ThreadSource , and generate bytes from a sin() generator.
- stream ( AudioOutput ) – The AudioOutput instance to use
- frequency (integer) – The sin() frequency, for example: 440.
Load a PureData patch, and read the generated output.
Indices and tables¶
© Copyright 2012, Mathieu Virbel, Dustin Lacewell. Revision 9e9f4162927b9340b24226e7ef0b5f3c1af7596b .
Versions latest Downloads pdf htmlzip epub On Read the Docs Project Home Builds Free document hosting provided by Read the Docs.