How do I pipe a subprocess call to a text file?
RIght now I have a script that I run. When I run it and it hits this line, it starts printing stuff because run.sh has prints in it. How do I pipe this to a text file also? (And also print, if possible)
2 Answers 2
If you want to write the output to a file you can use the stdout-argument of subprocess.call .
- None (the default, stdout is inherited from the parent (your script))
- subprocess.PIPE (allows you to pipe from one command/process to another)
- a file object or a file descriptor (what you want, to have the output written to a file)
You need to open a file with something like open and pass the object or file descriptor integer to call :
f = open("blah.txt", "w") subprocess.call(["/home/myuser/run.sh", "/tmp/ad_xml", "/tmp/video_xml"], stdout=f) I’m guessing any valid file-like object would work, like a socket (gasp :)), but I’ve never tried.
As marcog mentions in the comments you might want to redirect stderr as well, you can redirect this to the same location as stdout with stderr=subprocess.STDOUT . Any of the above mentioned values works as well, you can redirect to different places.
@Goldname you need a shell to do redirection, so: subprocess.call([«echo», «1», «>>», «t.txt»], shell=True) should work.
@WesMason Thanks, but i thought subprocess.call worked in a way that was basically like copying and pasting into the shell?
@Goldname nah, everything in subprocess relies on the underlying Popen class, which opens an process, shell=True will force the process to be executed in the system default shell (e.g. /bin/sh on POSIX systems, which is usually bash or dash), it’s more secure that way as you’re not open to all of the extra functionality provided by the shell (a whole programming language unto itself, with it’s own exploits if passing in user provided data). Also take a look at shlex.quote if you are passing any possibly dirty data to a subshell.
Should we also close the file after we are done? this will close the file after we are done: with open(«blah.txt»,»w») as f: subprocess.call([«/home/myuser/run.sh», «/tmp/ad_xml», «/tmp/video_xml»], stdout=f) @Skurmedel
Python write pipe to file
In the previous section it was shown how pipes were represented in the operating system in Unix, and how they were written and read from in a way that was very much analagous to how ordinary files are. This is one of the major insights of the Unix family of operating systems, they make explicit the analogies between interprocess communication and ordinary input and output – if a program is made to operate based on the input of plain text and produce plain text as an output it does not matter about the origins of that input, be it a human with a keyboard, another program or even input travelling from across the world over the internet.
It should come as no surprise that Python continues in this happy tradition, treating named pipes in much the same way as files.
Writing to Pipes in Python
Here is a Python program for writing a short string of text to a file:
with open("my_file", "w") as f: print "have opened file, commencing writing. " f.write("hello to a file\n")
and here is how we would read the contents of that file from a virtual console:
The equivalent program for writing to our pipe my_pipe is fundamentally exactly the same,
with open("my_pipe", "w") as f: print "have opened pipe, commencing writing. " f.write("hello through a pipe\n")
will read the contents of the pipe in a way exactly analagous to a file.
One thing worth noting is that the message, “have opened pipe, commencing writing” does not get displayed until we open the other end of the pipe for reading – the call to open in Python is blocking until the pipe has got an end to be written to.
Reading from Pipes in Python
Reading from pipes is also exactly analagous to reading from a pipe:
# read_from_pipe.py with open("my_pipe", "r", 0) as f: print "have opened pipe, commencing reading. " for l in f: print l,
by running write_to_pipe.py in one virtual console and read_from_pipe.py in another we recover the same behaviour as we observed in using echo and cat before. Indeed we can run the read and write and the opposite order, and see that this time it’s the call to open in the read program that blocks, until the write program is ran and opens the other end of the pipe.
In the next section we look a bit more at the subtleties around these operations, specifically the subtleties around buffering.