- Python: execute shell commands (and get the output) with the os package
- Execute shell commands using the os package
- Execute shell commands and get the output using the os package
- How To Run Shell Command And Get Output In Python
- 1. Run Shell Command Use subprocess Module.
- 2. Run Shell Command Use os Module system And popen Function.
- 2.1 os.system(command).
- 2.2 os.popen(command, [mode, [bufsize]]).
- 3. Other Python os, shutil Module Functions Example.
- 4. Python System Environment Variable.
- 5. The os.path Module.
- 6. Run Shell Command In Python Examples.
- 6.1 The following is a simple script case for copying files.
- 6.2 The following example renames all files in a folder to 10001 ~ 10999.
- 7. File Redirection.
- Python: Get system command output
- Python: Tips of the Day
Python: execute shell commands (and get the output) with the os package
This article is part of a two-part series related to running shell commands from within Python.
Execute shell commands using the os package
The most straightforward solution to running shell commands via Python is simply by using the system method of the os package. The os package “provides a portable way of using operating system dependent functionality.” The system method executes a string as a command in a subshell, which basically is an independent instance of the command processor of your operating system. For Windows, commands will be run using cmd.exe, for Linux this will be Bash, Mac will use Bash or Z shell.
The following code will open a subshell, run the command, and return the process exit code — a 0 (zero) if the command has run successfully, other numbers mean something has gone wrong (see Linux and Windows).
import os os.system('echo "Hello World!"')
Execute shell commands and get the output using the os package
Sometimes, you’re not only interested in running shell commands, but you’d also like to know what printed output is. The os package also has a solution for this: the popen method. This method opens a pipe into the command, which is used to transfer its output to a Python variable.
The following code will once again open a subshell and run the command, but instead of returning the process exit code, it will return the command output. If you ‘d like to remove the trailing nextline (\n), you can use the strip method
output_stream = os.popen('echo "Hello World!"') output_stream.read()
- If the command passed to the shell generates errors, it will only return two single quotes.
- If you would like to know if a command completed successfully, you can access the process exit code using the close method. Like this:
output_stream = os.popen('non-existing-command') output_stream.close()
The popen method of the os package uses the subprocess module. So instead of using popen, you might as well interface with the subprocess module directly. We discuss this in the next article.
How To Run Shell Command And Get Output In Python
In the early python version, the os module’s system or popen function is usually used to execute operating system commands. However, recently python has gradually abandoned these functions officially. It recommends using the built-in subprocess module to execute operating system-related commands. This article will tell you how to use the subprocess module and os module’s system, popen function to run a shell command.
1. Run Shell Command Use subprocess Module.
# the child process will print it's standard output to a pipe line, the pipe line connect the child process and it's parent process. >>> child = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>>> import subprocess >>> child = subprocess.Popen(['java','-version'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) >>> print(child.stdout.read()) b'openjdk version "1.8.0_212"\nOpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03)\nOpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode)\n'
2. Run Shell Command Use os Module system And popen Function.
2.1 os.system(command).
- Run operating system commands and display the results directly on the standard output device( ie: screen console).
- But the function’s return value is 0 or -1, and the data displayed on the screen cannot be obtained in the source code.
- The command parameter is the command string to be executed.
>>> import os >>> ret = os.system('java -version') openjdk version "1.8.0_212" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode) >>> ret 0
2.2 os.popen(command, [mode, [bufsize]]).
- Start a child process to execute the command specified by the command parameter, and establish a pipeline between the parent process and the child process for communication between the parent and child processes.
- This method returns a file object, which can be read or written, depending on the parameter mode‘s value.
- If the mode parameter’s value is ‘r’, then the file is read-only. If the mode parameter’s value is ‘w’, then the file is write-only and it will throw an error when you want to get the file object’s content using it’s read() method.
- In short, popen method can run operating system commands and can return the result of the command through the result file object’s read() method.
- Below is an example.
>>> import os # invoke os.popen method to run a shell command. >>> ret = os.popen('ifconfig') # get the shell command execution result through the returned file object's read() method. >>> ret.read() 'lo0: flags=8049 mtu 16384\n\toptions=1203\n\tinet 127.0.0.1 netmask 0xff000000 \n\tinet6 ::1 prefixlen 128 \n\tinet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 \n\tnd6 options=201\ngif0: flags=8010 mtu 1280\nstf0: flags=0<> mtu 1280\nen0: flags=8863 mtu 1500\n\toptions=50b\n\tether 10:dd:b1:9b:b0:b9 \n\tnd6 options=201\n\tmedia: autoselect (none)\n\tstatus: inactive\nen1: flags=8863 mtu 1500\n\toptions=400\n\tether 4c:8d:79:e1:bc:e2 \n\tinet6 fe80::82a:736f:f606:31f0%en1 prefixlen 64 secured scopeid 0x5 \n\tinet 192.168.31.31 netmask 0xffffff00 broadcast 192.168.31.255\n\tnd6 options=201\n\tmedia: autoselect\n\tstatus: active\nen3: flags=8963 mtu 1500\n\toptions=460\n\tether 82:0a:57:17:dd:80 \n\tmedia: autoselect \n\tstatus: inactive\nfw0: flags=8863 mtu 4078\n\tlladdr 10:dd:b1:ff:fe:5c:5f:76 \n\tnd6 options=201\n\tmedia: autoselect \n\tstatus: inactive\nbridge0: flags=8863 mtu 1500\n\toptions=63\n\tether 82:0a:57:17:dd:80 \n\tConfiguration:\n\t\tid 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0\n\t\tmaxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200\n\t\troot id 0:0:0:0:0:0 priority 0 ifcost 0 port 0\n\t\tipfilter disabled flags 0x0\n\tmember: en3 flags=3\n\t ifmaxaddr 0 port 6 priority 0 path cost 0\n\tnd6 options=201\n\tmedia: \n\tstatus: inactive\np2p0: flags=8843 mtu 2304\n\toptions=400\n\tether 0e:8d:79:e1:bc:e2 \n\tmedia: autoselect\n\tstatus: inactive\nawdl0: flags=8943 mtu 1484\n\toptions=400\n\tether 02:c2:69:6d:b3:5e \n\tinet6 fe80::c2:69ff:fe6d:b35e%awdl0 prefixlen 64 scopeid 0xa \n\tnd6 options=201\n\tmedia: autoselect\n\tstatus: active\nutun0: flags=8051 mtu 1380\n\tinet6 fe80::a4be:4aed:be5b:253%utun0 prefixlen 64 scopeid 0xb \n\tnd6 options=201\nutun1: flags=8051 mtu 2000\n\tinet6 fe80::cfcf:1568:4d94:3b0a%utun1 prefixlen 64 scopeid 0xc \n\tnd6 options=201\n'
3. Other Python os, shutil Module Functions Example.
- os.listdir(dirname): This function returns a list of strings containing all file names under the directory, but excluding “.” and “..”.
>>> import os >>> >>> os.listdir('.\\') ['.anaconda', '.cache', '.conda', '.condarc', '.config', '.continuum', '.eclipse', '.FAILED_FILE_CACHED_DIR', '.gitconfig', '.lemminx', '.m2', '.matplotlib', '.p2', '.VirtualBox']
>>> import os >>> >>> os.getcwd() 'C:\\Users\\zhaosong'
>>>import os >>> >>>os.getcwd() 'D:\\' >>> >>>os.chdir('c:\\') >>> >>>os.getcwd() “c:\\”
os.S_ISUID, os.S_ISGID, os.S_ENFMT, os.S_ISVTX, os.S_IREAD, os.S_IWRITE, os.S_IEXEC, os.S_IRWXU, os.S_IRUSR, os.S_IWUSR, os.S_IXUSR , os.S_IRWXG, os.S_IRGRP, os.S_IWGRP, os.S_IXGRP, os.S_IRWXO, os.S_IROTH, os.S_IWOTH, os.S_IXOTH R stands for read, W for write, and X for execute permission. USR represents the user, GRP represents the group, OTH represents others.
4. Python System Environment Variable.
- The python environment variable is stored in the os.environ dictionary. It can be modified by using the ordinary dictionary method. It will be inherited automatically when starting other programs with the system.
- The value of a python environment variable can only be a string. Unlike the shell, python does not have the concept of exporting environment variables.
- The below example will add a python environment variable.
5. The os.path Module.
- This module contains many functions for pathname processing. In the shell, pathname processing seems not very important, but it is often used in Python. The two most commonly used are separating and merging directory names and file names.
- os.path.split(path) -> (dirname, basename): This function will separate a path into two parts.
>>> import os >>> >>> os.path.split("/foo/bar.dat") ('/foo', 'bar.dat')
>>> import os >>> >>> os.path.join("/foo","bar.dat") '/foo\\bar.dat'
>>> import os >>> >>> os.path.splitext("/foo /bar.tar.bz2") ('/foo /bar.tar', '.bz2')
6. Run Shell Command In Python Examples.
6.1 The following is a simple script case for copying files.
- There is a directory of tens of thousands of files.
- I want to copy these files to other directories, but I do not want to copy the directory itself directly.
- I tried “cp src/ * dest/” and it reported an error that the command line was too long.
- So I write the below python source code to implement the task.
import sys,os.path,shutil # sys.argv[0] is the python program name. # sys.argv[1] is the source directory. # sys.argv[2] is the target directory. # loop in the source directory. for f in os.listdir(sys.argv[1]): # get the source file full path value. src_file_path = os.path.join(sys.argv[1],f) # copy the source file to the target directory shutil.copy(src_file_path,sys.argv[2])
6.2 The following example renames all files in a folder to 10001 ~ 10999.
import os.path,sys # get the files source folder. dirname=sys.argv[1] i = 10001 # loop all the files in the directory. for f in os.listdir(dirname): # get the source file. src=os.path.join(dirname,f) # check whether the file is a directory or not. if os.path.isdir(src): # if the file is a directory then continue. continue # otherwise rename the file name. os.rename(src,str(i)) i += 1
7. File Redirection.
- There is an existing py file new1.py.
- When you enter the command new1 > new.txt on the command line to output the running result of new1 to the file new.txt, this is called stream redirection
Python: Get system command output
Follow us on Facebook and Twitter for latest update.
Python: Tips of the Day
How to upgrade all Python packages with pip?
There isn’t a built-in flag yet, but you can use.
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
Note: there are infinite potential variations for this. I’m trying to keep this answer short and simple, but please do suggest variations in the comments!
In older version of pip, you can use this instead:
pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
The grep is to skip editable («-e») package definitions, as suggested by @jawache. (Yes, you could replace grep+cut with sed or awk or perl or. ).
The -n1 flag for xargs prevents stopping everything if updating one package fails.
- Weekly Trends
- Java Basic Programming Exercises
- SQL Subqueries
- Adventureworks Database Exercises
- C# Sharp Basic Exercises
- SQL COUNT() with distinct
- JavaScript String Exercises
- JavaScript HTML Form Validation
- Java Collection Exercises
- SQL COUNT() function
- SQL Inner Join
- JavaScript functions Exercises
- Python Tutorial
- Python Array Exercises
- SQL Cross Join
- C# Sharp Array Exercises
We are closing our Disqus commenting system for some maintenanace issues. You may write to us at reach[at]yahoo[dot]com or visit us at Facebook