How to Run bash scripts Using Python?
Invicti Web Application Security Scanner – the only solution that delivers automatic verification of vulnerabilities with Proof-Based Scanning™.
If you are using Linux, then you would definitely love the shell commands.
And if you are working with Python, then you may have tried to automate things. That’s a way to save time. You may also have some bash scripts to automate things.
Python is handy to write scripts than bash. And managing Python scripts are easy compared to bash scripts. You will find it difficult to maintain the bash scripts once it’s growing.
But what if you already have bash scripts that you want to run using Python?
Is there any way to execute the bash commands and scripts in Python?
Yeah, Python has a built-in module called subprocess which is used to execute the commands and scripts inside Python scripts. Let’s see how to execute bash commands and scripts in Python scripts in detail.
Executing Bash Commands
As you may have already seen the module subprocess is used to execute the bash commands and scripts. It provides different methods and classes for the same.
There are mainly one method and one class to know about from the subprocess module. They are run and Popen. These two help us to execute the bash commands in Python scripts. Let’s see them one by one.
subprocess.run()
The method subprocess.run() will take a list of strings as a positional argument. This is mandatory as it has the bash command and arguments for it. The first item in the list is the command name and the remaining items are the arguments to the command.
import subprocess subprocess.run(["ls"]) The above script list all the items in the current working directory as the script lies. There are no arguments to the command in the above script. We have given only the bash command. We can provide additional arguments to the ls command like -l , -a , -la , etc.
Let’s see a quick example with command arguments.
import subprocess subprocess.run(["ls", "-la"]) The above command displays all the files including hidden files along with the permissions. We have provided the argument la which displays files and directories extra information and hidden files.
We may end up making some mistakes while writing the commands. Errors will raise according to the mistakes. What if you want to capture them and use them later? Yeah, we can do that using the keyword argument stderr.
import subprocess result = subprocess.run(["cat", "sample.txt"], stderr=subprocess.PIPE, text=True) print(result.stderr) Make sure you don’t have the file with the name sample.txt in the working directory. The value to the keyword argument stderr is PIPE which helps to return the error in an object. We can access it later with the same name. And the keyword argument text helps to tell that the output should be a string.
Similarly, we can capture the output of the command using the stdout keyword argument.
import subprocess result = subprocess.run(["echo", "Hello, World!"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) print(result.stdout) subprocess.run() – input
You can give input to the commands using the input keyword argument. We will give inputs in a string format. So, we need to set the keyword argument text to True . By default, it takes it in bytes.
import subprocess subprocess.run(["python3", "add.py"], text=True, input="2 3") In the above program, the Python script add.py will take two numbers as input. We have given the input to the Python script using the input keyword argument.
subprocess.Popen()
The class subprocess.Popen() is advanced than the method subprocess.run(). It gives us more options to execute the commands. We will create an instance of the subprocess.Popen() and use it for various things like knowing the status of the command execution, getting output, giving input, etc.
There are several methods of the class subprocess.Popen() that we need to know. Let’s see them one by one along with the code examples.
wait
It is used to wait until the completion of the execution of the command. The next lines of the Python script won’t execute until the completion of the previous command that is written after the wait method. Let’s see the example.
import subprocess process = subprocess.Popen(["ls", "-la"]) print("Completed!") Run the above code and observe the output. You will see that the message Completed! is printed before the execution of the command. We can avoid it using the wait method. Let’s wait till the completion of the command.
import subprocess process = subprocess.Popen(["ls", "-la"]) process.wait() print("Completed!") If you see the output for the above code, then you will realize that wait is actually working. The print statement is executed after the completion of the command execution.
communicate
The method communicate is used to get the output, error and give input to the command. It returns a tuple containing output and error respectively. Let’s see an example.
import subprocess process = subprocess.Popen(["echo", "Hello, World!"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) result = process.communicate() print(result)subprocess.Popen() – input
We can’t pass the input to the class Popen directly. We need to use the keyword argument called stdin to give the input to the command. The instance of the class Popen will provide us stdin object. It has a method called write which is used to give the input to the command.
As we discussed earlier, it will take inputs as bytes-like objects by default. So, don’t forget to set the keyword argument text to True while creating the instance of Popen .
import subprocess process = subprocess.Popen(["python3", "add.py"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) process.stdin.write("2 3") process.stdin.close() print(process.stdout.read())poll
The method poll is used to check whether the execution of the command is completed or not. This method will return None if the command is still executing. Let’s see an example.
import subprocess process = subprocess.Popen(['ping', '-c 5', 'geekflare.com'], stdout=subprocess.PIPE, text=True) while True: output = process.stdout.readline() if output: print(output.strip()) result = process.poll() if result is not None: break In the above code, we have used the ping command with 5 requests. There is an infinite loop that iterates until the completion of the command execution. We have used the method poll to check the status of the command execution. If the method poll returns code other than None , then the execution completes. And the infinite loop breaks.
Executing Bash Scripts
We have seen two ways to execute the commands. Now, let’s see how to execute the bash scripts in Python scripts.
The subprocess has a method called call. This method is used to execute the bash scripts. The method returns the exit code from the bash script. The default exit code for the bash scripts is 0. Let’s see an example.
Create a bash script with the name practice.sh as follows.
#!/bin/bash echo "Hello, World!" exit 1 Now, write a Python script execute the above bash script.
import subprocess exit_code = subprocess.call('./practice.sh') print(exit_code) You will get the following output once you run the above Python script.
Conclusion
We have seen how to execute bash commands and scripts in Python. You can use them to automate things more efficiently.
How to call a shell script from python code?
Where test.sh is a simple shell script and 0 is its return value for this run.
Note: it’s preferable to pass subprocess.call() a list rather than a string (see command to Hugo24 below for the example and reasons).
This gives: OSError: [Errno 13] Permission denied. my script does not required to run with sudo. @Manoj Govindan
@alper go the folder where you have placed the script and run the command, chmod +x script.sh . Note: script.sh is a placeholder for your script, replace it accordingly.
There are some ways using os.popen() (deprecated) or the whole subprocess module, but this approach
import os os.system(command) why isn’t this the most upvoted answer? Isn’t not having to import a module the better solution? Must be some drawback here?
With subprocess you can manage input/output/error pipes. It is also better when you have many arguments — with os.command() you will have to create whole command line with escaping special characters, with subprocess there is simple list of arguments. But for simple tasks os.command() may be just sufficient.
To quote from that link: The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; *using that module is preferable to using this function.*
In case you want to pass some parameters to your shell script, you can use the method shlex.split():
import subprocess import shlex subprocess.call(shlex.split('./test.sh param1 param2')) with test.sh in the same folder:
#!/bin/sh echo $1 echo $2 exit 0 $ python test.py param1 param2 Assuming test.sh is the shell script that you would want to execute
Use the subprocess module as mentioned above.
Note: calling subprocess with a list is safer since it doesn’t necessitate passing the (potentially unsanitized) string through a shell for parsing/interpretation. The first item in the list will be the executable and all other items will be passed as arguments.
I’m running python 3.5 and subprocess.call([‘./test.sh’]) doesn’t work for me.
I give you three solutions depends on what you wanna do with the output.
1 — call script. You will see output in your terminal. output is a number.
import subprocess output = subprocess.call(['test.sh']) 2 — call and dump execution and error into string. You don’t see execution in your terminal unless you print(stdout). Shell=True as argument in Popen doesn’t work for me.
import subprocess from subprocess import Popen, PIPE session = subprocess.Popen(['test.sh'], stdout=PIPE, stderr=PIPE) stdout, stderr = session.communicate() if stderr: raise Exception("Error "+str(stderr)) 3 — call script and dump the echo commands of temp.txt in temp_file
import subprocess temp_file = open("temp.txt",'w') subprocess.call([executable], stdout=temp_file) with open("temp.txt",'r') as file: output = file.read() print(output) Don’t forget to take a look at the doc subprocess