Shell script and python

How to Use a Bash Script to Run Your Python Scripts

Python is a popular programming language used in a wide range of applications, from data analysis and scientific computing to web development and artificial intelligence. Bash is a powerful scripting language used primarily in Unix and Linux environments for automating tasks and running commands. By combining the two languages, developers can create powerful scripts that can automate complex tasks and workflows. In this article, we’ll explore various techniques for running Python scripts with bash, including passing arguments, activating virtual environments, and running scripts in the background. We’ll also provide examples of real-world scripts that demonstrate these techniques in action.

In this tutorial you will learn:

• How to run Python scripts from a bash script
• Passing arguments to Python scripts from a bash script
• Activating virtual environments for Python scripts in a bash script
• Running Python scripts in the background using a bash script

Software Requirements and Linux Command Line Conventions

Category	Requirements, Conventions or Software Version Used
System	Any Linux distro
Software	N/A
Other	Privileged access to your Linux system as root or via the sudo command.
Conventions	# – requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command $ – requires given linux commands to be executed as a regular non-privileged user

How to run Python scripts from a bash script

To run a Python script from a bash script, we should first change to the directory containing the Python script using the cd command, and then use the python command to execute the script. Here’s an updated example:

#!/bin/bash # Change to the directory containing the Python script cd /path/to/python/script/directory # Run the Python script python myscript.py

In this updated script, we first change to the directory containing the Python script using the cd command, and then run the python command to execute the myscript.py Python script. Note that the path to the script file should be specified relative to the current directory or as an absolute path.

Passing arguments to Python scripts from a bash script

We can pass arguments to a Python script from a bash script by specifying them after the Python script file name when using the python command. The arguments will be accessible within the Python script through the sys.argv list. Here’s an example:

#!/bin/bash cd /path/to/python/script/directory python myscript.py $1 $2 $3

In this script, we run the myscript.py Python script and pass three arguments ( $1 , $2 , and $3 ) to it.

Within the Python script, we can access the arguments using the sys.argv list:

import sys print("Argument 1:", sys.argv[1]) print("Argument 2:", sys.argv[2]) print("Argument 3:", sys.argv[3]) 

In this example Python script, we access the three arguments passed from the bash script using the sys.argv list and print them to the console. By passing arguments from a bash script to a Python script, we can customize the behavior of the script and make it more flexible and reusable.

Activating virtual environments for Python scripts in a bash script

When working with Python, it’s common to use virtual environments to isolate project dependencies and ensure consistent runtime environments. If you’re using a virtual environment for your Python script, you’ll need to activate it before running the script. Here’s an example of how to activate a virtual environment for a Python script within a bash script:

#!/bin/bash # Activate the virtual environment source /path/to/venv/bin/activate # Change to the directory containing the Python script cd /path/to/python/script/directory # Run the Python script python myscript.py

In this script, we use the source command to activate the virtual environment located at /path/to/venv . We then change to the directory containing the Python script using the cd command and run the script using the python command.

Note that the path to the virtual environment may vary depending on your setup, so be sure to modify the path to match your own environment. Additionally, if your virtual environment requires specific dependencies that aren’t installed on the system by default, you may need to install them first within the virtual environment before running the script.

Running Python scripts in the background using a bash script

Running a Python script in the background can be useful when you want to run the script without keeping the terminal window open. In a bash script, you can use the nohup command to run a Python script in the background, and redirect its output to /dev/null to prevent it from filling up the disk space. Here’s an example:

#!/bin/bash # Change to the directory containing the Python script cd /path/to/python/script/directory # Run the Python script in the background nohup python myscript.py > /dev/null 2>&1 & 

In this script, we first change to the directory containing the Python script using the cd command. We then use the nohup command to run the Python script in the background, followed by the python command and the name of the script file.

We redirect the output of the script to /dev/null , which is a special file that discards any data written to it. This prevents the script’s output from filling up the disk space or interfering with other processes. Finally, we append the & symbol to run the script in the background.

When you run this script, the Python script will start running in the background and you’ll see a message similar to this:

nohup: ignoring input and appending output to ‘nohup.out’

You can now close the terminal window and the Python script will continue running in the background until it finishes or is terminated manually.

Running Python scripts in the background can be useful for long-running scripts, such as web servers or data processing tasks, that don’t require user input or interaction. By using a bash script to run the Python script in the background, you can automate the process and ensure that it continues running even after you log out of the terminal.

Conclusion

In this article, we explored various techniques for running Python scripts with bash, including passing arguments, activating virtual environments, and running scripts in the background. These techniques can greatly enhance the power and flexibility of Python scripts, enabling developers to automate complex tasks and workflows with ease. By mastering the art of running Python scripts with bash, developers can unlock the full potential of these powerful scripting languages and take their skills to the next level. With the foundational knowledge and skills provided in this article, developers can create robust and efficient scripts that can handle a wide range of use cases and unlock new possibilities for automation and efficiency.

Related Linux Tutorials:

Comments and Discussions

Источник

Run Python Code in a Shell Script

I’m a big fan of “the right tool for the right job”. For some things, the right tool is a shell script. For others, it is Python. But sometimes… it is both.

A lot of times I find myself writing a shell script and wanting a little extra complexity than I care for in a shell script. Sometimes, it is easier to write a few lines in Python than trying to figure it out in a shell script. But… sometimes it is also convenient not to maintain (or transfer) an entire Python file (.py) to contain this logic.

Ad hoc Python in a shell script

So what can you do? Put your Python directly in your shell scripts! How can you do this? Construct your multiline Python string and then pass the ad hoc Python code in with the -c parameter. An example:

days_remaining.sh

1 2 3 4 5 6 7 8 9 10 11 12 13

#!/bin/bash PYCMD=$(cat EOF from datetime import datetime first_day_of_new_year = datetime(2022, 1, 1) days_remaining = (first_day_of_new_year - datetime.now()).days print('<> days remaining in this year'.format(days_remaining)) EOF ) python3 -c "$PYCMD" 

And now for the sake of completeness:

$ ./days_remaining.sh 335 days remaining in this year

Indentation

Let’s say we had our Python code indented in the shell script. Maybe it’s in a loop, or a function like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

#!/bin/bash get_remaining_days ()  PYCMD=$(cat <EOF from datetime import datetime first_day_of_new_year = datetime(2022, 1, 1) days_remaining = (first_day_of_new_year - datetime.now()).days print('<> days remaining in this year'.format(days_remaining)) EOF ) python3 -c "$PYCMD" > get_remaining_days

This might look ok, but it isn’t correct syntax:

$ ./days_remaining.sh ./days_remaining.sh: line 4: unexpected EOF while looking for matching `)' ./days_remaining.sh: line 18: syntax error: unexpected end of file

For both the here-document and the Python code, we need to keep the code unindented. Here is the correct form:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

#!/bin/bash get_remaining_days ()  PYCMD=$(cat <EOF from datetime import datetime first_day_of_new_year = datetime(2022, 1, 1) days_remaining = (first_day_of_new_year - datetime.now()).days print('<> days remaining in this year'.format(days_remaining)) EOF ) python3 -c "$PYCMD" > get_remaining_days

Debugging Python inside a shell script

Code doesn’t work the way we think it should all the time. Debugging is a necessary part of any software development, and you might need to debug that Python code that is directly in your shell scripts too.

As a Python programmer, we’re quite familiar with setting a breakpoint in code with import pdb; pdb.set_trace() . So let’s put that breakpoint in our shell script and see what happens:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

#!/bin/bash PYCMD=$(cat EOF from datetime import datetime first_day_of_new_year = datetime(2022, 1, 1) import pdb; pdb.set_trace() days_remaining = (first_day_of_new_year - datetime.now()).days print('<> days remaining in this year'.format(days_remaining)) EOF ) python3 -c "$PYCMD" 

$ ./days_remaining.sh > (7)() (Pdb) l . [EOF] (Pdb) first_day_of_new_year datetime.datetime(2022, 1, 1, 0, 0) (Pdb) datetime.now() datetime.datetime(2021, 1, 30, 14, 41, 13, 475833) (Pdb)

Hmmm we have a partial debugging experience (because we passed in ad hoc code to Python with -c ). We can analyze the code, but it’s hard to know where we are, especially when we are stepping through the code, because it can’t show any lines of code. Typically when you do l . in the debugger, it’ll list the current line of code that the debugger is on and some surrounding lines of code. Very useful and almost necessary for proper debugging.

How can we get around this? Create a temporary file and dump the Python code directly to it and then pass the file directly to Python for a full debugging experience:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

#!/bin/bash PYCMD=$(cat EOF from datetime import datetime first_day_of_new_year = datetime(2022, 1, 1) import pdb; pdb.set_trace() days_remaining = (first_day_of_new_year - datetime.now()).days print('<> days remaining in this year'.format(days_remaining)) EOF ) TEMP_SCRIPT=$(mktemp) echo "$PYCMD" > "$TEMP_SCRIPT" python3 "$TEMP_SCRIPT" # python3 -c "$PYCMD" 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

$ ./days_remaining.sh > /tmp/tmp.ZDdkh5A4me(7)() -> days_remaining = (first_day_of_new_year - datetime.now()).days (Pdb) l . 2 3 first_day_of_new_year = datetime(2022, 1, 1) 4 5 import pdb; pdb.set_trace() 6 7 -> days_remaining = (first_day_of_new_year - datetime.now()).days 8 print('<> days remaining in this year'.format(days_remaining)) [EOF] (Pdb) first_day_of_new_year datetime.datetime(2022, 1, 1, 0, 0) (Pdb) n > /tmp/tmp.ZDdkh5A4me(8)() -> print('<> days remaining in this year'.format(days_remaining)) (Pdb) days_remaining 335 (Pdb)

Great! Now we can debug the Python code with all of the features and comfort of pdb.

Warnings and final thoughts

If you do find your embedded ad hoc Python code getting long, it’s probably a good idea to move that to a separate file so that it can be properly tested and maintained. Or perhaps even just write the whole utility in Python and forget about the shell script approach!

Hopefully this blog post has illustrated the flexibility of embedding Python code directly in your shell scripts to make the development experience a bit better than having to solve all problems just in shell scripting!

Источник