Python subprocess execute in directory

[python] Subprocess changing directory

I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can’t get subprocess to enter my subdirectory:

[email protected]:~/Projekty/tests/ve$ python Python 2.7.4 (default, Sep 26 2013, 03:20:26) [GCC 4.7.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> import os >>> os.getcwd() '/home/tducin/Projekty/tests/ve' >>> subprocess.call(['cd ..']) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/subprocess.py", line 524, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory 

Python throws OSError and I don’t know why. It doesn’t matter whether I try to go into an existing subdir or go one directory up (as above) — I always end up with the same error.

This question is related to python subprocess

The answer is

What your code tries to do is call a program named cd .. . What you want is call a command named cd .

But cd is a shell internal. So you can only call it as

subprocess.call('cd ..', shell=True) # pointless code! See text below. 

But it is pointless to do so. As no process can change another process’s working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.

What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.

For example, to execute ls in the root directory, you either can do

wd = os.getcwd() os.chdir("/") subprocess.Popen("ls") os.chdir(wd) 

To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in @wim’s answer:

import subprocess subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir) 

A child process can’t change its parent’s working directory (normally). Running cd .. in a child shell process using subprocess won’t change your parent Python script’s working directory i.e., the code example in @glglgl’s answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.

You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.

If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.

subprocess.call and other methods in the subprocess module have a cwd parameter.

This parameter determines the working directory where you want to execute your process.

So you can do something like this:

subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/') 

I guess these days you would do:

import subprocess subprocess.run(["pwd"], cwd="sub-dir") 

This allows you to execute multiple commands (e.g cd ) in the same process.

import subprocess commands = ''' pwd cd some-directory pwd cd another-directory pwd ''' process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE) out, err = process.communicate(commands.encode('utf-8')) print(out.decode('utf-8')) 

If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow ‘cd’ commands to work.

import subprocess import os def cd(cmd): #cmd is expected to be something like "cd [place]" cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command out = p.stdout.read() err = p.stderr.read() # read our output if out != "": print(out) os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline if err != "": print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that return 

just use os.chdir
Example:

>>> import os >>> import subprocess >>> # Lets Just Say WE want To List The User Folders >>> os.chdir("/home/") >>> subprocess.run("ls") user1 user2 user3 user4 

If you need to change directory, run a command and get the std output as well:

import os import logging as log from subprocess import check_output, CalledProcessError, STDOUT log.basicConfig(level=log.DEBUG) def cmd_std_output(cd_dir_path, cmd): cmd_to_list = cmd.split(" ") try: if cd_dir_path: os.chdir(os.path.abspath(cd_dir_path)) output = check_output(cmd_to_list, stderr=STDOUT).decode() return output except CalledProcessError as e: log.error('e: <>'.format(e)) 

def get_last_commit_cc_cluster(): cd_dir_path = "/repos/cc_manager/cc_cluster" cmd = "git log --name-status HEAD^..HEAD --date=iso" result = cmd_std_output(cd_dir_path, cmd) return result log.debug("Output: <>".format(get_last_commit_cc_cluster())) 

Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1[email protected]>\nDate: 2020-04-23 09:58:49 +0200\n\n 

Источник

Use subprocess To Execute Commands In Python 3

While working in Python you may come across cases where you want a quick way to run programs you have already written, without tampering with the source code – cases like this are where the subprocess module can come into play.

The subprocess module, present in a standard installation of Python is used to run new applications or programs through Python code by creating new processes.

It gives us the ability to:

1. spawn new processes
2. connect to their input, output and error pipes
3. obtain their return codes

NOTE: The subprocess module works better on a Linux system as quite a few of the operations are Linux commands.

How to use subprocess to list files in a directory

A common use for subprocess would be to list files and folders in a directory. This makes use of the run method of subprocess .

The code will differ depending on what system you are on.

import subprocess subprocess.run('ls')

For Windows

import subprocess subprocess.run("dir", shell=True)

• import subprocess
• call the run module from it
• pass the list directory command based on your system (ls/dir)
  • if you are on Windows you will have to additionally pass shell=True because dir is a shell command and you need to tell the system that you want to use it.

We automatically got output in the terminal even though we did not print it ourselves. That is how the run command works – it does not capture our command output by default.

Let’s see what happens if we try to capture our command by placing it in a variable

import subprocess result = subprocess.run("dir", shell=True) print(result)

The output is not what you would expect. We get a message containing the arguments that were passed and the return code.

If we wanted to capture the output we would modify out code to this:

import subprocess result = subprocess.run(["dir"], shell=True, capture_output=True, text=True) print(result.stdout)

• capture_output=True: to capture the output
• text=True: to decode the output to a readable format since it is captured as bytes

We then print result.stdout : the result as standard output

The same can be achieved with the follow code:

import subprocess result = subprocess.run(["ls", "-la"], stdout=subprocess.PIPE, text=True) print(result.stdout)

The difference is stdout=subprocess.PIPE This code performs the same function as capture_output=True

In addition, we use the ls command with the argument of -la both passed as a list

How to redirect subprocess output to a file

Having the output of our command in the console is not that useful. It would serve use better if we could store it in a file. This is possible by modifying our code to look like this:

import subprocess with open("output.txt", "w") as file: result = subprocess.run(["ls", "-la"], stdout=file, text=True)

Instead of having the output display in the terminal, we set the stdout to our file.

When we run this code: python sub.py it does not look like anything happened. However, if we check our folder a text file, with the output we had prior has been created

How to read the contents of a file with subprocess

Now let’s see how we can use subprocess to read the contents of a file. Below is a text file with a line of text

This is how we would read the contents of it:

import subprocess result = subprocess.run(["cat", "file.txt"], capture_output=True, text=True) print(result.stdout)

In the above code, we pass in our list; the cat command and the file name. The rest is the same as before

NOTE: The cat command allows us to create files, view the contents of a file, concatenate them, and redirect their output.

Additionally, we can use this output as input to access specific parts of the file. We do this using the file we wrote to previosly

import subprocess result = subprocess.run(["cat", "output.txt"], capture_output=True, text=True) # print(result.stdout) input = subprocess.run( ["grep", "-n", "sub.py"], capture_output=True, text=True, input=result.stdout ) print(input.stdout)

After running the cat command on output.txt, we do not print that output. Instead, we use it as the input of the next command – input

In the second list of our input line of code:

• we use grep to search for a string of text in the file
• pass the -n flag to specify we are looking for the entire line of text
• add “sub.py” as what we are searching for

Lastly, we print the output of this second command

We have successfully searched for a line of text in our file and output it.

How to call an external command using subprocess

Now that we have seen several ways we can make use of subprocess , let’s have a look at something more interesting…

If you remember, we built a script that gives us current weather in this article: Click Package To Create Command Line Interfaces in Python

We ran that program with this command

[program name] [location] -a [api_key]

Now, we are going to run it through subprocess.

In that same directory, create a new file and type the command we used to run the weather program in the run command.

import subprocess subprocess.run("weather Harare -a [your api key]", shell=True)

After importing subprocess , we have added the same command we used to run our weather program the parentheses and quotation marks.

As you see, we are able to get the script to execute without touching weather.py

How to use subprocess.Popen to start programs

Another feature of subprocess is the Popen class. Yes, class. It gives us the ability to open files and or programs from Python.

import subprocess subprocess.Popen("start excel", shell=True)

Executing the code above will start Excel. Alternatively, you could substitute excel with any other program on your system to open it with Python.

You could also choose to go further and create a CLI to run the command more efficiently.

In this article, we have taken a look at the subprocess module – what it is and what it is used for. We have used it to list files in a directory, output that results to a file and read the contents of a file. We have also used subprocess to execute a program we created before and we have seen how we can run open programs using the Popen class of subprocess.

Error SendFox Connection: JSON Parse

Источник