Python file search replace

Search the word, and replace the whole line containing the word in a file in Python using fileinput

I want to search a particular word in a text file. For each line,where the word is present, I want to completely change the line by a new text. I want to achieve this using fileinput module of python. There are two observation, I am seeing with following variations :- Code piece 1 :-

text = "mov9 = " # if any line contains this text, I want to modify the whole line. new_text = "mov9 = Alice in Wonderland" x = fileinput.input(files="C:\Users\Admin\Desktop\DeletedMovies.txt", inplace=1) for line in x: if text in line: line = new_text print line, x.close() 

The above piece of code wipes out all the content of the file, and writes the new_text i.e. the file content is only

text = "mov9 = " # if any line contains this text, I want to modify the whole line. new_text = "mov9 = Alice in Wonderland" x = fileinput.input(files="C:\Users\Admin\Desktop\DeletedMovies.txt", inplace=1) for line in x: if text in line: line = line.replace(text, new_text) print line, x.close() 

The above piece of code, even though adds the needed line i.e. new_text where text is found, but doesn’t deletes the line, but keeps the previous data also. That is if the line was earlier :-

mov9 = Alice in WonderlandFast & Furios 

And other content of the files remain untouched, not deleted as in code in Code piece 1. But my goal is to find the word mov9 = , and whatever is present along with it, I want to replace the whole line as mov9 = Alice in Wonderland . How can I achieve that? Thanks in advance.

Источник

Search and replace a line in a file in Python

I want to loop over the contents of a text file and do a search and replace on some lines and write the result back to the file. I could first load the whole file in memory and then write it back, but that probably is not the best way to do it. What is the best way to do this, within the following code?

f = open(file) for line in f: if line.contains('foo'): newline = line.replace('foo', 'bar') # how to write this newline back to the file 

13 Answers 13

The shortest way would probably be to use the fileinput module. For example, the following adds line numbers to a file, in-place:

import fileinput for line in fileinput.input("test.txt", inplace=True): print('<> <>'.format(fileinput.filelineno(), line), end='') # for Python 3 # print "%d: %s" % (fileinput.filelineno(), line), # for Python 2 

1. The original file is moved to a backup file
2. The standard output is redirected to the original file within the loop
3. Thus any print statements write back into the original file

fileinput has more bells and whistles. For example, it can be used to automatically operate on all files in sys.args[1:] , without your having to iterate over them explicitly. Starting with Python 3.2 it also provides a convenient context manager for use in a with statement.

While fileinput is great for throwaway scripts, I would be wary of using it in real code because admittedly it’s not very readable or familiar. In real (production) code it’s worthwhile to spend just a few more lines of code to make the process explicit and thus make the code readable.

1. The file is not overly large, and you can just read it wholly to memory. Then close the file, reopen it in writing mode and write the modified contents back.
2. The file is too large to be stored in memory; you can move it over to a temporary file and open that, reading it line by line, writing back into the original file. Note that this requires twice the storage.

Источник

Recursively find and replace string in text files

I want to recursively search through a directory with subdirectories of text files and replace every occurrence of within the files with the contents of a multi line string. How can this be achieved with Python? So far all I have is the recursive code using os.walk to get a list of files that are required to be changed.

import os import sys fileList = [] rootdir = "C:\\test" for root, subFolders, files in os.walk(rootdir): if subFolders != ".svn": for file in files: fileParts = file.split('.') if len(fileParts) > 1: if fileParts[1] == "php": fileList.append(os.path.join(root,file)) print fileList 

Why do you need to use Python for this? A combination of find and sed would do the job more elegantly, in my opinion. developmentality.wordpress.com/2010/09/07/… Basically, find . -type f -exec sed -i ‘.bk’ ‘s/search regexp/replacement string/g’ <> \; (Haven’t tested but I think that’s the correct syntax)

@I82Much — but your solution would require learning sed. If OP already knows basic Python, and is not obsessed learning new tool (prefers mastering existing ones), solving the task in Python makes more sense.

@I82Much: c:\test might be one reason (Windows). find /c/test -type d -path \*/.svn -prune -o -type f -name \*.php -exec sed -i ‘s//multiline\nstring/g’ <> + shell command is close to this Python code

10 Answers 10

os.walk is great. However, it looks like you need to filer file types (which I would suggest if you are going to walk some directory). To do this, you should add import fnmatch .

import os, fnmatch def findReplace(directory, find, replace, filePattern): for path, dirs, files in os.walk(os.path.abspath(directory)): for filename in fnmatch.filter(files, filePattern): filepath = os.path.join(path, filename) with open(filepath) as f: s = f.read() s = s.replace(find, replace) with open(filepath, "w") as f: f.write(s) 

This allows you to do something like:

findReplace("some_dir", "find this", "replace with this", "*.txt") 

import os replacement = """some multi-line string""" for dname, dirs, files in os.walk("some_dir"): for fname in files: fpath = os.path.join(dname, fname) with open(fpath) as f: s = f.read() s = s.replace("", replacement) with open(fpath, "w") as f: f.write(s) 

The above solution has flaws, such as the fact that it opens literally every file it finds, or the fact that each file is read entirely into memory (which would be bad if you had a 1GB text file), but it should be a good starting point.

You also may want to look into the re module if you want to do a more complex find/replace than looking for a specific string.

For those using Python 3.5+ you can now use a glob recursively with the use of ** and the recursive flag.

Here’s an example replacing hello with world for all .txt files:

for filepath in glob.iglob('./**/*.txt', recursive=True): with open(filepath) as file: s = file.read() s = s.replace('hello', 'world') with open(filepath, "w") as file: file.write(s) 

For windows might end up with error ‘UnicodeDecodeError: ‘charmap’ codec can’t decode byte 0x9d in position 676628: character maps to’ This is encoding error. Refer this stackoverflow.com/a/9233174/452708

To avoid recursing into .svn directories, os.walk() allows you to change the dirs list inplace. To simplify the text replacement in a file without requiring to read the whole file in memory, you could use fileinput module. And to filter filenames using a file pattern, you could use fnmatch module as suggested by @David Sulpy:

#!/usr/bin/env python from __future__ import print_function import fnmatch import os from fileinput import FileInput def find_replace(topdir, file_pattern, text, replacement): for dirpath, dirs, files in os.walk(topdir, topdown=True): dirs[:] = [d for d in dirs if d != '.svn'] # skip .svn dirs files = [os.path.join(dirpath, filename) for filename in fnmatch.filter(files, file_pattern)] for line in FileInput(files, inplace=True): print(line.replace(text, replacement), end='') find_replace(r"C:\test", "*.php", '', "multiline\nreplacement") 

This is an old question but I figured I’d provide an updated and simpler answer using current libraries in python3.8.

from pathlib import Path import re rootdir = Path("C:\\test") pattern = r'REGEX for the text you want to replace' replace = r'REGEX for what to replace it with' for file in [ f for f in rootdir.glob("**.php") ]: #modify glob pattern as needed file_contents = file.read_text() new_file_contents = re.sub(f"", f"", file_contents) file.write_text(new_file_contents) 

As mentioned by others, the way to go is glob. However, use binary mode to prevent changing line endings when you find/replace.

#! /usr/bin/env python3 replacements = [ # find, replace (b'#include ', b'#include '), (b'#include ', b'#include '), (b'#include ', b'#include '), ] # path to files. here all cpp files in the current working directory globpath = './**/*.cpp' import glob for filepath in glob.iglob(globpath, recursive=True): with open(filepath, 'rb') as file: s = file.read() for f, r in replacements: s = s.replace(f, r) with open(filepath, "wb") as file: file.write(s) 

This lets you use a decorator to create something like:

@manip(at='.php$', recursive=True) # to apply to subfolders def replace_on_php(text, find, replacement): return text.replace(find, replacement) 

Now in your prompt you should be able to call

replace_on_php('explode', 'myCustomExplode', path='./myPhPFiles', modify=True) 

and this should make the function apply itself on the entire folder.

Here’s my code (which I think is the same as the above but I’m including it just in case there’s something subtly different about it):

import os, fnmatch, sys def findReplace(directory, find, replace, filePattern): for path, dirs, files in os.walk(os.path.abspath(directory)): for filename in fnmatch.filter(files, filePattern): filepath = os.path.join(path, filename) with open(filepath) as f: s = f.read() s = s.replace(find, replace) with open(filepath, "w") as f: f.write(s) 

it runs without error. BUT, the file, in z:\test is unchanged. I’ve put in print statements, like print(«got here») but they don’t print out either.

Sulpy’s answer is good but incomplete. The user would be likely to want to input the parameters through an entry widget, so we might have something more like this (also incomplete, but left as an exercise):

import os, fnmatch from Tkinter import * fields = 'Folder', 'Search', 'Replace', 'FilePattern' def fetch(entvals): # print entvals # print ents entItems = entvals.items() for entItem in entItems: field = entItem[0] text = entItem[1].get() print('%s: "%s"' % (field, text)) def findReplace(entvals): # print ents directory = entvals.get("Folder").get() find = entvals.get("Search").get() replace = entvals.get("Replace").get() filePattern = entvals.get("FilePattern").get() for path, dirs, files in os.walk(os.path.abspath(directory)): for filename in fnmatch.filter(files, filePattern): # print filename filepath = os.path.join(path, filename) print filepath # Can be commented out -- used for confirmation with open(filepath) as f: s = f.read() s = s.replace(find, replace) with open(filepath, "w") as f: f.write(s) def makeform(root, fields): entvals = <> for field in fields: row = Frame(root) lab = Label(row, width=17, text=field+": ", anchor='w') ent = Entry(row) row.pack(side=TOP, fill=X, padx=5, pady=5) lab.pack(side=LEFT) ent.pack(side=RIGHT, expand=YES, fill=X) entvals[field] = ent # print ent return entvals if __name__ == '__main__': root = Tk() root.title("Recursive S&R") ents = makeform(root, fields) # print ents root.bind('', (lambda event, e=ents: fetch(e))) b1 = Button(root, text='Show', command=(lambda e=ents: fetch(e))) b1.pack(side=LEFT, padx=5, pady=5) b2 = Button(root, text='Execute', command=(lambda e=ents: findReplace(e))) b2.pack(side=LEFT, padx=5, pady=5) b3 = Button(root, text='Quit', command=root.quit) b3.pack(side=LEFT, padx=5, pady=5) root.mainloop() 

Источник