Faster file search with Python
Computer users often have a problem with file search as they tend to forget the location or path of a file even though Windows provides a file search utility. The Explorer in Windows 7 offers a search facility but it takes around two to three minutes to search a file. In this article, I will give you a Python program which will search a file on your computer’s hard disk, within hardly one second.
Let us first understand the program’s logic. Figure 1 explains this. Let us first do indexing or, in Python language terms, let’s construct a dictionary in which the file will be the key of the dictionary and the value will be the path of the file. The dictionary will be dumped into the pickle file. The next time, the file will be searched in the dictionary (dumped in the pickle file).
Now that you have understood the logic of the program, let us look at the program in detail. I have broken it into different functions. Let’s see what each function does.
#program created by mohit #offical website L4wisdom.com # email-id mohitraj.cs@gmail.com
The block of code below imports the essential modules:
import os import re import sys from threading import Thread from datetime import datetime import subprocess import cPickle dict1 = <>
Next, let’s write a function to acquire the drives. This function gets all the drives in your Windows machine. If you have inserted any external/USB pen drive or hard drive disk, the function also obtains details for them.
def get_drives(): response = os.popen("wmic logicaldisk get caption") list1 = [] total_file = [] t1= datetime.now() for line in response.readlines(): line = line.strip("\n") line = line.strip("\r") line = line.strip(" ") if (line == "Caption" or line == ""): continue list1.append(line) return list1 
Our next function is the search1 function, which constructs a dictionary in which the file name is the key and the path is the value of the dictionary.
def search1(drive): for root, dir, files in os.walk(drive, topdown = True): for file in files: file= file.lower() if file in dict1: file = file+”_1” dict1[file]= root else : dict1[file]= root
The create function opens the thread process for each drive, and each thread process calls the search1 function.
def create(): t1= datetime.now() list2 = [] # empty list is created list1 = get_drives() print list1 for each in list1: process1 = Thread(target=search1, args=(each,)) process1.start() list2.append(process1) for t in list2: t.join() # Terminate the threads
After creating the dictionary, the following code dumps the dictionary into the hard disk as a pickle file.
pickle_file = open(“finder_data”,”w”) cPickle.dump(dict1,pickle_file) pickle_file.close() t2= datetime.now() total =t2-t1 print “Time taken to create “ , total print “Thanks for using L4wisdom.com”
Next time, when you search any file, the program will search the file in the dumped dictionary, as follows:
if len(sys.argv) < 2 or len(sys.argv) >2: print “Please use proper format” print “Use to create database file” print “Use to search file” print “Thanks for using L4wisdom.com” elif sys.argv[1] == ‘-c’: create() else: t1= datetime.now() try: pickle_file = open(“finder_data”, “r”) file_dict = cPickle.load(pickle_file) pickle_file.close() except IOError: create() except Exception as e : print e sys.exit() file_to_be_searched = sys.argv[1].lower() list1= [] print “Path \t\t: File-name”
Here, we used the search method of regular expressions so that we can use a regular expression to find the file.
for key in file_dict: if re.search(file_to_be_searched, key): str1 = file_dictWindows search python files+” : “+key list1.append(str1) list1.sort() for each in list1: print each print “-----------------------” t2= datetime.now() total =t2-t1 print “Total files are”, len(list1) print “Time taken to search “ , total print “Thanks for using L4wisdom.com”
The rest of the code is very easy to understand.
Let us save the complete code as finder.py (you can also download it from https://www.opensourceforu.com/article_source_code/sept16/finder.zip) and make it a Windows executable (exe) file using the Pyinstaller module. Run the command shown in Figure 2. After running it successfully, you can find the finder.exe in folder C:\PyInstaller-2.1\finder\dist .
You can put the finder.exe file in the Windows folder, but if you place this in a different folder, you will have to set the path to that folder. Let us run the program. You can see from Figure 3 that just 33 seconds are required to create the database. Now search the file and see the power of the program.
I am going to search for songs which contain the string waada.
Look at Figure 4. You can see that two searches have taken approximately half a second. The program is case insensitive, so using upper case or lower case doesn’t matter. The program also has the power of regular expressions. Let’s assume that you want to search files which contain the wada or waada strings. Let us look at Figure 5. The regular expression ‘a+’ means the letter ‘a’ can appear once or many times. Again, you can get the result in less than one second. Let us consider one more example of a regular expression search. Let’s assume that you want to search the files which contain wa+da with digit numbers (see the first search of Figure 6). Assume that you want to search the files that start with the string wa+da (see the results of the second search in Figure 6).
The program is indeed very useful. Suppose, for instance, you have forgotten the file path but have a vague idea of the file name. You can search the file within one second by using regular expressions. The best part is the speed of the search. You can try searching with the file name repeatedly. But if you use Windows Explorer, each search will take around two to four minutes.
Find Files Using Python
- Find File With the os.walk() Function in Python
- Find File With the glob.glob() Function in Python
- Find File With the Path.glob() Function in Python
This tutorial will discuss the methods to find a file in Python.
Find File With the os.walk() Function in Python
If we want to find the path of a specific file on our machine with python, we can use the os module. The os module provides many os-related functionalities to our code. The os.walk() function takes a path string as an input parameter and gives us the directory path, the directory name, and the filename for each file in the path . The sample code below shows us how to find a file in Python with the os.walk() function.
import os def findfile(name, path): for dirpath, dirname, filename in os.walk(path): if name in filename: return os.path.join(dirpath, name) filepath = findfile("file2.txt", "/") print(filepath) /Users\maisa\Documents\PythonProjects\file2.txt In the above code, we declared the findfile() function that uses os.walk() function to find our file. The findfile() function takes the file’s name and the root path as input parameters and returns the path of our specified file. This approach gives us the absolute path of the file.
Find File With the glob.glob() Function in Python
We can also use the glob.glob() function to solve our current problem. The glob.glob() function takes a pathname as an input parameter and returns a list of all the file paths that match the input argument. We can specify a regular expression as an input parameter that matches our file only. The sample code below shows us how to find a file in Python with the glob.glob() function.
import glob filepath = glob.glob('**/file.txt', recursive=True) print(filepath) We passed our file name as the input parameter to the glob.glob() function, and it returned the relative path of our file. This method can give us the relative path as well as the absolute path of our file.
Find File With the Path.glob() Function in Python
Another approach is to use the pathlib module. This Python module offers classes that represent filesystem paths for different operating systems. We can use the Path.glob() function inside the pathlib module to solve our specific problem. This function is similar to the glob() function inside the glob module. The Path.glob() function takes a pattern as an input parameter and returns a list of path objects that match the input argument. The sample code snippet shows us how to find a file in Python with the pathlib module.
import pathlib filepath = sorted(pathlib.Path('.').glob('**/file2.txt')) print(filepath) We passed a pattern string that matches our file to the Path.glob() function. The Path.glob() function returns us a list of WindowsPath objects that match the pattern. With this method, we get path objects specific to our operating system.
Maisam is a highly skilled and motivated Data Scientist. He has over 4 years of experience with Python programming language. He loves solving complex problems and sharing his results on the internet.