Remove duplicate string python

Remove duplicate words from a string in Python

In this tutorial, we are going to learn about how to Remove duplicate words from a string in Python along with some examples.

What are duplicate words/strings in a string?

Example: “powerful people come from powerful places”

So here powerful is the duplicate word in our sentence and we have to remove this one “powerful” from our string/sentence using Python.

output: powerful people come from places

There are many ways to find duplicate words in a string but here only we discuss two of them

1. Using for loop to delete duplicate words from a string

string=“powerful people come from powerful places”

Remember Python is case sensitive, if you use “P” capital in 1 st powerful and “p” small in 2 nd powerful then Python takes them as a different word, not a similar word.

• Then we will use an inbuilt function split() which split our string into a list where each word is a list item and we will store this item into a variable “i”.
• Then we will create an empty list “e” in which we will store those words that are not duplicated.
• Then we create or will run a for loop through the list items and then an if statement in which we use a
  1. “count()” function in our string which will return the number of elements that were there in our sentence.
  2. “(i not in e)” – This will check whether “i”(Have a list item from a list “l”) in “e”(new list “e”) or not and return a respective value True or False.
  3. The conditional statement “and” will return a True if both conditions are True or False if one of the conditions is False.

If both conditions are true then our if statement will execute and we will append that particular list item “i”(Have a list item from list “l”) to our new list “e” which is not a duplicate word for “e”.

This for loop and if statement will filter those duplicate word

• At last, we use “ ‘ ‘.join(d)” The .join() method- Join all items in our list into a string followed by a or separated by a space “ ”.

And the last line code prints our new string or words which will not contain any duplicate words.

string EnlighterJSRAW» data-enlighter-language=»python»>string 13fc075bf6c5dd8612a85ab1-text/javascript» data-no-defer=»1″>const post_content = document.getElementById(‘post-content’); const paragraph_for_ad = post_content.getElementsByTagName(«p»); // console.log(«FRK»); //console.log(«frk tag»+paragraph_for_ad.length); if (window.innerWidth 5) < let adparagraph = paragraph_for_ad[2]; adparagraph.insertAdjacentHTML('afterEnd', '

‘); > if (paragraph_for_ad.length > 15) < let adparagraph = paragraph_for_ad[10]; adparagraph.insertAdjacentHTML('afterEnd', '

‘); > if (paragraph_for_ad.length > 30) < let adparagraph = paragraph_for_ad[25]; adparagraph.insertAdjacentHTML('afterEnd', '

‘); > > else if(window.innerWidth > 1360) < if (paragraph_for_ad.length >15) < let adparagraph = paragraph_for_ad[10]; // 71161633/article_incontent_1/article_incontent_1 adparagraph.insertAdjacentHTML('afterEnd', '

‘); > if (paragraph_for_ad.length > 22) < let adparagraph = paragraph_for_ad[18]; // 71161633/article_incontent_2/article_incontent_2 adparagraph.insertAdjacentHTML('afterEnd', '

‘); > if (paragraph_for_ad.length > 30) < let adparagraph = paragraph_for_ad[29]; // 71161633/article_incontent_1/article_incontent_1 adparagraph.insertAdjacentHTML('afterEnd', '

Источник

Remove all duplicates from a given string in Python

In this tutorial, you will learn to remove all duplicates from a given string in Python. Strings in Python are a sequence of characters wrapped inside single, double, or triple quotes. For a given string we have to remove all the characters which occur more than once in the string. We will follow the order in which the characters appear. For example,

Input: «stringis»

Output: «string»

To solve this problem, there are several different approaches,

We will look at these approaches separately

Approach 1: using OrderedDict() function

In this approach, we will use the OrderedDict() method from the collections class and fromkeys() in our program.

OrderedDict is a dictionary subclass that remembers the order of the keys that were inserted first. Since there can’t be duplicate keys this method will return the string after removing the duplicate characters.

Algorithm

Follow the algorithm to understand the approach better:

Step 1- Import OrderedDict from collections class

Step 2- Define a function that will remove duplicates

Step 3- Declare a string with characters

Step 4- Call function to remove characters in that string

Step 5- Print value returned by the function

Python Program 1

Look at the program to understand the implementation of the above-mentioned approach.

from collections import OrderedDict def remove_duplicate(s): return "".join(OrderedDict.fromkeys(s)) # test s="abcfgbsca" print(s) print("After removing duplicates: ",remove_duplicate(s))

abcfgbsca
After removing duplicates: abcfgs

Approach 2: OrderedDict

In this approach, we will convert the string to a set by using the set() method. Then we will declare another string and store characters that are not already in the string. This new string will contain the resultant string.

Algorithm

Follow the algorithm to understand the approach better

Step 1- Import OrderedDict from collections class

Step 2- Define a function that will remove duplicates

Step 3- Create a set of string and store

Step 4- Declare a new empty string

Step 5- Run a loop and add each character in the empty string if it is already not present in the string

Python Program 2

Look at the program to understand the implementation of the above-mentioned approach.

# remove duplicates in string from collections import OrderedDict def remove_duplicate(s): string=set(s) string="".join(string) dup="" for i in s: if(i in dup): pass else: dup=dup+i print("After removing: ",dup) s="stdsrdthw" print(s) print(remove_duplicate(s))

stdsrdthw
After removing: stdrhw

Conclusion

In this tutorial, we have discussed two approaches for removing duplicates from a string. We have also discussed how to use methods of the collection class and use them to remove duplicates.

Источник

Python – Remove Consecutive Duplicates From String

In this tutorial, we will look at how to remove consecutive duplicates from a string in Python with the help of some examples.

How to remove consecutive duplicates from a string?

You can use a loop to iterate over each character in a string and create a new string with the consecutive duplicates removed. The following are the steps –

1. Iterate over each character in the string.
2. For each character check if it’s the same as the previous character (stored in a variable). If it is, skip to the next iteration, else add the character to our result string.
3. Return the result string.

Note that, strings are immutable in Python, thus, you cannot modify the original string but you can create a new string with the consecutive duplicates removed.

You can use the above method in programming languages other than Python (for example, C++, Java, etc.) since it does not use any language-specific functions or constructs.

Let’s write a function in Python to perform the above steps.

# remove consecutive duplicates from string def remove_consec_duplicates(s): new_s = "" prev = "" for c in s: if len(new_s) == 0: new_s += c prev = c if c == prev: continue else: new_s += c prev = c return new_s

Now, let’s apply the above function on the string “aabbbbcccd”.

# string with consecutive duplicates s = "aabbbbcccd" print(s) # remove consecutive duplicates s = remove_consec_duplicates(s) print(s)

Here we remove consecutive duplicates from string s and store the resulting string back to s. Note that the resulting string does not have any adjacent repeating characters.

Note that this function does not give a string of unique characters from the string rather it removes adjacent repeating characters. For example –

# string with consecutive duplicates s = "aabbcccaaaa" print(s) # remove consecutive duplicates s = remove_consec_duplicates(s) print(s)

In the above result, you can see that we get ‘a’ two times in the resulting string, this is because our objective is to remove consecutive duplicates and not to get unique characters in the string.

You might also be interested in –

Subscribe to our newsletter for more informative guides and tutorials.
We do not spam and you can opt out any time.

Author

Piyush is a data professional passionate about using data to understand things better and make informed decisions. He has experience working as a Data Scientist in the consulting domain and holds an engineering degree from IIT Roorkee. His hobbies include watching cricket, reading, and working on side projects. View all posts

Data Science Parichay is an educational website offering easy-to-understand tutorials on topics in Data Science with the help of clear and fun examples.

Career Guides

• Start Your Path —
• Data Scientist
• Data Analyst
• Data Engineer
• Machine Learning Engineer
• Statistician
• Data Architect
• Software Developer

Источник

Python | Remove all duplicates words from a given sentence

Given a sentence containing n words/strings. Remove all duplicates words/strings which are similar to each others.

Input : Geeks for Geeks Output : Geeks for Input : Python is great and Java is also great Output : is also Java Python and great

We can solve this problem quickly using python Counter() method. Approach is very simple.

1) Split input sentence separated by space into words.
2) So to get all those strings together first we will join each string in given list of strings.
3) Now create a dictionary using Counter method having strings as keys and their frequencies as values.
4) Join each words are unique to form single string.

Python

and great Java Python is also

Time Complexity: O(N)
Auxiliary Space: O(N)

Python

Python is great and Java also

Time Complexity: O(N*N)
Auxiliary Space: O(N)

Method 3: Another shorter implementation:

Python3

Python is great and Java also

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 4: Using set()

Python3

Java also great and Python is

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 5: using operator.countOf()

Python3

Python is great and Java also

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 6:

It uses a loop to traverse through each word of the sentence, and stores the unique words in a separate list using an if condition to check if the word is already present in the list.

Follow the steps below to implement the above idea:

• Split the given sentence into words/strings and store it in a list.
• Create an empty set to store the distinct words/strings.
• Iterate over the list of words/strings, and for each word, check if it is already in the set.
• If the word is not in the set, add it to the set.
• If the word is already in the set, skip it.
• Finally, join the words in the set using a space and return it as the output.

Below is the implementation of the above approach:

Python3

Python is great and Java also

Time complexity: O(n^2) because of the list result that stores unique words, which is searched for every word in the input sentence.
Auxiliary space: O(n) because we are storing unique words in the result list.

Method 7: Using Recursive method.

1. Split the input sentence into words.
2. If there is only one word, return it.
3. If the first word is present in the rest of the words, call the function recursively with the rest of the words.
4. If the first word is not present in the rest of the words, concatenate it with the result of calling the function recursively with the rest of the words.
5. Return the final result as a string.

Python3

Python and Java is also great

Time complexity:
The time complexity of this algorithm is O(n^2), where n is the number of words in the input sentence. This is because for each word in the input sentence, we are checking if it is present in the rest of the words using the in operator, which has a time complexity of O(n) in the worst case. Therefore, the total time complexity of the algorithm is O(n^2).

Space complexity:
The space complexity of this algorithm is O(n), where n is the number of words in the input sentence. This is because we are using recursion to call the function with smaller subsets of the input sentence, which results in a recursive call stack. The maximum depth of the call stack is equal to the number of words in the input sentence, so the space complexity is O(n). Additionally, we are creating a list to store the words in the output, which also takes O(n) space. Therefore, the total space complexity of the algorithm is O(n).

Method #8:Using reduce

1. The remove_duplicates function takes an input string as input and splits it into a list of words using the split() method. This takes O(n) time where n is the length of the input string.
2. The function initializes an empty list unique_words to store the unique words in the input string.
3. The function uses the reduce() function from the functools module to iterate over the list of words and remove duplicates. The reduce() function takes O(n) time to execute where n is the number of words in the input string.
4. The lambda function inside the reduce() function checks if a word is already in the accumulator list x and either returns x unchanged or appends the new word y to the list x.
5. Finally, the function returns a string joined from the list of unique words using the join() method. This takes O(n) time where n is the length of the output string.

Источник