Python split sentence into words

How to Split a Sentence into a List of Words in Python

In this Python tutorial, we will discuss how to split a sentence into a list of words in Python. We will explore different ways to split a sentence into a list of words in Python, with examples and output.

There are different ways to split a sentence into a list of words in Python. A few are:

• Using the split() method
• Using list comprehension with split()
• Using regular expressions (re module)
• Using the nltk library

Split a Sentence into a List of Words in Python

Now, let us see the above methods with examples.

Method-1: Using the split() method

The simplest and most common way to split a sentence into a list of words in Python is by using the split() method available for strings in Python. The split() method splits a string based on a specified delimiter (by default, it is a whitespace).

sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words words = sentence.split() print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America.']

Method-2: Using list comprehension with split()

List comprehension provides a more concise way to create lists in Python. You can use list comprehension along with the split() method to split a sentence into list of words in Python.

sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using list comprehension words = [word for word in sentence.split()] print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America.']

Method 3: Using regular expressions (re module)

The re module in Python provides functionality to work with regular expressions. The split() function from the re module can be used to split a sentence into a list of words in Python based on a given pattern.

import re sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using regular expressions words = re.split(r'\W+', sentence) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America', '']

Note that the output contains an empty string at the end, which can be removed using the filter() function.

import re sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using regular expressions words = re.split(r'\W+', sentence) # Filtering out the empty strings words = list(filter(None, words)) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America']

Method 4: Using the nltk library

The Natural Language Toolkit (nltk) is a popular Python library for working with human language data. It provides a wide range of functionalities for text processing and natural language processing. One of the tools provided by nltk is the word_tokenize() function, which can be used to split a sentence into a list of words in Python.

First, you will need to install the nltk library if you haven’t already:

Then, you will need to download the ‘punkt’ tokenizer models:

import nltk nltk.download('punkt')

Now you can use the word_tokenize() function to split a sentence into a list of words.

import nltk sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using nltk.word_tokenize() words = nltk.word_tokenize(sentence) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America', '.']

Note that the output includes punctuation marks as separate tokens. If you want to exclude them, you can use a list comprehension along with the isalnum() method to filter out the non-alphanumeric tokens.

import nltk sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using nltk.word_tokenize() words = nltk.word_tokenize(sentence) # Filtering out non-alphanumeric tokens words = [word for word in words if word.isalnum()] print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America']

Conclusion

In this tutorial, we have explored three different ways to split a sentence into a list of words in Python: using the split() method, using list comprehension with split() , and using the re module.

You may also like the following Python string tutorials:

I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.

Источник

Как разбить предложение на список слов в Python

В этом руководстве по Python мы обсудим, как разбить предложение на список слов в Python. Мы рассмотрим различные способы разделения предложения на список слов в Python с примерами и выводом.

В Python есть разные способы разбить предложение на список слов. Вот некоторые из них:

• Использование метода split()
• Использование понимания списка с помощью split()
• Использование регулярных выражений (модуль re)
• Использование библиотеки nltk

Разделить предложение на список слов в Python

Теперь давайте рассмотрим приведенные выше методы на примерах.

Способ 1: Использование метода split()

Самый простой и распространенный способ разбить предложение на список слов в Python заключается в использовании split() метод, доступный для строк в Python. split() метод разбивает строку на основе указанного разделителя (по умолчанию это пробел).

sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words words = sentence.split() print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America.']

Способ 2: Использование понимания списка с помощью split()

Понимание списков обеспечивает более краткий способ создания списков в Python. Вы можете использовать понимание списка вместе с split() метод разделения предложения на список слов в Python.

sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using list comprehension words = [word for word in sentence.split()] print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America.']

Способ 3: использование регулярных выражений (перемодуль)

re Модуль в Python предоставляет функциональные возможности для работы с регулярными выражениями. split() функцию от re Модуль можно использовать для разделения предложения на список слов в Python на основе заданного шаблона.

import re sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using regular expressions words = re.split(r'\W+', sentence) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America', '']

Обратите внимание, что вывод содержит пустую строку в конце, которую можно удалить с помощью команды filter() функция.

import re sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using regular expressions words = re.split(r'\W+', sentence) # Filtering out the empty strings words = list(filter(None, words)) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America']

Способ 4: Использование библиотеки nltk

Natural Language Toolkit (nltk) — это популярная библиотека Python для работы с данными человеческого языка. Он предоставляет широкий спектр функций для обработки текста и обработки естественного языка. Одним из инструментов, предоставляемых nltk, является word_tokenize() Функция, которую можно использовать для разделения предложения на список слов в Python.

Во-первых, вам нужно будет установить библиотеку nltk, если вы еще этого не сделали:

Затем вам нужно будет загрузить модели токенизатора punkt:

import nltk nltk.download('punkt')

Теперь вы можете использовать word_tokenize() Функция для разделения предложения на список слов.

import nltk sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using nltk.word_tokenize() words = nltk.word_tokenize(sentence) print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America', '.']

Обратите внимание, что выходные данные включают знаки препинания в виде отдельных токенов. Если вы хотите исключить их, вы можете использовать понимание списка вместе с isalnum() метод для фильтрации небуквенно-цифровых токенов.

import nltk sentence = "Python is one of the popular programming languages in the United States of America." # Splitting the sentence into words using nltk.word_tokenize() words = nltk.word_tokenize(sentence) # Filtering out non-alphanumeric tokens words = [word for word in words if word.isalnum()] print(words)

['Python', 'is', 'one', 'of', 'the', 'popular', 'programming', 'languages', 'in', 'the', 'United', 'States', 'of', 'America']

Заключение

В этом уроке мы рассмотрели три различных способа разбить предложение на список слов в Python: используя split() метод, используя понимание списка с split() и с помощью re модуль.

Вам также могут понравиться следующие руководства по строкам Python:

Python — один из самых популярных языков в Соединенных Штатах Америки. Я давно работаю с Python и имею опыт работы с различными библиотеками на Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn и т. д. У меня есть опыт работы с различными клиентами в таких странах, как США, Канада, Великобритания, Австралия, Новая Зеландия и т. д. Проверьте мой профиль.

Источник

Split Sentence Into Words in Python

1. Split Sentence Into Words With the str.split() Function in Python
2. Split Sentence Into Words With List Comprehensions in Python
3. Split Sentence Into Words With the nltk Library in Python

This tutorial will discuss the methods to split a sentence into a list of words in Python.

Split Sentence Into Words With the str.split() Function in Python

The str.split() function in Python takes a separator as an input parameter and splits the calling string into multiple strings based on the separator. If we don’t specify any separator, the str.split() function splits the string on the basis of empty spaces. The following code snippet shows us how to split a sentence into a list of words with the str.split() function.

sentence = "This is a sentence" words = sentence.split() print(words) 

We declared a string variable sentence that contains some data. We then split the sentence variable into a list of strings with the sentence.split() function and stored the results into the words list. The str.split() function is the easiest way to convert a sentence into a list of words in Python.

Split Sentence Into Words With List Comprehensions in Python

We can also use list comprehensions to split a sentence into a list of words. However, this approach isn’t as straightforward as the str.split() function. The advantage of using list comprehensions is that we can also perform some operations on the obtained words. The operations could range from appending something to each word or removing something from each word. The following code snippet shows us how to split a sentence into words with list comprehensions and the str.split() function.

sentence = "This is a sentence" words = [word for word in sentence.split()] print(words) 

We declared a string variable sentence that contains some data. We then split the sentence variable into a list of strings with list comprehension and stored the results into the words list. This method is useful to modify each obtained word before storing the word into the words list.

Источник