Unknown encoding unicode python

LookupError: unknown encoding in Python

In Python, multiple errors are generated while dealing with variables, functions, and modules in a program. These errors include ImportError, TypeError, NameError, IndexError, etc. Python also causes “LookupError” when unsupported encoding is done in the program. To resolve this error, there are several supported encodings provided by Python.

This Python article will provide a comprehensive guide on the “LookupError: unknown encoding” reason and solutions using the following aspects:

• Encoding in Python
• Reason: Specify Unsupported Encoding
• Solution: Use Standard Encoding

Encoding in Python

In Python, encoding is defined as the representation of a Unicode string as a string of bytes. Encoding is done when the data is transferred over the network. Data is encoded while saving the file on a disk, such as writing data to a “txt” file in Python requires encoding. We can only use those standard encodings that are supported by Python. The list of supported encoding by Python is given below in the official documentation:

Reason: Specify Unsupported Encoding

The main reason invokes the “LookupError: unknown encoding” is when the user specified unsupported encoding in a function. An example of this error is shown in the following snippet:

The above output shows “LookupError: unknown encoding: example” because the specified encoding is not supported by Python.

Solution: Use Standard Encoding

To resolve this error, we specify the encoding supported by Python, such as “utf-8” encoding. Over a million useful Unicode character code points can be encoded with the “utf-8” encoding. The code and output using standard encoding are shown in the below snippet:

with open('itslinuxfoss.txt', 'w', encoding='utf-8') as f: f.write('Python' + '\n') f.write('Guide' + '\n') f.write('Available on' + '\n') f.write('itslinuxfoss' + '\n')

In the above code, the “open()” function is used to open the “itslinuxfoss” text file in “writing” mode. The standard encoding “utf-8” is specified inside the “open()” function. The “write()” function is used multiple times to write the file name

The above output shows the text file having multiple lines written using the “write()” function.

Conclusion

The “LookupError: unknown encoding” occurs in Python programs when a user tries to specify the unsupported encoding in a function. To fix this issue, use Python-supported encoding in a program. Some normally used encodings are utf-8, utf-32, ASCII, Latin-1, etc. This Python article discussed the reason and solution of “LookupError: unknown encoding” using an example.

Источник

Unknown encoding unicode python

Last updated: Feb 17, 2023
Reading time · 2 min

# LookupError: unknown encoding in Python

The Python «LookupError: unknown encoding» occurs when we specify an encoding that is not supported.

To solve the error, use the utf-8 encoding or pick any of the other standard encodings that suit your use case, e.g. latin-1 or ascii .

Here is an example of how the error occurs.

Copied! 
# ⛔️ LookupError: unknown encoding: example with open('example.txt', 'w', encoding='example') as my_file: my_file.write('first line' + '\n') my_file.write('second line' + '\n') my_file.write('third line' + '\n') 

We specified an encoding that is not in the list of standard encodings which caused the error.

# Use the utf-8 encoding instead

Chances are you meant to use the utf-8 encoding which is capable of encoding over a million valid character code points in Unicode.

Copied! 
# ✅ specify 'utf-8' encoding with open('example.txt', 'w', encoding='utf-8') as my_file: my_file.write('first line' + '\n') my_file.write('second line' + '\n') my_file.write('third line' + '\n') 

You can view all of the standard encodings in this table of the official docs.

Some of the common encodings are ascii , latin-1 , utf-16 and utf-32 .

# Calling the encode() method with an unknown encoding

The error also occurs when you call the encode() method with an unknown encoding.

Copied! 
# ⛔️ LookupError: unknown encoding: utf100 my_bytes = 'bobbyhadz.com'.encode('utf100') 

The solution is the same, make sure to pass the correct encoding in the call to str.encode() , e.g. utf-8 .

Copied! 
my_bytes = 'bobbyhadz.com'.encode('utf-8') print(my_bytes) # 👉️ b'bobbyhadz.com' 

The str.encode method returns an encoded version of the string as a bytes object. The default encoding is utf-8 .

You can view all of the standard encodings in this table of the official docs.

# Set the PYTHONIOENCODING environment variable

If the error persists, try to set the PYTHONIOENCODING environment variable.

Copied! 
# on Linux and macOS export PYTHONIOENCODING=utf-8 # on Windows setx PYTHONIOENCODING=utf-8 setx PYTHONLEGACYWINDOWSSTDIO=utf-8 

If the PYTHONIOENCODING environment variable is set before running the interpreter, it overrides the encoding used for stdin and stdout .

If you are on Windows, you also have to set the PYTHONLEGACYWINDOWSSTDIO environment variable.

You can also set the encoding globally by adding the following lines to the top of your file.

Copied! 
import sys sys.stdin.reconfigure(encoding='utf-8') sys.stdout.reconfigure(encoding='utf-8') 

The 3 lines of code override the encoding used for stdin and stdout .

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.

Источник

Fix Python LookupError: unknown encoding issue

For example, suppose you try to encode a string using unicode encoding as follows:

  Since there’s no unicode encoding in Python, the code above causes the LookupError as follows:

This error can occur when working with text files, streams, or any other scenario where you need to specify an encoding format.

To solve this error, you need to select an encoding format that’s officially supported by Python.

For example, you can use either utf-8 or utf-16 when encoding a string as shown below:

 Also, make sure there’s no type when specifying the encoding format.

Python has several aliases for some encoding format. You can use utf8 , utf-8 , or utf_8 but you can’t use uft8 as shown below:

Another common cause for this error is when you call the open() function to open a file stream.

If you are passing an encoding argument to the function, you need to make sure the format is supported by Python:

In the example above, we pass the named argument encoding='utf8' to the open() function. Since the encoding format is supported, the function runs without error.

And that’s how you solve the Python LookupError: unknown encoding issue. Nice work!

Take your skills to the next level ⚡️

I’m sending out an occasional email with the latest tutorials on programming, web development, and statistics. Drop your email in the box below and I’ll send new stuff straight into your inbox!

About

Hello! This website is dedicated to help you learn tech and data science skills with its step-by-step, beginner-friendly tutorials.
Learn statistics, JavaScript and other programming languages using clear examples written for people.

Search

Type the keyword below and hit enter

Tags

Click to see all tutorials tagged with:

Источник

Fix “LookupError: unknown encoding: cp65001” error in Python

Python is a simple, minimalistic, and easy-to-comprehend programming language that is globally-accepted and universally-used today. It’s simple, easy-to-learn and abstract away many low level computation details to allow users to code as if Python is their own mother tongue.

This article will show you what causes “LookupError: unknown encoding: cp65001” error message and a few ways to fix it.

Why does “LookupError: unknown encoding: cp65001” occurs?

If you’re getting this error, you are probably running a legacy version of Python (usually Python 2.x) on Windows.

But first, what is cp65001 ? cp65001 is the Windows way of calling UTF-8, or Unicode. The “LookupError: unknown encoding: cp65001” error message simply means that Unicode characters that your Python script is trying to print cannot be represented on the screen by current encoding.

Windows uses a “code page” system to support multiple languages and characters in the Windows console (Command Prompt and the PowerShell). It’s not only about displaying the characters, but also about encoding/decoding the input and output when the console interacts with external programs.

In this case, you have several options :

• Enable beta support for Unicode UTF-8 on modern versions of Windows (Windows 10+)
• Set encoding for current session using chcp command.
• Set default character encoding in Windows Registry.
• Set PYTHONIOENCODING environment variable to UTF-8.
• Try installing win-unicode-console package from PyPI
• Try another terminal such as Tabby or CMDer

Enable Windows support for UTF-8

Unicode on Windows have always been a complex subject. Starting from Windows 10 insider build 17035 (nominal build 17134), Microsoft has added the option to setting the locale code page to UTF-8. That means most up-to-date Windows 10 and Windows 11 machines are applicable to this method.

1. Press Win+R keyboard shortcut and open intl.cpl .
2. Switch to Administrative tab
3. Click the Change system locale button.
4. Enable Beta: Use Unicode UTF-8 for worldwide language support
5. Reboot for the changes to take effect.

Alternatively, you can create a *.reg file with the following contents and run it with administrator rights to achieve the same results.

Windows Registry Editor Version 

Alternatively, you can put the command chcp 1252 in your PowerShell Profile, which will run it automatically when you open PowerShell. However, this won’t do anything for cmd.exe.

Set default character encoding in Windows Registry

1. Start Run (Win+R)
2. Type regedit.exe
3. Browse to HKEY_CURRENT_USER\Console\%SystemRoot%_system32_cmd.exe
4. Change the value of CodePage to 1252
5. Reboot the machine for the changes to take effect

Set PYTHONIOENCODING environment variable

Python relies on a set of environment variables to initially set up the Python environment at startup, one of them is PYTHONIOENCODING . If this is set before running the interpreter, it overrides the encoding used for stdin/stdout/stderr.

If you’ve tried all the methods above without suceeding, try to run set PYTHONIOENCODING=UTF-8 in Command Prompt without reopening the window to see if everything going well. For PowerShell, run $env:PYTHONIOENCODING = «UTF-8»

Install win-unicode-console package

win-unicode-console is a Python package to enable Unicode input and display when running Python from Windows console. This is great in case you want an automatic way to fix encoding error messages programmatically.

The package is not needed on Python 3.6 and newer since the underlying issue has been resolved (see https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep528).

There are several ways to use win-unicode-console:

• Python patch (recommended)
• Opt-in runner
• Opt-out runner
• Customized runner

As this involves in programming knowledge, you can head over to win-unicode-console page to find out more about how to use the package.

Docker/WSL on Windows

For those of you who are using Docker inside WSL on Windows, you’re essentially inside a double-virtualized environment. This means that things are harder to debug and fix. Users have reported that the following method works for them, however, we haven’t had the chance to confirm it.

If you encounter “LookupError: unknown encoding: cp65001” while working inside Docker/WSL, try creating a file at /usr/bin/docker-compose with the following contents.

 trap INT /mnt/c/Windows/System32/cmd.exe /C chcp After that, you should be able to run docker-compose without any other issues.

We hope that the article helped you successfully debugged LookupError: unknown encoding: cp65001” error in Python, as well as avoid encountering it in the future. We’ve also written a few other guides for fixing common Python errors, such as Fix “Max retries exceeded with URL” ,Python Unresolved Import in VSCode or “IndexError: List Index Out of Range” in Python. If you have any suggestion, please feel free to leave a comment below.

Leave a Comment Cancel reply

Hey! I’m Daan. I work as a SysAdmin in the Netherlands. Whenever I find free time, I write about IT, Linux, Open Source and hardware on this site.

VS Code Tutorials

• Automatic code indentation
• Comment out multiple lines
• Quickly duplicate a line in VSCode
• Create a comment block in VSCode
• Show hidden files in VSCode
• Quickly find a file in VSCode
• How to delete the whole line in VSCode
• Collapse code blocks in VSCode
• Enable/disable word wrap in VSCode
• Bind terminal commands to VSCode keyboard shortcuts
• VSCode Command Palette
• VSCode Format On Save
• VSCode «go to definition» guide

Источник