UnicodeEncodeError: ‘ascii’ codec can’t encode character in position 0: ordinal not in range(128)
I’m working on a Python script that uses the scissor character (9986 — ✂) and I’m trying to port my code to Mac, but I’m running into this error. The scissor character shows up fine when run from IDLE (Python 3.2.5 — OS X 10.4.11 iBook G4 PPC) and the code works entirely fine on Ubuntu 13.10, but when I attempt to run this in the terminal I get this error/traceback:
Traceback (most recent call last): File "snippets-convert.py", line 352, in main() File "snippets-convert.py", line 41, in main menu() File "snippets-convert.py", line 47, in menu print ("|\t ",snipper.decode(),"PySnipt'd",snipper.decode(),"\t|") UnicodeEncodeError: 'ascii' codec can't encode character '\u2702' in position 0: ordinal not in range(128) - UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xef’ in position 0: ordinal not in range(128) — Different character
- «UnicodeEncodeError: ‘ascii’ codec can’t encode character» — Using 2.6, so don’t know if it applies
- UnicodeEncodeError: ‘ascii’ codec can’t encode character? — Seems to be a plausible solution to my problem, .encode(‘UTF-8’) , I don’t get the error. However, it displays a character code, not the character I want, and .decode() just gives me the same error. Not sure if I’m doing this right.
- UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 0-6: ordinal not in range(128) — Not sure if this applies, he’s using a GUI, getting input, and all in Greek.
What’s causing this error? Is it the age of the system/OS, the version of Python, or some programming error?
EDIT: This error crops up later with this duplicate issue (just thought I’d add it as it is within the same program and is the same error):
Traceback (most recent call last): File "snippets-convert.py", line 353, in main() File "snippets-convert.py", line 41, in main menu() File "snippets-convert.py", line 75, in menu main() File "snippets-convert.py", line 41, in main menu() File "snippets-convert.py", line 62, in menu search() File "snippets-convert.py", line 229, in search print_results(search_returned) # Print the results for the user File "snippets-convert.py", line 287, in print_results getPath(toRead) # Get the path for the snippet File "snippets-convert.py", line 324, in getPath snipXMLParse(path) File "snippets-convert.py", line 344, in snipXMLParse print (chr(164),child.text) UnicodeEncodeError: 'ascii' codec can't encode character '\xa4' in position 0: ordinal not in range(128) I went into the terminal character settings and it does in fact support that character (as you can see in this screenshot:
when I insert it into terminal it prints out this: \342\234\202 and when I press Enter I get this: -bash: ✂: command not found
EDIT Ran commands as @J.F. Sebastian asked:
PYTHONIOENCODING: None locale(False): US-ASCII device(stdout): US-ASCII stdout.encoding: US-ASCII device(stderr): US-ASCII stderr.encoding: US-ASCII device(stdin): US-ASCII stdin.encoding: US-ASCII locale(False): US-ASCII locale(True): US-ASCII python3 -S test-io-encoding.py :
PYTHONIOENCODING: None locale(False): US-ASCII device(stdout): US-ASCII stdout.encoding: US-ASCII device(stderr): US-ASCII stderr.encoding: US-ASCII device(stdin): US-ASCII stdin.encoding: US-ASCII locale(False): US-ASCII locale(True): US-ASCII EDIT Tried the «hackerish» solution provided by @PauloBu:
As you can see, this caused one (Yay!) scissor, but I am now getting a new error. Traceback/error:
+-=============================-+ ✂Traceback (most recent call last): File "snippets-convert.py", line 357, in main() File "snippets-convert.py", line 44, in main menu() File "snippets-convert.py", line 52, in menu print("|\t "+sys.stdout.buffer.write(chr(9986).encode('UTF-8'))+" PySnipt'd "+ sys.stdout.buffer.write(chr(9986).encode('UTF-8'))+" \t|") TypeError: Can't convert 'int' object to str implicitly EDIT Added results of @PauloBu’s fix:
Python3: UnicodeEncodeError: ‘ascii’ codec can’t encode character ‘\xfc’
I’am trying to get running a very simple example on OSX with python 3.5.1 but I’m really stucked. Have read so many articles that deal with similar problems but I can not fix this by myself. Do you have any hints how to resolve this issue? I would like to have the correct encoded latin-1 output as defined in mylist without any errors. My code:
# coding= mylist = [u'Glück', u'Spaß', u'Ähre',] print(mylist) Traceback (most recent call last): File "/Users/abc/test.py", line 4, in print(mylist) UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 4: ordinal not in range(128) mylist = [u'Glück', u'Spaß', u'Ähre',] for w in mylist: print(w.encode("latin-1")) LANG="de_AT.UTF-8" LC_COLLATE="de_AT.UTF-8" LC_CTYPE="de_AT.UTF-8" LC_MESSAGES="de_AT.UTF-8" LC_MONETARY="de_AT.UTF-8" LC_NUMERIC="de_AT.UTF-8" LC_TIME="de_AT.UTF-8" LC_ALL= Python 3.5.1 (default, Jan 22 2016, 08:54:32) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.getdefaultencoding() 'utf-8' Works fine on my python 3.4 windows version: mylist = [u’Glück’, u’Spaß’, u’Ähre’,] for w in mylist: print(w) output: Glück Spaß Ähre so no problem here. Note that if you use encode you get a bytes type, not that you want.
3 Answers 3
Try running your script with explicitly defined PYTHONIOENCODING environment variable:
PYTHONIOENCODING=utf-8 python3 script.py Cool example, the fact that it works and it’s different from other solutions to this issue makes this worthwhile.
Those character are often used in examples to indicate where the encoding name goes, but the literal characters < and >should not be included in your file.
For that to work, your file must be encoded using latin-1. If your file is actually encoded using utf-8, the encoding line should be
For example, when I run this script (saved as a file with latin-1 encoding):
# coding=latin-1 mylist = [u'Glück', u'Spaß', u'Ähre',] print(mylist) for w in mylist: print(w.encode("latin-1")) I get this output (with no errors):
['Glück', 'Spaß', 'Ähre'] b'Gl\xfcck' b'Spa\xdf' b'\xc4hre' That output looks correct. For example, the latin-1 encoding of ü is ‘\xfc’ .
I used my editor to save the file with latin-1 encoding. The contents of the file in hexadecimal are:
$ hexdump -C codec-question.py 00000000 23 20 63 6f 64 69 6e 67 3d 6c 61 74 69 6e 2d 31 |# coding=latin-1| 00000010 0a 0a 6d 79 6c 69 73 74 20 3d 20 5b 75 27 47 6c |..mylist = [u'Gl| 00000020 fc 63 6b 27 2c 20 75 27 53 70 61 df 27 2c 20 75 |.ck', u'Spa.', u| 00000030 27 c4 68 72 65 27 2c 5d 0a 70 72 69 6e 74 28 6d |'.hre',].print(m| 00000040 79 6c 69 73 74 29 0a 0a 66 6f 72 20 77 20 69 6e |ylist)..for w in| 00000050 20 6d 79 6c 69 73 74 3a 0a 20 20 20 20 70 72 69 | mylist:. pri| 00000060 6e 74 28 77 2e 65 6e 63 6f 64 65 28 22 6c 61 74 |nt(w.encode("lat| 00000070 69 6e 2d 31 22 29 29 0a |in-1")).| 00000078 Note that the first byte (represented in hexadecimal) in the third line (i.e. the character at position 0x20) is fc . That is the latin-1 encoding of ü. If the file was encoded using utf-8, the character ü would be represented using two bytes, c3 bc .
UnicodeEncodeError: ‘ascii’ codec can’t encode character in print function
My company is using a database and I am writing a script that interacts with that database. There is already an script for putting the query on database and based on the query that script will return results from database. I am working on unix environment and I am using that script in my script for getting some data from database and I am redirecting the result from the query to a file. Now when I try to read this file then I am getting an error saying-
UnicodeEncodeError: 'ascii' codec can't encode character '\u2013' in position 9741: ordinal not in range(128) I know that python is not able to read file because of the encoding of the file. The encoding of the file is not ascii that’s why the error is coming. I tried checking the encoding of the file and tried reading the file with its own encoding. The code that I am using is-
os.system("Query.pl \"select title from bug where (ste='KGF-A' AND ( status = 'Not_Approved')) \">patchlet.txt") encoding_dict3=<> encoding_dict3=chardet.detect(open("patchlet.txt", "rb").read()) print(encoding_dict3) # Open the patchlet.txt file for storing the last part of titles for latest ACF in a list with codecs.open("patchlet.txt",encoding='<>'.format(encoding_dict3['encoding'])) as csvFile readCSV = csv.reader(csvFile,delimiter=":") for row in readCSV: if len(row)!=0: if len(row) > 1: j=len(row)-1 patchlets_in_latest.append(row[j]) elif len(row) ==1: patchlets_in_latest.append(row[0]) patchlets_in_latest_list=[] # calling the strip_list_noempty function for removing newline and whitespace characters patchlets_in_latest_list=strip_list_noempty(patchlets_in_latest) # coverting list of titles in set to remove any duplicate entry if present patchlets_in_latest_set= set(patchlets_in_latest_list) # Finding duplicate entries in list duplicates_in_latest=[k for k,v in Counter(patchlets_in_latest_list).items() if v>1] # Printing imp info for logs print("list of titles of patchlets in latest list are : ") for i in patchlets_in_latest_list: **print(str(i))** print("No of patchlets in latest list are : <>".format(str(len(patchlets_in_latest_list)))) Where Query.pl is the perl script that is written to bring in the result of query from database.The encoding that I am getting for «patchlet.txt» (the file used for storing result from HSD) is:
Even when I have provided the same encoding for reading the file, then also I am getting the error. Please help me in resolving this error. EDIT: I am using python3.6 EDIT2: While outputting the result I am getting the error and there is one line in the file which is having some unknown character. The line looks like:
I am using gvim and in gvim the «vtrace» looks like «~Vvtrace» . Then I checked on database manually for this character and the character is «–» which is according to my keyboard is neither hyphen nor underscore.These kinds of characters are creating the problem. Also I am working on linux environment. EDIT 3: I have added more code that can help in tracing the error. Also I have highlighted a «print» statement (print(str(i))) where I am getting the error.