TensorFlow: use gfile.FastGfile() method can't not read a file with its path include Chinese characters - unicode

I want to read use gfile.FastGFile(image_path, 'rb').read() to read a picture and use it as the input of my project, and I use the directory name as the lable of these pictures which are include in the directory, when the directory name is in English, my code works fine, but when the directory name is in Chinese, it throws this Error:
Traceback (most recent call last):
File "F:/pythonWS/imageFilter/jpegFileJudge.py", line 27, in <module>
image_data = gfile.FastGFile(image_path, 'rb').read()
File "C:\Program Files\Python35\lib\site-
packages\tensorflow\python\lib\io\file_io.py", line 106, in read
self._preread_check()
File "C:\Program Files\Python35\lib\site-
packages\tensorflow\python\lib\io\file_io.py", line 73, in _preread_check
compat.as_bytes(self.__name), 1024 * 512, status)
File "C:\Program Files\Python35\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Program Files\Python35\lib\site-
packages\tensorflow\python\framework\errors_impl.py", line 466, in
raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile
failed to Create/Open: F:\vsWorkspace\pics\test\三宝鸟
\0ff41bd5ad6eddc403fa02d13bdbb6fd526633fe.jpg :
ϵͳ\udcd5Ҳ\udcbb\udcb5\udcbdָ\udcb6\udca8\udcb5\udcc4\udcceļ\udcfe\udca1\udca3
my test code is :
# -*- coding: utf-8 -*-
import glob
import os
import random
import numpy as np
import tensorflow as tf
from tensorflow.python.platform import gfile
image_folder='F:/vsWorkspace/pics/test'
os.chdir(image_folder)
count=0
for each in os.listdir(image_folder):
each=os.path.abspath(each)
os.chdir(each)
for image_path in os.listdir(each):
image_path = os.path.abspath(image_path)
print(image_path)
image_data = gfile.FastGFile(image_path, 'rb').read()
count += 1
os.chdir(image_folder)
My envirorment is Windows7 x64, python 3.5.3 and TensorFlow 1.0, How can I solve this problem?
By the way,I have to use Chinese directories' name use my pictures lables.

Related

python AttributeError when attempting to save excel chart using PIL

I am trying to save an chart from excel into a file, which I want to use later in a powerpoint presentation, but the code I am running keeps on coming up with
"AttributeError: 'NoneType' object has no attribute 'save'" .
Have been looking around google/stackoverflow but none of the suggestions I can find actually help, I keep on getting the error.
The code I am trying is below,
import win32com.client
import PIL
folder_path = r'C:/temp/Monthly_Graphs.xlsm'
xlApp = win32com.client.DispatchEx('Excel.Application')
wb = xlApp.Workbooks.Open(folder_path)
xlApp = win32com.client.DispatchEx('Excel.Application')
wb = xlApp.Workbooks.Open(folder_path)
wb.Sheets('Sheet1').Shapes('Sheet1_Pie_Chart').CopyPicture()
pie_image = PIL.ImageGrab.grabclipboard()
pie_image.savefig(r'C:/temp/pie_test.bmp','BMP')
the traceback is below
Traceback (most recent call last):
File "<ipython-input-12-b8e52c17e4d1>", line 1, in <module>
runfile('C:/python/stackoverflow_1.py', wdir='C:/python')
File "C:\Users\xxxxxxx\AppData\Local\conda\conda\envs\py64bit\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "C:\Users\xxxxxxx\AppData\Local\conda\conda\envs\py64bit\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/python/stackoverflow_1.py", line 26, in <module>
pie_image.savefig(r'C:/temp/pie_test.bmp','BMP')
AttributeError: 'NoneType' object has no attribute 'savefig'
Managed to get it to work by referring to the below Q and setting the format of the CopyPicture line. Issue seems to be that excel default copy of the image is not in a format that PIL understands
Python Export Excel Sheet Range as Image
import win32com.client
from PIL import ImageGrab
import win32clipboard as clip
folder_path = r'C:/temp/Monthly_Graphs.xlsm'
xlApp = win32com.client.DispatchEx('Excel.Application')
wb = xlApp.Workbooks.Open(folder_path)
xlApp = win32com.client.DispatchEx('Excel.Application')
wb = xlApp.Workbooks.Open(folder_path)
wb.Sheets('Sheet1').Shapes('Sheet1_Pie_Chart').CopyPicture(Format=clip.CF_BITMAP)
pie_image = ImageGrab.grabclipboard()
pie_image.save(r'C:/temp/pie_test.bmp','BMP')

PyPDF2.PdfFileReader hangs indefinitely

I'm trying to read this pdf file (https://www.accessdata.fda.gov/cdrh_docs/pdf14/K141693.pdf) and am following these suggestions from SO
Opening pdf urls with pyPdf
I have actually downloaded the file locally and am running the following code
import PyPDF2
pdf_file = open("K141693.pdf")
pdf_read = PyPDF2.PdfFileReader(pdf_file)
but my code hangs indefinitely. I'm running Python 2.7 and here is the stacktrace.
Traceback (most recent call last):
File "", line 1, in
runfile('C:/PoC/pdf_reader.py', wdir='C:/PoC')
File
"C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py",
line 880, in runfile
execfile(filename, namespace)
File
"C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py",
line 87, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "C:/PoC/pdf_reader.py", line 13, in
pdf_read = PyPDF2.PdfFileReader(pdf_file)
File "C:\ProgramData\Anaconda2\lib\site-packages\PyPDF2\pdf.py",
line 1084, in init
self.read(stream)
File "C:\ProgramData\Anaconda2\lib\site-packages\PyPDF2\pdf.py",
line 1697, in read
line = self.readNextEndLine(stream)
File "C:\ProgramData\Anaconda2\lib\site-packages\PyPDF2\pdf.py",
line 1938, in readNextEndLine
x = stream.read(1)
KeyboardInterrupt
I came across another post here PyPDF2 hangs on processing but that too doesn't have a response.
You need to parse the file in binary ('rb') mode. (This works in Python 3:)
import PyPDF2
pdf_file = open("K141693.pdf", "rb")
read_pdf = PyPDF2.PdfFileReader(pdf_file)

PyDev in Debug mode: NameError: dict_pop when importing pandas or xarray

I'm running code to read data from a NetCDF file from a URL with xarray. When I run this code in Eclipse/PyDev in debug mode I am seeing errors that don't happen when launching as a normal Python run or from the command line or as cells of a Jupyter notebook.
I get the following error when I include an import xarray in my code:
NameError: name 'dict_pop' is not defined
Full stack trace looks like this:
pydev debugger: starting (pid: 4004)
URL: https://www.ncei.noaa.gov/data/nclimgrid/nclimgrid_prcp.nc
Traceback (most recent call last):
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 741, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:15515)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 254, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.do_wait_suspend (_pydevd_bundle/pydevd_cython_win32_35_64.c:5631)
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 764, in do_wait_suspend
self._activate_mpl_if_needed()
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 415, in _activate_mpl_if_needed
activate_function = dict_pop(self.mpl_modules_for_patching, module)
NameError: name 'dict_pop' is not defined
2017-08-16 15:25:06 ERROR Failed to complete ERROR
Traceback (most recent call last):
File "C:\home\git\indices_github\process_xarray.py", line 27, in <module>
ncdata = url.read()
File "C:\home\git\indices_github\process_xarray.py", line 27, in <module>
ncdata = url.read()
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 982, in _pydevd_bundle.pydevd_cython_win32_35_64.SafeCallWrapper.__call__ (_pydevd_bundle/pydevd_cython_win32_35_64.c:19346)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 498, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:18639)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 750, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:15669)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 741, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:15515)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 254, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.do_wait_suspend (_pydevd_bundle/pydevd_cython_win32_35_64.c:5631)
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 764, in do_wait_suspend
self._activate_mpl_if_needed()
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 415, in _activate_mpl_if_needed
activate_function = dict_pop(self.mpl_modules_for_patching, module)
NameError: name 'dict_pop' is not defined
Traceback (most recent call last):
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 1621, in <module>
main()
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 1615, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 1022, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:\home\git\indices_github\process_xarray.py", line 27, in <module>
ncdata = url.read()
File "C:\home\git\indices_github\process_xarray.py", line 27, in <module>
ncdata = url.read()
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 982, in _pydevd_bundle.pydevd_cython_win32_35_64.SafeCallWrapper.__call__ (_pydevd_bundle/pydevd_cython_win32_35_64.c:19346)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 498, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:18639)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 750, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:15669)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 741, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_35_64.c:15515)
File "_pydevd_bundle\pydevd_cython_win32_35_64.pyx", line 254, in _pydevd_bundle.pydevd_cython_win32_35_64.PyDBFrame.do_wait_suspend (_pydevd_bundle/pydevd_cython_win32_35_64.c:5631)
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 764, in do_wait_suspend
self._activate_mpl_if_needed()
File "C:\User Programs\eclipse\neon\plugins\org.python.pydev_5.9.0.201708101613\pysrc\pydevd.py", line 415, in _activate_mpl_if_needed
activate_function = dict_pop(self.mpl_modules_for_patching, module)
NameError: name 'dict_pop' is not defined
I'm using recent/latest versions of xarray and pandas installed on Python 3.5.3 (Anaconda).
I get the same error with an 'import pandas' statement as well, so this may be pandas related instead since xarray depends upon pandas internally.
The packages are installed in Anaconda as shown below:
$ conda list pandas
# packages in environment at C:\home\Anaconda3:
#
pandas 0.20.3 py35_1 conda-forge
$ conda list xarray
# packages in environment at C:\home\Anaconda3:
#
xarray 0.9.6 py35_0 conda-forge
Again I can run this code with the imports of pandas and xarray with no issue from the command line and/or within a Jupyter notebook, this only happens when I launch the code in Eclipse/PyDev in debug mode (regular Python run works as expected).
Example Python code I'm using to test this:
import sys
import xarray as xr
if __name__ == '__main__':
try:
# get the command line arguments
input_netcdf_url = sys.argv[1]
ds = xr.open_dataset(input_netcdf_url)
except:
raise
What can I try next?
Upgrade to 5.9.2... this was one of the critical issues fixed witch prompted the new release.

using boto3 in a python3 virtual env in AWS Lambda

I am trying to use Python3.4 and boto3 to walk an S3 bucket and publish some file locations to an RDS instance. The part of this effort I am having trouble with is when using boto3. My lambda function looks like the following:
import subprocess
def lambda_handler(event, context):
args = ("venv/bin/python3.4", "run.py")
popen = subprocess.Popen(args, stdout=subprocess.PIPE)
popen.wait()
output = popen.stdout.read()
print(output)
and, in my run.py file I have some lines:
import boto3
s3c = boto3.client('s3')
which cause an exception. The run.py file is not relevant for this question however, so in order make this post more concise, I've found that the cause of this error is generated with executing the lambda function:
import subprocess
def lambda_handler(event, context):
args = ("python3.4", "-c", "import boto3; print(boto3.client('s3'))")
popen = subprocess.Popen(args, stdout=subprocess.PIPE)
popen.wait()
output = popen.stdout.read()
print(output)
My logstream reports the error:
Event Data
START RequestId: 2b65421a-664d-11e6-81db-974c7c09d283 Version: $LATEST
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/var/runtime/boto3/__init__.py", line 79, in client
return _get_default_session().client(*args, **kwargs)
File "/var/runtime/boto3/session.py", line 250, in client
aws_session_token=aws_session_token, config=config)
File "/var/runtime/botocore/session.py", line 818, in create_client
client_config=config, api_version=api_version)
File "/var/runtime/botocore/client.py", line 63, in create_client
cls = self._create_client_class(service_name, service_model)
File "/var/runtime/botocore/client.py", line 85, in _create_client_class
base_classes=bases)
File "/var/runtime/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/var/runtime/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/var/runtime/boto3/utils.py", line 61, in _handler
module = import_module(module)
File "/var/runtime/boto3/utils.py", line 52, in import_module
__import__(name)
File "/var/runtime/boto3/s3/inject.py", line 13, in <module>
from boto3.s3.transfer import S3Transfer
File "/var/runtime/boto3/s3/transfer.py", line 135, in <module>
from concurrent import futures
File "/var/runtime/concurrent/futures/__init__.py", line 8, in <module>
from concurrent.futures._base import (FIRST_COMPLETED,
File "/var/runtime/concurrent/futures/_base.py", line 357
raise type(self._exception), self._exception, self._traceback
^
SyntaxError: invalid syntax
END RequestId: 2b65421a-664d-11e6-81db-974c7c09d283
REPORT RequestId: 2b65421a-664d-11e6-81db-974c7c09d283 Duration: 2673.45 ms Billed Duration: 2700 ms Memory Size: 1024 MB Max Memory Used: 61 MB
I need to use boto3 downstream of run.py. Any ideas on how to resolve this are much appreciated. Thanks!

Cannot ennable plugin in gedit

I can see many plugins in "Plugin" tab, but I cannot enable or disable any one of them. If I click to enable/disable, Gedit will disappear right away!!: Here is the output from terminal:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site.py", line 548, in <module>
main()
File "/usr/local/lib/python2.7/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File "/usr/local/lib/python2.7/site.py", line 266, in addusersitepackages
user_site = getusersitepackages()
File "/usr/local/lib/python2.7/site.py", line 241, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/local/lib/python2.7/site.py", line 231, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/local/lib/python2.7/sysconfig.py", line 516, in get_config_var
return get_config_vars().get(name)
File "/usr/local/lib/python2.7/sysconfig.py", line 449, in get_config_vars
import re
File "/usr/local/lib/python2.7/re.py", line 105, in <module>
import sre_compile
File "/usr/local/lib/python2.7/sre_compile.py", line 14, in <module>
import sre_parse
File "/usr/local/lib/python2.7/sre_parse.py", line 17, in <module>
from sre_constants import *
File "/usr/local/lib/python2.7/sre_constants.py", line 18, in <module>
from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT
Please help me! many thanks.
Ubuntu Release 12.04 (precise) 32-bit
Kernel Linux 3.5.0-39-generic
GNOME 3.4.2
I think the problem is because you use gnome 3 and gedit 2.