Cannot access FTP directory with CP1250/CP852/UTF-8 encoding - encoding

I am trying to read in some files from the following directory structure:
/jc/06 Önéletrajzok/Profession/Előszűrés sablonok név szerint
But for some strange reason I cannot enter not even in the upper level directories.
I already tried with PHP/Python3.6/Ruby but without much luck. At least with PHP and Python I can CWD() at least until the /jc/06 Önéletrajzok/Profession part.
Here is my python code for reference:
from ftplib import FTP
ftp = FTP('hostname')
ftp.login('username','pwd')
ftp.cwd('jc') # Just for demonstration purposes as step by step
ftp.cwd('06 Önéletrajzok')
ftp.cwd('Profession')
print(ftp.nlst()[2]) # Which gives: 'ElÅ\x91szűrés sablonok név szerint
# But when I am trying:
ftp.cwd('ElÅ\x91szűrés sablonok név szerint')
# Or either:
ftp.cwd('Előszűrés sablonok név szerint')
# It gives:
# UnicodeEncodeError: 'latin-1' codec can't encode character '\u0151' in position 6: ordinal not in range(256)
# So I am trying encoding CP1250 or CP852 (for Hungarian)
dir = 'Előszűrés sablonok név szerint'.encode('cp852') # which gives: b'El\x8bsz\xfbr\x82s sablonok n\x82v szerint'
ftp.cwd(dir.decode('utf-8'))
# and it gives the following error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 2: invalid start byte
So I am starting to give up on this one, I don't know how to access those files. The directory structure was created with Windows laptops accessing a Synology File server.
I have already tried with ftp.encoding = "utf-8" too.
Any ideas?

Related

How to specify encoding in Alembic migrations

I am working to add Alembic migration to a legacy project. I am hoping to execute some raw sql, including some insert statements with unicode that I want to be encoded with UTF-8, but am getting a UnicodeDecodeError. To reproduce this error, I created this example:
def upgrade():
op.execute("SELECT '𝔥𝔢𝔩𝔩𝔬'")
When I run this migration, I get:
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1263, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 324, in _execute_on_connection
self, multiparams, params, execution_options
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1462, in _execute_clauseelement
cache_hit=cache_hit,
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1678, in _execute_context
e, util.text_type(statement), parameters, None, None
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 8: ordinal not in range(128)
I confirmed that my Postgres database encoding is set to UTF-8. I also tried to work out whether execution_options parameters exist to set the encoding, but I was not able to discern this. I also tried abandoning op.execute() and tried creating my own engine instance using sa.create_engine(url, encoding='utf-8'), which mysteriously still gave me an encoding error and was wanting to use ascii.
Which Alembic, SQLAlchemy or Psycopg2 subsystem is expecting ascii, and is there a way to change that expectation either in Alembic's configuration or for a specific migration?
I'm happy to dig into the internal APIs and use a hacky solution, but would hope there is a straightforward way to do this that I have just not encountered in a couple hours searching the documentation.
Note: this project uses Python 2.7.16, and despite my efforts I do not have authorization to port it to Python3 yet.

how to truly turn off LaTex output in Doxygen

My group is using Doxygen 1.8.5 on RHEL7 to generate HTML-only documentation for a large C++ project. We only want HTML documentation and do not desire any other output format. The project's Doxygen configuration file (Doxyfile) carries the following settings which differ from the default (among others, of course):
OUTPUT_DIRECTORY="../Docs"
GENERATE_HTML=YES
GENERATE_LATEX=NO
When we run Doxygen with this config file, towards the end of processing, errors start coming to screen from LaTeX components, and processing is held up until the user hits enter a bunch of times to get past these errors, e.g.
...
sh: epstopdf: command not found
error: Problems running epstopdf. Check your TeX installation!
Generating graph info page...
Generating directory documentation...
</home/abilich/Src/GNSS_Software/Lib/LibgpsC++/>:3: warning: Found unknown command `\reference'
</home/abilich/Src/GNSS_Software/Lib/LibgpsC++/>:2: warning: Found unknown command `\reference'
</home/abilich/Src/GNSS_Software/Lib/LibgpsC++/>:2: warning: Found unknown command `\reference'
Generating bitmaps for formulas in HTML...
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)
restricted \write18 enabled.
entering extended mode
(./_formulas.tex
LaTeX2e <2011/06/27>
Babel <v3.8m> and hyphenation patterns for english, dumylang, nohyphenation, lo
aded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/epsfig.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/dvips.def))))
No file _formulas.aux.
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]
! Undefined control sequence.
l.53 \[ \Vert v - w \Vert \leqslant
p\,\min(\Vert v\Vert, \Vert w\Vert). \]
?
...
! Undefined control sequence.
l.533 $ \vert singular value \vert \leqslant
threshold \times \vert max sing...
?
[177] [178] (./_formulas.aux) )
Output written on _formulas.dvi (178 pages, 20576 bytes).
Transcript written on _formulas.log.
error: Problems running latex. Check your installation or look for typos in _formulas.tex and check _formulas.log!
Generating image form_0.png for formula
sh: gs: command not found
error: Problem running ghostscript gs -q -g60x50 -r384x384x -sDEVICE=ppmraw -sOutputFile=_form0.pnm -dNOPAUSE -dBATCH -- _form0.ps. Check your installation!
Generating index page...
/home/abilich/Src/GNSS_Software/Doxy/mainpage.h:15: warning: image file GNSSSoftwareTimeline.png is not found in IMAGE_PATH: assuming external image.
Generating page index...
Generating module index...
Generating namespace index...
Generating namespace member index...
I want to create a Doxyfile that runs without user intervention and does not throw LaTeX errors. Where have I gone astray?
Thanks for #albert's comment, actually the newest version of doxygen did support to turn latex generation off by setting GENERATE_LATEX=NO:
#---------------------------------------------------------------------------
# Configuration options related to the LaTeX output
#---------------------------------------------------------------------------
# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
# The default value is: YES.
GENERATE_LATEX = NO
The parameter USE_PDFLATEX in your config file should set to NO (the default value is YES)
# If the USE_PDFLATEX tag is set to YES, doxygen will use the engine as
# specified with LATEX_CMD_NAME to generate the PDF file directly from the LaTeX
# files. Set this option to YES, to get a higher quality PDF documentation.
#
# See also section LATEX_CMD_NAME for selecting the engine.
# The default value is: YES.
# This tag requires that the tag GENERATE_LATEX is set to YES.
USE_PDFLATEX = NO

unoconv fails to save in my specified directory

I am using unoconv to convert an ods spreadsheet to a csv file.
Here is the command:
unoconv -vvv --doctype=spreadsheet --format=csv --output= ~/Dropbox
/mariners_site/textFiles/expenses.csv ~/Dropbox/Aldeburgh/expenses
/expenses.ods
It saves the output file in the same directory as the source file, not in the specified directory. The error message is:
Output file: /home/richard/Dropbox/mariners_site/textFiles/expenses.csv
unoconv: UnoException during export phase:
Unable to store document to file:///home/richard/Dropbox/mariners_site
/textFiles/expenses.csv (ErrCode 19468)
I'm sure that this worked initially, but it has since stopped.
I have checked for permissions and they are identical for both directories.
I translated ErrCode 19468 for you and it boils down to meaning ERRCODE_SFX_DOCUMENTREADONLY.
You can find more information about the specific meaning of LibreOffice ErrCode numbers from the unoconv documentation at: https://github.com/dagwieers/unoconv/blob/master/doc/errcode.adoc
The clue here is that you have a whitespace-character between --output= and the filename (--output= ~/Dropbox
/mariners_site/textFiles/expenses.csv) and because of that unoconv gets an empty output value (which means the current directory) and is given 2 files. And that explains why you get this specific error IMO

perl pdftk functionality not working , getting error

I was testing the functionality of PDF::Tk by installing in cpan module and installed the pdftk binary file and the path to variable and tried running source code.
source code:
use PDF::Tk;
my $doc = PDF::Tk->new( pdftk => '/apps/free/pdftk/' );
$doc->call_pdftk( 'input.pdf', 'outPDF.pdf', 'cat', '1-14' );
getting error as below:
pdftk input.pdf cat 1-14 releasenote.pdf failed: -1 at
/usr/lib/perl5/site_perl/5.10.0/PDF/Tk.pm line 73.
please guide me in resolving it.
Seems you are passing the wrong argument to the constructor of PDF::Tk. Have a look here.
You're supposed to pass a hash, with the key pdftk, and this should be the path of the executable, not a directory. As you can see here, this will be executed via system, so of course, executing a directory does not work.
To clarify, you should be using:
my $doc=PDF::Tk->new(pdftk => '/path/to/pdftk/executable');
If your pdftk executable is /usr/bin/pdftk, then you do not have to pass it at all as this is the default.
for testing the encryption function (by default 128 bit encryption) , i created a pdf file 'apps.pdf' with password protected 'abcd' as password.
source code 1:
use PDF::Tk;
my $doc=PDF::Tk->new(pdftk=>'/apps/free/pdftk/1.44/bin/pdftk');
$doc->call_pdftk('apps.pdf', '1.128.pdf', 'owner_pw', 'abcd');
getting error:
Error: Unexpected command-line data:
owner_pw
where we were expecting an input PDF filename,
operation (e.g. "cat") or "input_pw". Exiting.
Errors encountered. No output created.
Done. Input errors, so no output created.
pdftk apps.pdf owner_pw abcd 1.128.pdf failed: 256 at /usr/lib/perl5/site_perl/5.10.0/PDF/Tk.pm line 73.
note: created a new pdf 'apps.pdf' with Document Open Password as 'abcd' and permission Password as 'abcd123'.
Please let me know how to resolve it.

Python 3 doesn't read unicode file on a new server

My webpages are served by a script that dynamically imports a bunch of files with
try:
with open (filename, 'r') as f:
exec(f.read())
except IOError: pass
(actually, can you suggest a better method of importing a file? I'm sure there is one.)
Sometimes the files have strings in different languages, like
# contents of language.ru
title = "Название"
Those were all saved as UTF-8 files. Python has no problem running the script in command line or serving a page from my MacBook:
OK: [server command line] python3.0 page.py /index.ru
OK: http://whitebox.local/index.ru
but it throws an error when trying to serve a page from a server we just moved to:
157 try:
158 with open (filename, 'r') as f:
159 exec(f.read())
160 except IOError: pass
161
/usr/local/lib/python3.0/io.py in read(self=, n=-1)
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 627: ordinal not in range(128)
All the files were copied from my laptop where they were perfectly served by Apache. What is the reason?
Update: I found out the default encoding for open() is platform-dependent so it was utf8 on my laptop and ascii on server. I wonder if there is a per-program function to set it in Python 3 (sys.setdefaultencoding is used in site module and then deleted from the namespace).
Use open(filename, 'r', encoding='utf8').
See Python 3 docs for open.
Use codecs library, I'm using python 2.6.6 and I do not use the usual open with encoding argument:
import codecs
codecs.open('filename','r',encoding='UTF-8')
You can use something like
with open(fname, 'r', encoding="ascii", errors="surrogateescape") as f:
data = f.read()
# make changes to the string 'data'
with open(fname + '.new', 'w',
encoding="ascii", errors="surrogateescape") as f:
f.write(data)
more information is on python unicode documents