Unable to parse email (.msg) in python 3.6 - email

I have set of .msg files stored in E:/ drive that I have to read and extract some information from it. For that i am using the below code in Python 3.6
from email.parser import Parser
p = Parser()
headers = p.parse(open('E:/Ratan/msg_files/Test1.msg', encoding='Latin-1'))
print('To: %s' % headers['To'])
print('From: %s' % headers['From'])
print('Subject: %s' % headers['subject'])
In the output I am getting as below.
To: None
From: None
Subject: None
I am not getting the actual values in To, FROM and subject fields.
Any thoughts why it is not printing the actual values?
Please download my sample msg file from this link:
drive.google.com/file/d/1pwWWG3BgsMKwRr0WmP8GqzG3WX4GmEy6/vi‌​ew

Here is a demonstration of how to use some of python's standard email libraries.
You didn't show us your input file in the question, and the g-drive URL is a deadlink.
The code below looks just like yours and works fine, so I don't know what is odd about your environment, modulo some Windows 'rb' binary open nonsense, CRLFs, or the Latin1 encoding.
I threw in .upper() but it does nothing beyond showing that the API is case insensitive.
#! /usr/bin/env python3
from email.parser import Parser
from pathlib import Path
import mailbox
def extract_messages(maildir, mbox_file, k=2, verbose=False):
for n, message in enumerate(mailbox.mbox(mbox_file)):
with open(maildir / f'{n}.txt', 'w') as fout:
fout.write(str(message))
hdrs = 'From Date Subject In-Reply-To References Message-ID'.split()
p = Parser()
for i in range(min(k, n)):
with open(maildir / f'{i}.txt') as fin:
msg = p.parse(fin)
print([len(msg[hdr.upper()] or '')
for hdr in hdrs])
for k, v in msg.items():
print(k, v)
print('')
if verbose:
print(msg.get_payload())
if __name__ == '__main__':
# from https://mail.python.org/pipermail/python-dev/
maildir = Path('/tmp/py-dev/')
extract_messages(maildir, maildir / '2018-January.txt')

Related

How to efficiently rename many files or partially select the names of these files during import?

how to rename the files efficiently by the number in the name (see picture)? I did not succeed with Windows PowerToys and I dont wana click each file and rename to the number (e.g. 290)
or how to read the files in this order and define a name? If I try it with a script (see below) the following output occurs:
[![ValueError: invalid literal for int() with base 10: '211001_164357_P_Scripted_Powermeasurement_Wavelength_automatic_Powermeter1_0'][1]][1]
or how to select only the numbers (290 to 230 - see picture) within the name when reading?
Script:
#import libraries
import pandas as pd
import os
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path
data_location = r'C:\Users\...\Characterization_OPO\Data'
data_folder = Path(data_location)
data = {}
allist = list(data_folder.glob('*'))
for i, file in enumerate(allist):
file = str(file)
file_name = file.split('\\')[-1]
wavelength = int(file_name.split('.')[0])
tmp = pd.read_csv(file, skiprows=20, skipfooter=59, index_col="PixelNo")
data[f'{wavelength} nm'] = tmp;
#data.plot(x='Wavelength',y='CCD_1', label=f"{wavelength} nm")
Picture:
I removed all words with windows power rename and than took the last three digits:
for i, file in enumerate(allist):
file = str(file)
file_name = file.split('\\')[-1]
wavelength = int(file_name.split('.')[0])
tmp = pd.read_csv(file, skiprows=26, skipfooter=5)
data[f'{wavelength % 1000} nm'] = tmp;
#data.plot(x='Wavelength',y='CCD_1', label=f"{wavelength} nm")

Python 3.7.3 variables with numbers in while loop conditionals

So I'm trying to learn some python3 with some simple code as stated below. The point of the code is to have a loop that runs until one of the files exists and contains data. For some reason i'm getting an error running this, saying that the variable has an invalid syntax just as if numbers in variables are illegal (which they arent?):
$ python3 test.py
File "test.py", line 14
While file1==False and file2==False and file3==False:
^
SyntaxError: invalid syntax
Code:
import os
filePath1 = '/some/path'
filePath2 = '/some/path'
filePath3 = '/some/path'
file1 = False
file2 = False
file3 = False
While file1==False and file2==False and file3==False:
if os.path.exists(filePath1):
with open(filePath1,'r') as f:
try:
file1 = f.read()
except:
print("No file data.")
if os.path.exists(filePath2):
with open(filePath2,'r') as f:
try:
file2 = f.read()
except:
print("No file data.")
if os.path.exists(filePath3):
with open(filePath3,'r') as f:
try:
file3 = f.read()
except:
print("No file data.")
I don't understand this because:
>>>file1=False
>>>file2=False
>>>file1==False and file2==False
True
I'd be grateful for any help
invalid syntax is caused by your capitalized While keyword, which python doesn't recognize. Use the reserved keyword while, small letters only.

Preview app doesn't open since I installed MACOS Catalina

import PIL
img = PIL.Image.new("RGB", (100,100))
img.show()
The error message:
FSPathMakeRef(/Applications/Preview.app) failed with error -43.
Following from Sean True's answer, an even quicker but temporary fix is to simply make a symbolic link to Preview.app in the old location. In the terminal run
ln -s /System/Applications/Preview.app /Applications/Preview.app
This fixed the problem for me.
There's an official fix in github for Pillow 7, but I'm still on 6.
This appears to be a PIL ImageShow issue, with the PIL MacViewer using /Applications/Preview.app as an absolute path to the OSX Preview app.
It's not there in Catalina. I did a quick hack to ImageShow.py changing /Applications/Preview.app to just Preview.app and the issue went away. That might or might not still work on pre-Catalina OSX, but I don't have an easy way to test.
It has apparently moved to /System/Applications/Preview.app so a quick check at run time would probably cover both cases.
elif sys.platform == "darwin":
class MacViewer(Viewer):
format = "PNG"
options = {'compress_level': 1}
preview_locations = ["/System/Applications/Preview.app","/Applications/Preview.app"]
preview_location = None
def get_preview_application(self):
if self.preview_location is None:
for pl in self.preview_locations:
if os.path.exists(pl):
self.preview_location = pl
break
if self.preview_location is None:
raise RuntimeError("Can't find Preview.app in %s" % self.preview_locations)
return self.preview_location
def get_command(self, file, **options):
# on darwin open returns immediately resulting in the temp
# file removal while app is opening
pa = self.get_preview_application()
command = "open -a %s" % pa
command = "(%s %s; sleep 20; rm -f %s)&" % (command, quote(file),
quote(file))
return command
def show_file(self, file, **options):
"""Display given file"""
pa = self.get_preview_application()
fd, path = tempfile.mkstemp()
with os.fdopen(fd, 'w') as f:
f.write(file)
with open(path, "r") as f:
subprocess.Popen([
'im=$(cat);'
'open %s $im;'
'sleep 20;'
'rm -f $im' % pa
], shell=True, stdin=f)
os.remove(path)
return 1

python OverflowError: 34, raised for no apparent reason

I've written a python script to evaluate a physical quantity and, for some reason, python decided to raise an OverflowError for no justified reason. Here's the script
import numpy as np
from math import sqrt,cos, log, pi
from scipy import integrate as sciint
from scipy import optimize as sciopt
rt=np.inf
ymin=cos(np.radians(0.5))
def func(u,y, D, rt):
return (1.+u)**(-4)/u/sqrt(u*u-D**2*(1-y*y))
def lim_u(y, D, rt):
return [D*sqrt(1-y*y), rt]
def lim_y(D, rt):
return [ymin,1]
def Jfactor(D,rt,r0,rho0,tmax):
ymin=cos(np.radians(tmax))
Dprime=D/r0
rtprime=rt/r0
Msun2kpc5_GeVcm5 = 4463954.894661358
cst = 4*pi*rho0**2*r0*Msun2kpc5_GeVcm5
res = sciint.nquad(func, ranges=[lim_u, lim_y], args=(Dprime,rtprime),
opts=[{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000},
{'epsabs':1.e-10,'epsrel':1.e-10,'limit':1000}])
return cst*res[0]
def deltaJ(rho0,J,D,rt,r0,tmax):
#return J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.)
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
D=104.
rt=np.inf
tmax=0.5
J = 10.**18
for j,r0 in enumerate(np.logspace(-1.,np.log10(5),10)):
results = sciopt.minimize_scalar(deltaJ,bracket=(7.,9.),args=(J,D,rt,r0,tmax))
print r0,results.x,results.fun
Very quickly: I want scipy.optimize.minimize_scalar to minimize deltaJ but here's (part of) the error message I get
File "test2.py", line 26, in deltaJ
return long(J-Jfactor(D,rt,r0,1.,tmax)*10.**(rho0*2.))
OverflowError: (34, 'Numerical result out of range')
Now, if J = 10**18, Jfactor(D,rt,r0,1.,tmax) is ~ 5, I expect minimize_scalar to easily scan over rho0 to find ~ 8.5. Instead I get this error message.
As you can see, I've even bracketed the range and tried using long(), but nothing helped. Using instead another minimizing method, eg. bounded, gives again a weird result (function value of ~ -7e17)...
Does anybody have an idea to have this working? Thank you

NSLocalizedString managing translations over app versions [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
Here is a scenario:
I write an iPhone app using NSLocalizedString incase I decide to release it in different countries.
I decide to release the App over in France.
The translator takes my Localized.strings and does a great job translating
I update the app, and need some more translating.
I'm using genstrings and it overwrites the good work the translator did, is there a easy way for me to manage my translations over App versions?
Check out this project on GitHub, which provides a python scripts which makes genstrings a little bit smarter.
Since I don't like link-only answers (links may die), I'll also drop here the python script (all credits go to the author of the linked project)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Localize.py - Incremental localization on XCode projects
# João Moreno 2009
# http://joaomoreno.com/
# Modified by Steve Streeting 2010 http://www.stevestreeting.com
# Changes
# - Use .strings files encoded as UTF-8
# This is useful because Mercurial and Git treat UTF-16 as binary and can't
# diff/merge them. For use on iPhone you can run an iconv script during build to
# convert back to UTF-16 (Mac OS X will happily use UTF-8 .strings files).
# - Clean up .old and .new files once we're done
# Modified by Pierre Dulac 2012 http://friendcashapp.com
# Changes
# - use logging instead of print
# Adds
# - MIT Licence
# - the first parameter in the command line to specify the path of *.lproj directories
# - an optional paramter to control the debug level (set to info by default)
# Fixes
# - do not convert a file if it is already in utf-8
# - allow multiline translations generated by genstrings by modifing the re_translation regex
# -
# MIT Licence
#
# Copyright (C) 2012 Pierre Dulac
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
# associated documentation files (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all copies or substantial
# portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
# LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from sys import argv
from codecs import open
from re import compile
from copy import copy
import os
import shutil
import optparse
import logging
logging.getLogger().level = logging.INFO
__version__ = "0.1"
__license__ = "MIT"
USAGE = "%prog [options] <url>"
VERSION = "%prog v" + __version__
re_translation = compile(r'^"((?:[^"]|\\")+)" = "((?:[^"]|\\")+)";(?:\n)?$')
re_comment_single = compile(r'^/\*.*\*/$')
re_comment_start = compile(r'^/\*.*$')
re_comment_end = compile(r'^.*\*/$')
class LocalizedString():
def __init__(self, comments, translation):
self.comments, self.translation = comments, translation
self.key, self.value = re_translation.match(self.translation).groups()
def __unicode__(self):
return u'%s%s\n' % (u''.join(self.comments), self.translation)
class LocalizedFile():
def __init__(self, fname=None, auto_read=False):
self.fname = fname
self.reset()
if auto_read:
self.read_from_file(fname)
def reset(self):
self.strings = []
self.strings_d = {}
def read_from_file(self, fname=None):
self.reset()
fname = self.fname if fname == None else fname
try:
#f = open(fname, encoding='utf_8', mode='r')
f = open(fname, encoding='utf_8', mode='r')
except:
print 'File %s does not exist.' % fname
exit(-1)
try:
line = f.readline()
logging.debug(line)
except:
logging.error("Can't read line for file: %s" % fname)
raise
i = 1
while line:
comments = [line]
if not re_comment_single.match(line):
while line and not re_comment_end.match(line):
line = f.readline()
comments.append(line)
line = f.readline()
i += 1
# handle multi lines
while len(line) > 1 and line[-2] != u';':
line += f.readline()
i += 1
logging.debug("%d %s" % (i, line.rstrip('\n')))
if line and re_translation.match(line):
translation = line
else:
logging.error("Line %d of file '%s' raising the exception: %s" % (i, self.fname, line))
raise Exception('invalid file')
line = f.readline()
i += 1
while line and line == u'\n':
line = f.readline()
i += 1
string = LocalizedString(comments, translation)
self.strings.append(string)
self.strings_d[string.key] = string
f.close()
def save_to_file(self, fname=None):
fname = self.fname if fname == None else fname
try:
f = open(fname, encoding='utf_8', mode='w')
except:
print 'Couldn\'t open file %s.' % fname
exit(-1)
# sort by key
self.strings.sort(key=lambda item: item.key)
for string in self.strings:
f.write(string.__unicode__())
f.close()
def merge_with(self, new):
merged = LocalizedFile()
for string in new.strings:
if self.strings_d.has_key(string.key):
new_string = copy(self.strings_d[string.key])
new_string.comments = string.comments
string = new_string
merged.strings.append(string)
merged.strings_d[string.key] = string
return merged
def update_with(self, new):
for string in new.strings:
if not self.strings_d.has_key(string.key):
self.strings.append(string)
self.strings_d[string.key] = string
def merge(merged_fname, old_fname, new_fname):
try:
old = LocalizedFile(old_fname, auto_read=True)
new = LocalizedFile(new_fname, auto_read=True)
merged = old.merge_with(new)
merged.save_to_file(merged_fname)
except Exception, inst:
logging.error('Error: input files have invalid format.')
raise
STRINGS_FILE = 'Localizable.strings'
def localize(path, excluded_paths):
languages = [os.path.join(path,name) for name in os.listdir(path) if name.endswith('.lproj') and os.path.isdir(os.path.join(path,name))]
print "languages found", languages
for language in languages:
original = merged = language + os.path.sep + STRINGS_FILE
old = original + '.old'
new = original + '.new'
if os.path.isfile(original):
try:
open(original, encoding='utf_8', mode='r').read()
os.rename(original, old)
except:
os.system('iconv -f UTF-16 -t UTF-8 "%s" > "%s"' % (original, old))
# gen
os.system('find %s -name \*.m -not -path "%s" | xargs genstrings -q -o "%s"' % (path, excluded_paths, language))
try:
open(original, encoding='utf_8', mode='r').read()
shutil.copy(original, new)
except:
os.system('iconv -f UTF-16 -t UTF-8 "%s" > "%s"' % (original, new))
# merge
merge(merged, old, new)
logging.info("Job done for language: %s" % language)
else:
os.system('genstrings -q -o "%s" `find %s -name "*.m" -not -path "%s"`' % (language, path, excluded_paths))
os.rename(original, old)
try:
open(old, encoding='utf_8', mode='r').read()
except:
os.system('iconv -f UTF-16 -t UTF-8 "%s" > "%s"' % (old, original))
if os.path.isfile(old):
os.remove(old)
if os.path.isfile(new):
os.remove(new)
def parse_options():
"""parse_options() -> opts, args
Parse any command-line options given returning both
the parsed options and arguments.
"""
parser = optparse.OptionParser(usage=USAGE, version=VERSION)
parser.add_option("-d", "--debug",
action="store_true", default=False, dest="debug",
help="Set to DEBUG the logging level (default to INFO)")
parser.add_option("-p", "--path",
action="store", type="str", default=os.getcwd(), dest="path",
help="Path (relative or absolute) to use for searching for *.lproj directories")
parser.add_option("-e", "--exclude",
action="store", type="str", default=None, dest="excluded_paths",
help="Regex for paths to exclude ex. ``./Folder1/*``")
opts, args = parser.parse_args()
return opts, args
if __name__ == '__main__':
opts, args = parse_options()
if opts.debug:
logging.getLogger().level = logging.DEBUG
if opts.path:
opts.path = os.path.realpath(opts.path)
if opts.excluded_paths:
opts.excluded_paths = os.path.realpath(opts.excluded_paths)
logging.info("Running the script on path %s" % opts.path)
localize(opts.path, opts.excluded_paths)
I use:
http://www.loc-suite.com
To only translate the new parts
I was having a similar issue. I changed a lot of keys for my NSLocalizedString-macros and was frightened that I'd ship the App with missing translations (didn't want to run through the whole App manually and check if everything's there either...).
I tried out the github project that Gabriella Petronella posted but I wasn't really that happy with it, so I wrote my own python module to accomplish what I wanted to do.
(I'm not gonna post the code here, since it's a whole module and not only one script :D)
Here is the couple of options you can chose to go with:
You can use some hand-written solution like the script mentioned above which will not completely rewrite the old files while adding a recently translated strings to them.
You can also create an additional strings.h file which will contain all the strings you do have so you will not need to rewrite them all the time, just in one place. So genstrings is not necessary anymore. However there is a con of using this: the string.h file will be unstructured which is probably not convenient for the big projects.
Thanks to Best practice using NSLocalizedString
// In strings.h
#define YOUR_STRING_KEY NSLocalizedString(#"Cancel", nil)
// Somewhere else in you code
NSLog(#"%#", YOUR_STRING_KEY);
I actually started using a tool called PhraseApp https://phraseapp.com/projects
It's worth looking into if you have to localise an app!