Occasionally one of our users will receive an email from a known source, and the characters of the body of the email will be Chinese instead of English. They won't all be Chinese, but a mix of Chinese and some random characters, like this:
"格浴㹬†格慥㹤††䴼呅⁁瑨灴攭畱癩∽潃瑮湥祔数•潣瑮湥㵴琢硥⽴瑨汭※档牡敳㵴瑵ⵦ㘱㸢††洼瑥瑨灴攭畱癩∽潃瑮湥祔数•潣瑮湥㵴琢硥⽴瑨汭※档牡敳㵴卉ⵏ㠸㤵ㄭ㸢††琼瑩敬刾捩敫⁹效潲慭獮䠠獡䐠汥癩牥摥夠畯汆睯牥牏䜠晩㱴琯瑩敬ാ 㰠栯慥㹤†戼摯⁹杢潣潬㵲⌢晦晦晦㸢††琼扡敬眠摩"
It only seems to be happening to one or two users, and it's not every sender - in fact, one of the emails from the sender could be fine, and the next could be like this. Encoding seems to be fine, but we're not sure where else to look. One other thing - we have Barracuda as our email filter. If we view one of the problem emails in Barracuda first, it's English. It seems to be changed to Chinese on the client side.
We have an on prem Exchange 2016 server with Outlook 2016 as the mail client, and the OS is Windows 10. Thanks!
I can tell you what has happened although I cannot tell you why.
I saved your string to a text file. I created a small Excel macro to read that file and display the characters in hexadecimal:
683C 6D74 3E6C 0A0D 2020 683C 6165 3E64 0A0D 2020 2020 4D3C 5445 2041 7468 7074 652D 7571
7669 223D 6F43 746E 6E65 2D74 7954 6570 2022 6F63 746E 6E65 3D74 7422 7865 2F74 7468 6C6D
203B 6863 7261 6573 3D74 7475 2D66 3631 3E22 0A0D 2020 2020 6D3C 7465 2061 7468 7074 652D
7571 7669 223D 6F43 746E 6E65 2D74 7954 6570 2022 6F63 746E 6E65 3D74 7422 7865 2F74 7468
6C6D 203B 6863 7261 6573 3D74 5349 2D4F 3838 3935 312D 3E22 0A0D 2020 2020 743C 7469 656C
523E 6369 656B 2079 6548 6F72 616D 736E 4820 7361 4420 6C65 7669 7265 6465 5920 756F 2072
6C46 776F 7265 2073 724F 4720 6669 3C74 742F 7469 656C 0D3E 200A 3C20 682F 6165 3E64 0A0D
2020 623C 646F 2079 6762 6F63 6F6C 3D72 2322 6666 6666 6666 3E22 0A0D 2020 2020 743C 6261
656C 7720 6469
Each pair of hexadecimal digits represents a valid ASCII character. The fourth character is “0A0D” or “linefeed carriage-return”. This should be “carriage-return linefeed”. Somehow a valid ASCII email body has been interpreted as a little-endian UTF-16 email body. If you split these characters up and reverse them, you get:
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=utf-16">
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Rickey Heromans Has Delivered Your Flowers Or Gift</title>
</head>
<body bgcolor="#ffffff">
<table wid
My knowledge of Html does not extend to knowing the significance of having two charsets defined although it would appear the first has been obeyed. All the other tags (html, head, meta, title, body and table) are lower case so my guess is that the incorrect <META http-equiv="Content-Type" content="text/html; charset=utf-16"> has been added somewhere.
Hope this helps.
Related
MBox files, like Google Gmail export mailbox contains various types of data, among other things, the date of the mail message that interests me. The date of the message is in the format:
DayOfWeek, dd monthname yyyy hh:mm:ss +timezone
Mon, 03 Jun 2019 15:32:25 +0200
I was looking for some ready-made function in Delphi that I could translate a date string into TDateTime. If the whole string is impossible to parse to TDateTime, I can only try parse this part:
dd monthname yyyy hh:mm:ss
03 Jun 2019 15:32:25
but I admit that it is a bit of a hassle in my language to parse the month name. Could I ask for such a function if it exists? If such a ready-made function does not exist, I will make my own (or at least try).
Thank you in advance!
EDIT: SOLUTION
My guess is that this may not be useful to anyone, but if someone will have a similar problem, have a solution here. You can probably write it more elegantly, but it works.
// You can make your own formats here
const months : array[1..12,1..2] of string =
(('-01-',' Jan '),
('-02-',' Feb '),
('-03-',' Mar '),
('-04-',' Apr '),
('-05-',' May '),
('-06-',' Jun '),
('-07-',' Jul '),
('-08-',' Aug '),
('-09-',' Sep '),
('-10-',' Oct '),
('-11-',' Nov '),
('-12-',' Dec '));
function ChangeDateFormat(input: String): String;
var i: Integer;
begin
// day name and timezone is not needed, so we cut it
delete(input,1,Pos(',',input)+1);
delete(input,Pos('+',input)-1, Length(input));
for i := 1 to 12 do
input := StringReplace(input,
months[i,2],months[i,1],
[rfReplaceAll, rfIgnoreCase]);
result := Trim(input);
end;
For 'Date: Mon, 03 Jun 2019 15:32:25 +0200' mbox date string it looks like
WriteLn(ChangeDateFormat('Date: Mon, 03 Jun 2019 15:32:25 +0200'));
it gives back
03-06-2019 15:32:25
which is already recognized e.g. by Excel and allows sorting or other operations.
At the moment I haven't found any ready-made functions in Delphi.
It is also called RFC1123 format:
https://www.ietf.org/rfc/rfc1123.txt
kbmMW's date time features supports that and many other formats.
Eg.
var
dt:TkbmMWDateTime;
ndt:TDateTime;
begin
dt.RFC1123DateTime:='Mon, 03 Jun 2019 15:32:25 +0200';
ndt:=dt.Local; // ndt will contain the local time variant.
// If you want to get the UTC time variant use
// ndt:=dt.UTC
end;
This feature is included in kbmMW Community Edition, which is free and is available for various versions (incl. 10.4.2) of Delphi.
When Delphi 11 Community Edition is released, kbmMW Community Edition will follow. Notice that kbmMW Community Edition installs in all Delphi variants, also Pro, Ent and Architect.
kbmMW Community Edition is free to use, also for commercial use (within license limits).
Download kbmMW CE after registering at https://portal.components4developers.com
I tried to verify an email address using SMTP for all mailboxes it's working fine except for Yahoo email ID. While connecting telnet I got errors like "556 5.7.5 Invalid RFC missing body".
Welcome to Stack Overflow, Jayashree.
The error message you are receiving is related to your test email not having a body part. The body part begins immediately after 2 <CRLF><CRLF> following the Content-Transfer-Encoding.
It begins at Good morning. and ends at P | 999.555.1234
Below is a complete example of a properly formatted text/plain email.
To: "Jane Doe" <jane#example1.com>
From: "John Doe" <john#example1.com>
Subject: A text/plain email
MIME-Version: 1.0 (Created with SublimeText 3)
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Good morning.
This is a message in text/plain format.
It contains text/plain.
It contains no base64 inline images or attachments.
Each line is no more than 76 characters in length.
Please reach out if you have any questions.
Thank you,
John Doe
SENIOR DEVELOPER
XYZ Solutions Inc.
P | 999.555.1234
i send a mail from a monitoring system to another linux box that handles and parses the mail. The mail is sent with the following headers:
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
the problem is that the last line of the mail gets split and a = sign gets added:
CRIT - 93.2% used (466.06 of 500.0 GB), (levels at 80.00/90.00%), trend=
: +5.66MB / 24 hours=
do you have any idea how can i prevent that quote-printable problem so that the last line on the mail is not altered by the receiving MUA.
thank you
Mario.
quoted-printable does this by design, special characters are replaced
such that \r\l or \n are encoded.
The email client sees the mime-header
Content-Transfer-Encoding: quoted-printable
and reverses that to get back the original formatting.
I'd like to automatically generate a text file for an original reply email, for example, like the following:
Subject: hello
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
Content-Type: text/html;
charset=us-ascii
X-Apple-Base-Url: x-msg://5/
X-Universally-Unique-Identifier: f3193934-f4df-4b36-95ab-36f3171570b8
X-Apple-Mail-Remote-Attachments: YES
From: Some Person <usera#mydomain.com>
X-Apple-Windows-Friendly: 1
Date: Mon, 24 Feb 2014 14:43:11 -0800
X-Apple-Mail-Signature: SKIP_SIGNATURE
Content-Transfer-Encoding: 7bit
Message-Id: <41FFDBF3-D482-45A4-A11F-CA545621B513#mydomain.com>
X-Smtp-Server: mymachine
X-Uniform-Type-Identifier: com.apple.mail-draft
To: userb#mydomain.com
<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "></body></html>
This is obtained from first replying to an incoming email, then in the sent box, I can see show original source.
Is there any way to automatically generate this email as a text file, or print out on to stdout?
Those cryptic values assigned by SMTP/Exchnage servers, it should not be generated by client programs.
You could generate emails this way: http://learn.perl.org/examples/email.html
I realize that Perl has strftime, which allows you to pass a formatting object. The functionality I'm wondering if I can find is more like the following (from PHP):
$string1 = "Jun 6, 2012";
$string2 = "June 06 2012";
if (strtotime($string1) == strtotime($string2)) {
echo "BLAMMO!";
}
// will echo "BLAMMO!"
The reason for this is a business need in which user-provided dates need to be compared for validation and extended logic (does this date fall within another daterange also provided, etc). Now, I realize I can write an entire library devoted to doing this, and I realize there are any number of potential pitfalls with date-parsing and you should never trust user input, but here are some basic assumptions.
The input is actually output from any number of software packages that conform to their own internal specifications for date formatting. They all follow some standard, but those standards are not uniformly normalized between programs. That being said, I should always be comparing two dates from the same program, but I may never know what format they may follow.
I realize the standards of any given system are likely to be different, but the assumption here is that we're feeding ALL of our dates into the same thing, so we can trust a consistent implementation, hopefully something in CPAN or another easily updated module.
Date::Parse supplies str2time, which does this.
The documentation lists some examples that the module can parse:
1995:01:24T09:08:17.1823213 ISO-8601
1995-01-24T09:08:17.1823213
Wed, 16 Jun 94 07:29:35 CST Comma and day name are optional
Thu, 13 Oct 94 10:13:13 -0700
Wed, 9 Nov 1994 09:50:32 -0500 (EST) Text in ()'s will be ignored.
21 dec 17:05 Will be parsed in the current time zone
21-dec 17:05
21/dec 17:05
21/dec/93 17:05
1999 10:02:18 "GMT"
16 Nov 94 22:28:20 PST
The standard for date/time manipulation in Perl is the DateTime project.
https://metacpan.org/pod/DateTime
strptime for Perl can be obtained in the core module Time::Piece. This offer a core module solution whereas the rich DateTime module unfortunately isn't part of core Perl.