Not able to search outlook email by SearchTerm - email

I have written a sample code that bring email of one folder of Outlook email account between given date range. I am using Java Mail API 1.5 and Java 7.
For getting messages, I have written below code that search emails since 1-Jan-1970 to before 30-Oct-2013 -
Date FutureDate = new Date(2013 - 1900, 9, 30, 00, 00, 00);
Date PastDate = new Date(1970 - 1900, 0, 1);
SearchTerm newerThen = new ReceivedDateTerm(ComparisonTerm.LE, FutureDate);
SearchTerm olderThen = new ReceivedDateTerm(ComparisonTerm.GE, PastDate);
SearchTerm andTerm = new AndTerm(olderThen, newerThen);
Message[] msg = folder.search(andTerm);
This code is returning zero length msg i.e. no message is found since 1-Jan-1970 to before 30-Oct-2013, though my folder contains 204 emails between this range.
Surprisingly, the same code working fine to search email of folders of Gmail as well as of Yahoo IMAP account.
Following is the protocol trace for Outlook "Inbox" folder that contain 204 emails but not searching emails via date range search term -
DEBUG: setDebug: JavaMail version ${mail.version}
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc.,${mail.version}]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: trying to connect to host "imap-mail.outlook.com", port 993, isSSL true
* OK Outlook.com IMAP4rev1 server version 17.3.0.0 ready (BLU451-IMAP84)
A0 CAPABILITY
* CAPABILITY IMAP4rev1 CHILDREN ID NAMESPACE AUTH=PLAIN AUTH=XOAUTH2 SASL-IR
A0 OK CAPABILITY completed
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: XOAUTH2
DEBUG IMAPS: protocolConnect login, host=imap-mail.outlook.com, user=mymail#outlook.com, password=<non-null>
DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAPS: AUTHENTICATE PLAIN command result: A1 OK mymail#outlook.com authenticated successfully
A2 CAPABILITY
* CAPABILITY IMAP4rev1 CHILDREN ID NAMESPACE
A2 OK CAPABILITY completed
Store gmail:imaps://mymail%40outlook.com#imap-mail.outlook.com
DEBUG IMAPS: connection available -- size: 1
A3 SELECT Inbox
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* 204 EXISTS
* 0 RECENT
* OK [UNSEEN 101] Message 101 is first unseen
* OK [UIDVALIDITY 376204] UIDs valid
* OK [UIDNEXT 100235] Predicted next UID
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)] Limited
A3 OK [READ-WRITE] SELECT completed.
TOTAL MAILS: 204
A4 SEARCH DELETED ALL
* SEARCH 1
A4 OK SEARCH Completed
FOLDER: Inbox
FUTURE DATE: Wed Oct 30 00:00:00 IST 2013
PAST DATE: Thu Jan 01 00:00:00 IST 1970
A5 SEARCH OR SINCE 1-Jan-1970 ON 1-Jan-1970 OR BEFORE 30-Oct-2013 ON 30-Oct-2013 ALL
* SEARCH
A5 OK SEARCH Completed
MESSAGE FOUND: 0
The protocol trace for Yahoo Inbox that contains only one email, and that searching emails with search term -
DEBUG: setDebug: JavaMail version ${mail.version}
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc.,${mail.version}]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: trying to connect to host "imap.mail.yahoo.com", port 993, isSSL true
* OK [CAPABILITY IMAP4rev1 ID NAMESPACE X-ID-ACLID UIDPLUS LITERAL+ CHILDREN XAPPLEPUSHSERVICE XYMHIGHESTMODSEQ AUTH=PLAIN AUTH=LOGIN AUTH=XYMCOOKIE AUTH=XYMECOOKIE AUTH=XYMCOOKIEB64 AUTH=XYMPKI] IMAP4rev1 imapgate-0.7.68_14.446672 imap411.mail.bf1.yahoo.com
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: LOGIN
DEBUG IMAPS: AUTH: XYMCOOKIE
DEBUG IMAPS: AUTH: XYMECOOKIE
DEBUG IMAPS: AUTH: XYMCOOKIEB64
DEBUG IMAPS: AUTH: XYMPKI
DEBUG IMAPS: protocolConnect login, host=imap.mail.yahoo.com, user=mymail#yahoo.com, password=<non-null>
DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAPS: AUTHENTICATE PLAIN command result: A0 OK AUTHENTICATE completed - Mailbox size in bytes is 33054009
A1 CAPABILITY
* CAPABILITY IMAP4rev1 ID NAMESPACE X-ID-ACLID UIDPLUS LITERAL+ XAPPLEPUSHSERVICE XYMHIGHESTMODSEQ
A1 OK CAPABILITY completed
Store gmail:imaps://mymail%40yahoo.com#imap.mail.yahoo.com
DEBUG IMAPS: connection available -- size: 1
A2 SELECT Inbox
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1379569582] UIDs valid
* OK [UIDNEXT 7322] Predicted next UID
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)] Permanent flags
* OK [HIGHESTMODSEQ 5940064101685223608]
A2 OK [READ-WRITE] SELECT completed; now in selected state
TOTAL MAILS: 1
A3 SEARCH DELETED ALL
* SEARCH
A3 OK SEARCH completed
FOLDER: Inbox
FUTURE DATE: Wed Oct 30 00:00:00 IST 2013
PAST DATE: Thu Jan 01 00:00:00 IST 1970
A4 SEARCH OR SINCE 1-Jan-1970 ON 1-Jan-1970 OR BEFORE 30-Oct-2013 ON 30-Oct-2013 ALL
* SEARCH 1
A4 OK SEARCH completed
MESSAGE FOUND: 1
Please guide me where to made changes to get email via search term created with date range?
Is outlook do not support to search with SearchTerm of dates?
Thanks
Neelam Sharma

Looks like the outlook.com server is broken; please report the problem to Microsoft.

I stumbled upon a comparable problem while migrating mails with imapsync which also uses IMAP SEARCH to implement message filtering.
Searching outlook.com via IMAP SEARCH (see RFC 3501, section 6.4.4) works in general. But in my experience specifically searching for mails before a certain date does not seem to work in some folders. E.g. filtering by SENTBEFORE on the Sent folder is broken in outlook.com. I found your question while searching for solutions.
A workaround seems to be using the BEFORE IMAP search key which does not use the Date: header of messages but the internal IMAP server date. (Though I don't know how and if this translates to the JavaMail API.)
It seems that some things are (still) not implemented according to the spec by outlook.com.

Related

Error when trying to send mail over smtp with Office 365 account

An application, which obviously uses Chilkat plugin, does not correctly send E-mails anymore using an Office 365 account. After a migration from on-premise exchange to Microsoft 365, the account and server settings were changed at end of March this year. Furthermore, client authentication needed to be disabled for the mailbox. After that, sending E-mails from the application had worked. Now, it again doesn't. Yesterday, one E-mail could successfully be sent after a computer restart - the next ones failed again.
I'm stuck to find the cause of the error and also I don't know, what data exactly is sent from the application during the connection, which makes the analysis more difficult.
I found this post mentioning that entries for TLS 1.0 need to be in the registry. Those were missing, but there were entries for TLS 1.2, so I didn't assume this is an issue, espccially because the entries concerning cryptography (also mentioned in the post) did already exist. In the meantime, I have added them nonetheless.
Below is the Chilkat Log, showing that - after the server is ready - the clientHandshake2 fails with error code 0x2746.
The support person of the application he couldn't find any problem and asked concerning that error code - what i assumed he should be able to examine. When I talked to him on the phone, I found out that he doesn't know more about Chilkat than I do. He mentioned an interesting point, which need to be double checked: As far as he was informed from the user directly - or how he had understood it - the error doesn't show up, if the user chooses another "Mandant" (client) in the application and sends E-mails from there.
Can someone help identify the cause of the error?
SendEmail:
DllDate: May 25 2017
ChilkatVersion: 9.5.0.68
UnlockPrefix: WERNERMAILQ
Architecture: Little Endian; 32-bit
Language: Visual C++ 2017 (32-bit)
VerboseLogging: 0
sendEmailInner:
renderToMime:
createEmailForSending:
Auto-generating Message-ID
--createEmailForSending
renderToMime: Elapsed time: 0 millisec
--renderToMime
sendMimeInner:
ensureSmtpSession:
ensureSmtpConnection:
SmtpHost: smtp.office365.com
SmtpPort: 587
SmtpUsername: edith.beer#halten.ch
SmtpSsl: 0
StartTLS: 1
smtpConnect:
smtpHostname: smtp.office365.com
smtpPort: 587
connectionIsReady:
Need new SMTP connection
--connectionIsReady
smtpSocketConnect:
socketOptions:
SO_SNDBUF: 262144
SO_RCVBUF: 4194304
TCP_NODELAY: 1
SO_KEEPALIVE: 1
--socketOptions
--smtpSocketConnect
smtpGreeting:
readSmtpResponse:
SmtpCmdResp: 220 AM0PR02CA0163.outlook.office365.com Microsoft ESMTP MAIL Service ready at Mon, 23 Aug 2021 11:51:12 +0000
--readSmtpResponse
--smtpGreeting
startTLS:
sendCmdToSmtp:
SmtpCmdSent: EHLO EWGH-N-FV<CRLF>
--sendCmdToSmtp
readSmtpResponse:
SmtpCmdResp: 250-AM0PR02CA0163.outlook.office365.com Hello [213.221.219.37]
SmtpCmdResp: 250-SIZE 157286400
SmtpCmdResp: 250-PIPELINING
SmtpCmdResp: 250-DSN
SmtpCmdResp: 250-ENHANCEDSTATUSCODES
SmtpCmdResp: 250-STARTTLS
SmtpCmdResp: 250-8BITMIME
SmtpCmdResp: 250-BINARYMIME
SmtpCmdResp: 250-CHUNKING
SmtpCmdResp: 250 SMTPUTF8
--readSmtpResponse
sendCmdToSmtp:
SmtpCmdSent: STARTTLS<CRLF>
--sendCmdToSmtp
readSmtpResponse:
SmtpCmdResp: 220 2.0.0 SMTP server ready
--readSmtpResponse
clientHandshake:
clientHandshake2:
readHandshakeMessages:
WindowsError: Eine vorhandene Verbindung wurde vom Remotehost geschlossen.
WindowsErrorCode: 0x2746
maxToReceive: 5
Failed to receive data on the TCP socket
Failed to read beginning of SSL/TLS record.
b: 0
dbSize: 0
nReadNBytes: 0
idleTimeoutMs: 30000
--readHandshakeMessages
--clientHandshake2
--clientHandshake
Client handshake failed. (1)
connectionClosed: 0
Failed to establish TLS connection.
--startTLS
--smtpConnect
--ensureSmtpConnection
--ensureSmtpSession
--sendMimeInner
--sendEmailInner
Failed.
--SendEmail
--ChilkatLog
23.08.2021 13:48:27 -F- Beim Versand der Lohnabrechnung per Email an Mitarbeiter(in) Nadine Aeschlimann ist ein Fehler aufgetreten!
You are using an old version of Chilkat. Try testing with the latest version.
In the meantime we could solve it.
The reason was that the outgoing requests were blocked by the firewall. Strangely, a few of them went through, but not all. This is the reason why a network issue was initially not in the focus of our investigation.

Unable to hide CONNECT requests in Fiddler

I'm using Fiddler v4.6.20171.26113 on Windows 8.1. I have enabled the Hide CONNECTs option under the Rules menu and even tried put this script in the custom rules file:
if (oSession.HTTPMethodIs("CONNECT"))
{
oSession["ui-hide"] = "true";
}
However the CONNECT requests are still shown even when their flag UI-HIDE: true is set.
SESSION STATE: Done.
Response Entity Size: 0 bytes.
== FLAGS ==================
BitFlags: [ResponseGeneratedByFiddler, IsDecryptingTunnel, ProtocolViolationInRequest, RequestBodyDropped] 0x10a100
HTTPS-CLIENT-SESSIONID: empty
HTTPS-CLIENT-SNIHOSTNAME: mtalk.google.com
LOG-DROP-REQUEST-BODY: yes
LOG-DROP-RESPONSE-BODY: yes
UI-BACKCOLOR: LightYellow
UI-HIDE: true
X-CLIENTIP: ::ffff:***.***.**.**
X-CLIENTPORT: 5033
X-EGRESSPORT: 55428
X-HOSTIP: **.***.***.***
X-HTTPPROTOCOL-VIOLATION: [ProtocolViolation] HTTP/1.1 Request was missing the required HOST header.
X-ORIGINAL-HOST:
X-REQUESTBODYFINALLENGTH: 1,384
X-RESPONSEBODYTRANSFERLENGTH: 0
== TIMING INFO ============
ClientConnected: 07:05:03.136
ClientBeginRequest: 07:05:03.339
GotRequestHeaders: 07:05:03.339
ClientDoneRequest: 07:05:03.339
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 61ms
HTTPS Handshake: 215ms
ServerConnected: 07:05:03.777
FiddlerBeginRequest: 07:05:03.777
ServerGotRequest: 07:05:03.777
ServerBeginResponse: 00:00:00.000
GotResponseHeaders: 00:00:00.000
ServerDoneResponse: 00:00:00.000
ClientBeginResponse: 07:05:03.777
ClientDoneResponse: 07:05:03.777
Overall Elapsed: 0:00:00.437
The response was buffered before delivery to the client.
== WININET CACHE INFO ============
This URL is not present in the WinINET cache. [Code: 2]
* Note: Data above shows WinINET's current cache state, not the state at the time of the request.
* Note: Data above shows WinINET's Medium Integrity (non-Protected Mode) cache only.
So what should I do now?

MailKit: The IMAP server replied to the 'EXAMINE' command with a 'BAD' response

I got this error on an Exchange 2007 mailbox. I see it is returning the BAD response from the error which is thrown by MailKit:
The IMAP server replied to the 'EXAMINE' command with a 'BAD' response.
But this doesn't tell me enough detail. So I am running protocol logging, from which I can tell what the problem is. It says:
K00000005 BAD Duplicate folders "Conversation Action Settings", "News Feed", "Quick Step Settings", "Suggested Contacts" were detected in the mailbox. Therefore the user's connection was disconnected.
I would like to get this error text and display it to the user, so they can fix their mailbox (which they can do by deleting the duplicate folders, assuming they know which ones they are). Is there a simple way to get the full response text?
Full protocol log as follows:
S: * OK Microsoft Exchange Server 2007 IMAP4 service ready
C: K00000000 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN IDLE NAMESPACE LITERAL+
S: K00000000 OK CAPABILITY completed.
C: K00000001 AUTHENTICATE NTLM
...
S: K00000001 OK AUTHENTICATE completed.
C: K00000002 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN IDLE NAMESPACE LITERAL+
S: K00000002 OK CAPABILITY completed.
C: K00000003 NAMESPACE
S: * NAMESPACE (("" "/")) NIL NIL
S: K00000003 OK NAMESPACE completed.
C: K00000004 LIST "" "INBOX"
S: * LIST (\Marked \HasNoChildren) "/" INBOX
S: K00000004 OK LIST completed.
C: K00000005 EXAMINE INBOX
S: K00000005 BAD Duplicate folders "Conversation Action Settings", "News Feed", "Quick Step Settings", "Suggested Contacts" were detected in the mailbox. Therefore the user's connection was disconnected.
S: * BYE Connection is closed. 15
You can try parsing the ImapCommandException.Message property to get the text following the ':' character that is used in the exception message.
I've also just added a ResponseText property to ImapCommandException so you can get this text w/o needing to parse it if you want to use MailKit built source.

Connecting to Microsoft Exchange IMAP4 service using perl Mail::IMAPClient

I am trying to use Mail::IMAPClient to connect to a MS Exchange mailbox.
Read messages in the INBOX and depending on the contents formats and sends email to other mailboxes.
This snippet program is a test to only connect using IMAP and then disconnect. The snippet fails to connect, but I can connect using openssl.
!/usr/bin/perl
use strict;
use warnings;
use Mail::IMAPClient;
use lib "/usr/local/share/perl5";
my $IMAPserver = "imapx.servers.net";
my $IMAPuser = "emailUser";
my $IMAPpass = "resUliame";
my $IMAPport = 993;
print "Start\n";
# Attach a client using an SSL socket.
my $client = Mail::IMAPClient->new(
Server => $IMAPserver,
Port => $IMAPport,
User => $IMAPuser,
Password => $IMAPpass,
Ssl => 1,
DEBUG => 1
)
or die "new(): $#";
$client->State(Mail::IMAPClient::Connected);
# Close IMAP connections
$client->logout();
print "Done\n";
This is the output I get...
Start
Started at Mon Sep 26 10:28:16 2016
Using Mail::IMAPClient version 3.38 on perl 5.010001
Connecting with IO::Socket::SSL PeerAddr imapx.servers.net PeerPort 993 Proto tcp Timeout 600 Debug 1
ERROR: Unable to connect to imapx.servers.net: at /usr/local/share/perl5/Mail/IMAPClient.pm line 370.
Mail::IMAPClient::connect(Mail::IMAPClient=HASH(0x2064b98)) called at /usr/local/share/perl5/Mail/IMAPClient.pm line 313
Mail::IMAPClient::new("Mail::IMAPClient", "Server", "imapx.servers.net", "Port", 993, "User", "emailUser", "Password", "resUliame", ...) called at ./testEmail.pl line 17
new(): Unable to connect to imapx.servers.net: at ./testEmail.pl line 17
I can connect with openssl so I know the server, port, user & passwd is correct.
# openssl s_client -connect imapx.servers.net:993 -crlf
CONNECTED(00000003)
* OK The Microsoft Exchange IMAP4 service is ready.
* LOGIN emailUser resUliame
* OK LOGIN completed.
* SELECT Inbox
* 28027 EXISTS
* 1 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 27544] Is the first unseen message
* OK [UIDVALIDITY 64268] UIDVALIDITY value
* OK [UIDNEXT 46567390] The next unique identifier value
* OK [READ-WRITE] SELECT completed.

POP3 RETR Command Missing +OK Response

I've been searching and testing for days now without any rational explanation for the following to happen:
I have a mail server which serves all our users and everyone is happily running on IMAP/POP3 access. I need to develop a utility to check POP3 e-mail and started getting errors retrieving mail. I set up the same pop account on my outlook and windows live mail and they couldn't download the e-mails either. I tried another PC and it downloaded just fine. After much debugging and searching, I found out that after sending the RETR command, there wasn't an "+OK" response on my pc but there was on the other pc. So I went down to telnet and sure as day my PC wasn't getting the +OK response on RETR, just the actual mail but I was getting it from every other pc I tried. I even booted up my win XP virtual pc and it has the same result as my pc. Here is an excerpt of the logs from mine and my test pc:
RETR with +OK:
+OK Welcome to MailEnable POP3 Server
USER devtest#x.com
+OK
PASS <Removed>
+OK
LIST
+OK 3 26743
1 2118
2 23949
3 676
.
UIDL
+OK
1 BE1F75CAE417453581CF11F16CF09989
2 846882DB63B54C9E91C4643AA5CCA1F5
3 A7BAFC28B04A493689A150F6D4CD7FD0
.
RETR 1
+OK 2118 octets
Received: from x ([x.x.60.10]) by x.net with MailEnab
le ESMTP; Sun, 28 Dec 2014 11:30:16 +0200
RETR with +OK missing:
+OK Welcome to MailEnable POP3 Server
USER devtest#x.com
+OK
PASS <Removed>
+OK
LIST
+OK 3 26743
1 2118
2 23949
3 676
.
UIDL
+OK
1 BE1F75CAE417453581CF11F16CF09989
2 846882DB63B54C9E91C4643AA5CCA1F5
3 A7BAFC28B04A493689A150F6D4CD7FD0
.
RETR 1
Received: from x ([x.x.60.10]) by x.net with MailEnab
le ESMTP; Sun, 28 Dec 2014 11:32:53 +0200
I'm now going to place another hard drive in my pc and install windows and telnet client and see what it does but I was hoping someone might have had some experience with this. It's only that one time that the +OK is missing, every other command has it showing as well as it being there on every other PC I try it on so it's only on my pc that it's missing.
Appreciate any thoughts or assistance!
Well loading a new hard drive worked perfectly as expected which led me to the fact that it must be something installed on my pc itself. Disabled the usual suspects (firewall and AV) and when that didn't work, I proceeded to close down every app running on my pc which possibly works with Ports and eventually narrowed the culprit down to the Fortinet VPN Client running on my PC. I don't have it's AV component enabled so I'm rather at a loss as to how it could be the reason for a single line on port 110 to go missing but if it's running then the +OK line is missing and if it's shut down then it appears.
I'm going to leave this question here in case it can help someone else and will also try post something to Fortinet as well.