MailKit in winautomation V8 does not accept Arabic letter - mailkit

In Winautomation V.7 I could write arabic letters in filds but in winautoamtion V.8 it give me this Error
but if I write English letter or left it blank it accept it.
this Error when I put Arabic letter like "عد"
and in logs event typs I find this message
Failed to filter messages on mail-folder "INBOX"
MailKit.Net.Imap.ImapCommandException: The IMAP server replied to the
'SEARCH' command with a 'NO' response: The specified charset is not
supported.
at MailKit.Net.Imap.ImapFolder.Search(SearchQuery query, CancellationToken
cancellationToken)
at WinAutomation.Actions.Runtime.EmailActions.RetrieveEmails(Variant
imapServer, Variant username, Variant varPassword, Variant mailFolder,
Variant fromContains, Variant toContains, Variant subjectContains, Variant
bodyContains, Variant sentSince, Variant sentUpTo, Variant
saveAttachmentsInto, Variant& retrievedEmails, Int32 serverPort, Boolean
enableSsl, Boolean passwordDirectly, String password, Boolean
retrieveOnlyUnread, Boolean saveAttachments, Boolean markAsRead)"
but If I left it blank or write any English letter it accept it
here i add the word "rest"
in old version of winautomation it accept any letter in Arbic or english>

This error happens when the IMAP server only supports US-ASCII.
If you get a protocol log, you should be able to see a list of charsets that the server supports in the error message.
I'm 100% certain that it will list only US-ASCII as a possible charset and that's why you are getting this error.

Related

Requests fail authorization when query string contains certain characters

I'm making requests to Twitter, using the OAuth1.0 signing process to set the Authorization header. They explain it step-by-step here, which I've followed. It all works, most of the time.
Authorization fails whenever special characters are sent without percent encoding in the query component of the request. For example, ?status=hello%20world! fails, but ?status=hello%20world%21 succeeds. But the change from ! to the percent encoded form %21 is only made in the URL, after the signature is generated.
So I'm confused as to why this fails, because AFAIK that's a legally encoded query string. Only the raw strings ("status", "hello world!") are used for signature generation, and I'd assume the server would remove any percent encoding from the query params and generate its own signature for comparison.
When it comes to building the URL, I let URLComponents do the work, so I don't add percent encoding manually, ex.
var urlComps = URLComponents()
urlComps.scheme = "https"
urlComps.host = host
urlComps.path = path
urlComps.queryItems = [URLQueryItem(key: "status", value: "hello world!")]
urlComps.percentEncodedQuery // "status=hello%20world!"
I wanted to see how Postman handled the same request. I selected OAuth1.0 as the Auth type and plugged in the same credentials. The request succeeded. I checked the Postman console and saw ?status=hello%20world%21; it was percent encoding the !. I updated Postman, because a nice little prompt asked me to. Then I tried the same request; now it was getting an authorization failure, and I saw ?status=hello%20world! in the console; the ! was no longer being percent encoded.
I'm wondering who is at fault here. Perhaps Postman and I are making the same mistake. Perhaps it's with Twitter. Or perhaps there's some proxy along the way that idk, double encodes my !.
The OAuth1.0 spec says this, which I believe is in the context of both client (taking a request that's ready to go and signing it before it's sent), and server (for generating another signature to compare against the one received):
The parameters from the following sources are collected into a
single list of name/value pairs:
The query component of the HTTP request URI as defined by
[RFC3986], Section 3.4. The query component is parsed into a list
of name/value pairs by treating it as an
"application/x-www-form-urlencoded" string, separating the names
and values and decoding them as defined by
[W3C.REC-html40-19980424], Section 17.13.4.
That last reference, here, outlines the encoding for application/x-www-form-urlencoded, and says that space characters should be replaced with +, non-alphanumeric characters should be percent encoded, name separated from value by =, and pairs separated by &.
So, the OAuth1.0 spec says that the query string of the URL needs to be decoded as defined by application/x-www-form-urlencoded. Does that mean that our query string needs to be encoded this way too?
It seems to me, if a request is to be signed using OAuth1.0, the query component of the URL that gets sent must be encoded in a way that is different to what it would normally be encoded in? That's a pretty significant detail if you ask me. And I haven't seen it explicitly mentioned, even in Twitter's documentation. And evidently the folks at Postman overlooked it too? Unless I'm not supposed to be using URLComponents to build a URL, but that's what it's for, no? Have I understood this correctly?
Note: ?status=hello+world%21 succeeds; it tweets "hello world!"
I ran into a similar issue.
put the status in post body, not query string.
Percent-encoding:
private encode(str: string) {
// encodeURIComponent() escapes all characters except: A-Z a-z 0-9 - _ . ! ~ * " ( )
// RFC 3986 section 2.3 Unreserved Characters (January 2005): A-Z a-z 0-9 - _ . ~
return encodeURIComponent(str)
.replace(/[!'()*]/g, c => "%" + c.charCodeAt(0).toString(16).toUpperCase());
}

Google Cloud Storage list objects with name containing "%" return 403 error

I am getting my objects by calling
https://<bucket>.storage.googleapis.com/?prefix=folder%2F<object name>%2F&delimiter=/&max-keys=1000
I have tried with other special characters like !, #, #, $, ^, &, *, (, ), etc.
For the other special characters I just encode them in the , and I get the response just fine.
For example, with object "!#" under folder, the url is:
https://<bucket>.storage.googleapis.com/?prefix=folder%2F%21%22%2F&delimiter=/&max-keys=1000
However, when I try with object names with "%" and encode the percent sign to "%25", I get the following error:
<?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidSecurity</Code> <Message>The provided security credentials are not valid.</Message><Details>Request was not signed or contained a malformed signature</Details></Error>
What could be causing this issue ?
Edit
So I have tried double encoding the percent sign such that '%' character becomes "%2525" in the request. However, in the response, the prefix is strangely "%25". After testing with more cases, it turns out a request is successful only when "%25" is followed by 2 characters both within the range of '0' and 'f', however, the response prefix would be wrong. For example, "%25ab" in the request would result in "%ab" in the response prefix.
I believe this is a service side bug: see https://issuetracker.google.com/issues/117932947
I think a workaround is to encode the percent twice. But this may start failing in the future when the bug is fixed.
The error message you're seeing is because you don't have enough permissions to access to your object.
If you're using an authentication method (APIkey, bearer, etc) make sure that they have the needed Roles for GCS.
However, I can see that you're calling the objects just as a GET request. Try to Make your objects public and try it again with that encoding (%25). It should work.
Hope this is helpful!

How does SMTP escape an RSET command after a DATA command?

In RFC 821, it says that a reset (RSET) command can be sent after a DATA command and some mail data has been sent:
However, what distinguishes between a mail client sending an RSET command after DATA, and a mail that contains the word "RSET" on a line by itself?
I've checked RFC 5321 as well and I can't see anything that would mitigate or escape this. It does talk about escaping a mail line which starts with a ".", but not "RSET".
The client cannot terminate the mail data transfer with a period on a line by itself or the server will send the partial mail it has been given.
I imagine there's something I've missed in the RFCs, otherwise I can't help thinking that there's either an SMTP command injection attack vector in many implementations, or no-one can ever send a mail with "RSET" on a line by itself (I think people would have noticed).
The keyword here is after I believe. The DATA command is in progress until it is finished with a lone . on a line.
RFC 5321 § 4.1.1.5 (RSET) states "any stored sender, recipients, and mail data MUST be discarded." This refers to the MAIL FROM, RCPT TO, and presumably DATA commands.
However, upon receiving the . following DATA, the message "MUST" be delivered (which may result in a failure but not a partial failure, see § 4.1.1.4). This clears the buffer of everything RSET is supposed to do.
This means RSET merely elicits a 250 OK response from the receiving server (a keep-alive, much like NOOP) and confirms to the sender that there is indeed no saved sender or recipient queued for the next message.
I do not know of a way to interrupt a DATA command to issue a RSET. The only way I know of to do that is to terminate the connection and establish a new one—and, just to be safe in the case of some odd resumption capability, I'd issue an RSET right after the EHLO or HELO (which the spec says is a NOOP). If there were such a way, it should be in RFC 5321 § 4.1.1.4, § 4.1.1.5, and/or § 3.3.

TextSearchQuery not works in MailKit

I use MailKit and IMAP Client. I want get messages if they have specified subject:
client.Inbox.Open(FolderAccess.ReadOnly);
var query = SearchQuery.SubjectContains("my_subject");
var uids = client.Inbox.Search(query);
if (!uids.Any())
MessageBox.Show("Empty list!");
but all methods which returns TextSearchQuery (eg. SubjectContains, BodyContains, FromContains...) not works and I get MessageBox with empty list info. But if my query return SearchQuery eg.:
var query = SearchQuery.NotSeen;
or
var query = SearchQuery.NotFlagged;
it works correctly. Where is the problem? With chars encoding?
Different IMAP servers implement text matching differently. Some IMAP servers do a literal substring search while others tokenize the message text/subject text when the message arrives and keep a database of which messages contain which words.
Some IMAP servers decode the text (if it is encoded) and some do not.
Text-based searching is all completely dependent upon the IMAP server you are using.
I solved this problem. I changed "_" char in subject with "+" (permissible in Base64, perhaps it matters).

How to retrieve a IMAP body with the right charset?

I'm trying to make an "IMAP fetch" command to retrieve the message body. I need to pass/use the correct charset, otherwise the response will come with special characters.
How can I make the IMAP request/command to consider the charset I received in the BODYSTRUCTURE result??
The IMAP server will send non-ASCII message body data as an 8-bit "literal", which is essentially an array of bytes. You can't make the FETCH command use a specific charset, as far as I know.
It's up to the IMAP library or your application to decode the byte array into a string representation using the appropriate charset you received in the BODYSTRUCTURE response.