Using MailKit for a long queue of emails - email

We currently use SmtpClient to send emails, we usually have around 1000-5000 emails per day. We are having some performance issues, sometimes the Send command takes a very long time. Upon researching, I learned about MailKit and how it replaced SmtpClient. Reading through examples, every single one calls for
using (var client = new SmtpClient ()) {
client.Connect ("smtp.friends.com", 587, false);
// Note: only needed if the SMTP server requires authentication
client.Authenticate ("joey", "password");
client.Send (message);
client.Disconnect (true);
}
With Disconnect after every message. If I'm planning to send many messages in sequence, should I still call a new SmtpClient instance for each one and Disconnect it? What is the proper way to handle a long sequence of email sends?

You do not need to disconnect after sending a single message. You can keep calling the Send() method over and over until you are done sending messages.
A simple example might look like this:
static void SendALotOfMessages (MimeMessage[] messages)
{
using (var client = new SmtpClient ()) {
client.Connect ("smtp.friends.com", 587, false);
// Note: only needed if the SMTP server requires authentication
client.Authenticate ("joey", "password");
// send a lot of messages...
foreach (var message in messages)
client.Send (message);
client.Disconnect (true);
}
}
A more complex example would take into consideration handling of SmtpProtocolException and IOException which typically mean that the client got disconnected.
If you get an SmtpProtocolException or an IOException, you are guaranteed to need to reconnect. Those exceptions are always fatal.
SmtpCommandException, on the other hand, is typically not fatal and you will normally not need to reconnect. You can always check the SmtpClient.IsConnected property to verify.

Related

Azure Service Bus Queue gives control characters in fetched message

I am using BizTalk Server SB-Messaging adapter to retreive messages from Azure Service Bus Queue. I have successfully managed to send message to queue myself (using same adapter), and retreive message from queue and do further processing.
Problem arises when a 3rd party software supplier is sending messages to the queue, and for BizTalk Server to retreive and process message. I then receive the following additional "header"-information and control characters in the beginning of the message:
In text: #ACKstringBShttp://schemas.microsoft.com/2003/10/Serialization/?$SOH
Seems like there is some sort of enveloped message, including headers to handle ACKnowledgement of the message to the queue.
SB-Messaging adapter gave following initial error message:
"The WCF service host at address has faulted and as a result no
more messages can be received on the corresponding receive location.
To fix the issue, BizTalk Server will automatically attempt to restart
the service host."
And, another error message:
"No Disassemble stage components can recognize the data."
Did anyone hit this problem before, and, what can be the cause of the problem? Can character encoding be a possible cause of this problem?
Here comes the feedback!
Turned out 3rd party software supplier had a setting to send message as stream, instead of string. Turns out it is a .Net application using BrokeredMessage object. Using string makes message serialized, and meta-data is added to the message. Using stream, no such serialization takes place, and message is kept untouched.
So, problem was using string and automatic serialization when sending to Service Bus Queue.
I have legacy Microsoft.ServiceBus.Messaging clients sending BrokeredMessage Xml content as <string> and I want to receive using the latest Microsoft.Azure.ServiceBus library and Message type.
Using Encoding.UTF8.GetString(message.Body) I get a unusable string prefaced with
#\u0006string\b3http://schemas.microsoft.com/2003/10/Serialization/��#
My approach is to explicitly use XmlDictionaryReader binary deserialization to undo the hidden serialization magic from the legacy library
private static string GetMessageBodyHandlingLegacyBrokeredMessage(Message message)
{
string str = null;
if (message.ContentType == null)
{
using (var reader = XmlDictionaryReader.CreateBinaryReader(
new MemoryStream(message.Body),
null,
XmlDictionaryReaderQuotas.Max))
{
var doc = new XmlDocument();
doc.Load(reader);
str = doc.InnerText;
}
}
else
throw new NotImplementedException("Unhandled Service Bus ContentType " + message.ContentType);
return str;
}
References
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messages-payloads#payload-serialization
https://carlos.mendible.com/2016/07/17/step-by-step-net-core-azure-service-bus-and-amqp/

In UWP StreamSocket, can I read data with timeout and leave the connection open if timeout elapses

As I couldn't find any way to peek for data (read data without consuming the buffer) as asked at How to peek StreamSocket for data in UWP apps I'm now trying to make my own "peek" but still no luck.
I don't see how I can read data from StreamSocket in the manner which will let me use timeouts and leave the connection usable in case if timeout elapses.
In the end, the problem is as follows. In my, let's say, IMAP client, I get response from a server and if this response is negative, I need to wait a bit to see if the server immediately sends yet another response (sometimes, the server can do it, with extra details on the error or even a zero packet to close the connection). if the server didn't send another response, I'm fine, just leaving the method and returning to the caller. The caller can then send more data to the stream, receive more responses, etc.
So, after sending a request and getting initial response I need in some cases to read socket once again with a very small timeout interval and if no data arrives, just do nothing.
You can use a CancelationTokenSource to generate a timeout and stop an async operation.
The DataReader consumes the data from the input stream of the StreamSocket. Its LoadAsync() method will return when there is at least one byte of data. Here, we are adding a cancellation source that will cancel the asynchronous task after 1 second to stop the DataReader.LoadAsync() if no data has been consumed.
var stream = new StreamSocket();
var inputStream = stream.InputStream;
var reader = new DataReader(inputStream);
reader.InputStreamOptions = InputStreamOptions.Partial;
while(true)
{
try
{
var timeoutSource = new CancellationTokenSource(TimeSpan.FromSeconds(1));
var data = await reader.LoadAsync(1).AsTask(timeoutSource.Token);
while(reader.UnconsumedBufferLength > 0)
{
var read = reader.ReadUInt32();
}
}
catch(TaskCanceledException)
{
// timeout
}
}
Do no forget that disposing the DataReader will close the stream and the connection.

NETTY 4.1.4: TCP Socket Server which replies back towards clients after processing requests

i'm new to Netty and intend to create a tcp socket server which reads the info of each client and replies back towards client before processing requests immediately ,i.e. sort of an acknowledgement towards client as and when the message enters overriden channelRead method of ChannelInboundHandlerAdapter class.
Please guide me in the above specified objective.
i'm currently trying the basic netty 4.1.4 echo server example however i wanted server to send back acknowledgement to the client so i updated channelread method as follows :
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ctx.write(msg);
ChannelFuture cf = ctx.channel().write("FROM SERVER");
System.out.println("Channelfuture is "+cf);
}
and the output obtained was as follows:
Channelfuture is DefaultChannelPromise#3f4ee9dd(failure: java.lang.UnsupportedOperationException: unsupported message type: String (expected: ByteBuf, FileRegion))
I understand the error that it is expecting bytebuf but how do i achieve it? also, whether this method would be able to send out acknowledgement towards client
You can use String.getBytes(Charset) and Unpooled.wrappedBuffer(byte[]) to convert to ByteBuf.
ChannelFuture cf = ctx.channel()
.write(Unpooled.wrappedBuffer("FROM SERVER".getBytes(CharsetUtil.UTF_8)));
Also note that ctx.channel().write(...); may not be what you want. Consider ctx.write(...); instead. The difference is that if your handler is a ChannelDuplexHandler it would receive a write event when you do channel().write(). Using ctx instead of channel will send the write out from your handlers point in the pipeline instead of from the end of the pipeline, which is usually what you want.

My SMTP server can't process the incoming DATA of certain SMTP server

We used a SMTP server to receive emails, It works perfectly when I send an email to my server from gmail or hotmail. But I'm having problems with a specific company ( I will name it company x), we can read the sender, the recipient, etc, but when it comes to the DATA, the buffered reader hangs forever reading the line on the socket.
This is what happen when I receive the email from the company x:
When I receive a company x's email. they send me the EHLO command
my server returns 500 command unrecognised
the company x's server send the HELO command
my server sends 250 ok to the company x's server.
the copany x's server send me MAIL From: <sender#email.com>
my server sends 250 ok
the company x's server send me RCPT To:<recipient#email.com>
my server sends 250 ok
the company x's server sends DATA
my server sends 354 Start mail input; end with <CRLF>.<CRLF>
At this moment the server hangs forever reading the incoming data (in the in.readline()) and my server throws socket time out exception. (obviously we tried increasing the time out, but it didn't work)
what could be the difference with the company x's SMTP server and the gmail or hotmail server??, what is the problem???
We have the same error with the Java mail server and the james mail server.
The company x, is a bank so they have a high information security level.
here is the code how we send it.
private static final String MESSAGE_SEND_DATA = "354 Start mail input; end with <CRLF>.<CRLF>";
and this is the method that write the comand on the outputstream.
private void write( String message ) {
if( log.isDebugEnabled() ) { log.debug( "Writing: " + message ); }
out.print( message + "\r\n" );
out.flush();
}
we call write after we receive the data commanx from the client.
Most likely it hangs forever awaiting for line terminator symbol. BufferedReader#readLine is platform dependent in terms of line terminators, so it's a poor Reader choice for SMTP server implementation. As a matter of fact, I would advice not to use BufferedReader and PrintWriter for socket-originated streams to avoid platform-dependent issues like this one.
All SMTP server implementations in Java I'm aware of (Apache James, SubethaSMTP) use dedicated Reader implementations, focused on CRLF-terminated lines to read lines from a Socket, in full accordance with RFC 5321. So my advice would be to try one of these readers (for example this one) instead of BufferedReader.

Read from Half Open Socket

I am trying to connect to Apple Push Notification Service which uses a simple binary protocol over TCP protected with TLS (or SSL). The protocol indicates that when an error is encountered (there are about 10 well defined error conditions) APNS will send back an error response and then close the connection. This results in a half closed socket because the remote peer closed the socket. I can see its a graceful shutdown because APNS sends a FIN and RST using tcpdump.
Out of all the error conditions, I can deal with most before sending with validation. The situation in which this fails is when a notification is sent to an invalid device token which cannot be dealt with that easily because the tokens could be malformed. Tokens are opaque 32 byte values that are provided by APNS to a device and then registered with me. I have no way of knowing if it is valid when submitted to my service. Presumably APNS checksums the tokens in some way that they can do quick validation on the token fast.
Anyway,
I did what I thought was the right thing:-
a. open socket
b. try writing
c. if write failed, read the error response
Unfortunately, this doesn't seem to work. I figure APNS is sending an error response and I am not reading it back right or I am not setting the socket up right. I have tried the following techniques:-
Use a separate thread per socket to try-read the response if any every 5ms or so.
Use a blocking read after write failure.
Use a final read after remote disconnect.
I have tried this with C# + .NET 4.5 on Windows and Java 1.7 on Linux. In either case, I never seem to get the error response and the socket indicates that no data is available to read.
Are half-closed sockets supported on these operating systems and/or frameworks? There isn't anything that seems to indicate either way.
I know that the way I am setting things up works correctly because if I use a valid token with a valid notification, those do get delivered.
In response to one of the comments, I am using the enhanced notification format so a response should arrive from APNS.
Here is the code I have for C#:-
X509Certificate certificate =
new X509Certificate(#"Foo.cer", "password");
X509CertificateCollection collection = new X509CertificateCollection();
collection.Add(certificate);
Socket socket =
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.Connect("gateway.sandbox.push.apple.com", 2195);
NetworkStream stream =
new NetworkStream(socket, System.IO.FileAccess.ReadWrite, false);
stream.ReadTimeout = 1000;
stream.WriteTimeout = 1000;
sslStream =
new SslStream(stream, true,
new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
sslStream.AuthenticateAsClient("gateway.sandbox.push.apple.com", collection,
SslProtocols.Default, false);
sslStream.ReadTimeout = 10000;
sslStream.WriteTimeout = 1000;
// Task rdr = Task.Factory.StartNew(this.reader);
// rdr is used for parallel read of socket sleeping 5ms between each read.
// Not used now but another alternative that was tried.
Random r = new Random(DateTime.Now.Second);
byte[] buffer = new byte[32];
r.NextBytes(buffer);
byte[] resp = new byte[6];
String erroneousToken = toHex(buffer);
TimeSpan t = (DateTime.UtcNow - new DateTime(1970, 1, 1));
int timestamp = (int) t.TotalSeconds;
try
{
for (int i = 0; i < 1000; ++i)
{
// build the notification; format is published in APNS docs.
var not = new ApplicationNotificationBuilder().withToken(buffer).withPayload(
#'{"aps": {"alert":"foo","sound":"default","badge":1}}').withExpiration(
timestamp).withIdentifier(i+1).build();
sslStream.Write(buffer);
sslStream.Flush();
Console.Out.WriteLine("Sent message # " + i);
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd);
break;
}
// doesn't really matter how fast or how slow we send
Thread.Sleep(500);
}
}
catch (Exception ex)
{
Console.Out.WriteLine("Failed to write ...");
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd); ;
}
}
// rdr.Wait(); change to non-infinite timeout to allow error reader to terminate
I implemented server side for APNS in Java and have problems reading the error responses reliably (meaning - never miss any error response), but I do manage to get error responses.
You can see this related question, though it has no adequate answer.
If you never manage to read the error response, there must be something wrong with your code.
Using a separate thread for reading worked for me, though not 100% reliable.
Use a blocking read after write fail - that's what Apple suggest to do, but it doesn't always work. It's possible that you send 100 messages, and the first has an invalid token, and only after the 100th message you get a write failure. At this point it is sometimes too late to read the error response from the socket.
I'm not sure what you mean there.
If you want to guarantee that the reading of the error responses will work, you should try to read after each write, with a sufficient timeout. This, of course, is not practical for using in production (since it's incredibly slow), but you can use it to verify that your code of reading and parsing the error response is correct. You can also use it to iterate over all the device tokens you have, and find all the invalid ones, in order to clean your DB.
You didn't post any code, so I don't know what binary format you are using to send messages to APNS. If you are using the simple format (that starts with a 0 byte and has no message ID), you won't get any responses from Apple.