SMTP mail server (sendgrid) error on submission - sockets

I have an app that sends an email with image attachments thru an SMTP server (sendgrid).
When the application launches it initializes a Socket connection and authenticates the user(application. I see the following message returned from Sendgrid
SG ESMTP service ready at<foo..sendgrid.net
and I also get a successful authentication returned.
However, on sending an email I get the following 3 errors:
error 1 550 Unauthenticated senders not allowed
error 2 503 Must have sender before recipient
error 3 503 Must have valid receiver and originator
So this suggests to me that I must change the order of sender and recipient in my SMTP module. Internally I am taking an incoming ByteArray and converting to base64 string in order to send emails with attached files. So how would I change the following code excerpt?
writeUTFBytes ("MAIL FROM: <"+pFrom+">\r\n");
writeUTFBytes ("RCPT TO: <"+pDest+">\r\n");
writeUTFBytes ("DATA\r\n");
writeUTFBytes ("From: "+pFrom+"\r\n");
writeUTFBytes ("To: "+pDest+"\r\n");
This is the entire class I am using. I call authenticate just once after the class is initialized and then sendAttachedMail when sending email with image attachments. And when testing using a local SMTP server WITHOUT authentication, everything works fine(emails and image attachments sent)
package org.bytearray.smtp.mailer
{
import flash.events.ProgressEvent;
import flash.net.Socket;
import flash.utils.ByteArray;
import flash.utils.getTimer;
import org.bytearray.smtp.crypto.MD5;
import org.bytearray.smtp.encoding.Base64;
import org.bytearray.smtp.events.SMTPEvent;
import org.bytearray.smtp.infos.SMTPInfos;
public class SMTPMailer extends Socket
{
private var sHost:String;
private var buffer:Array = new Array();
// regexp pattern
private var reg:RegExp = /^\d{3}/img;
// PNG, JPEG header values
private static const PNG:Number = 0x89504E47;
private static const JPEG:Number = 0xFFD8;
// common SMTP server response codes
// other codes could be added to add fonctionalities and more events
private static const ACTION_OK:Number = 0xFA;
private static const AUTHENTICATED:Number = 0xEB;
private static const DISCONNECTED:Number = 0xDD;
private static const READY:Number = 0xDC;
private static const DATA:Number = 0x162;
private static const BAD_SEQUENCE:Number = 0x1F7;
public function SMTPMailer ( pHost:String, pPort:int)
{
super ( pHost, pPort );
sHost = pHost;
addEventListener(ProgressEvent.SOCKET_DATA, socketDataHandler,false,0,true);
}
public function reset():void{
removeEventListener(ProgressEvent.SOCKET_DATA, socketDataHandler);
}
/*
* This method lets you authenticate, just pass a login and password
*/
public function authenticate ( pLogin:String, pPass:String ):void
{
writeUTFBytes ("EHLO "+sHost+"\r\n");
writeUTFBytes ("AUTH LOGIN\r\n");
writeUTFBytes (Base64.encode64String (pLogin)+"\r\n");
writeUTFBytes (Base64.encode64String (pPass)+"\r\n");
flush();
}
/*
* This method is used to send emails with attached files and HTML
* takes an incoming Bytearray and convert it to base64 string
* for instance pass a JPEG ByteArray stream to get a picture attached in the mail ;)
*/
public function sendAttachedMail ( pFrom:String, pDest:String, pSubject:String, pMess:String, pByteArray:ByteArray, pFileName:String ) :void
{
try {
writeUTFBytes ("HELO "+sHost+"\r\n");
writeUTFBytes ("MAIL FROM: <"+pFrom+">\r\n");
writeUTFBytes ("RCPT TO: <"+pDest+">\r\n");
writeUTFBytes ("DATA\r\n");
writeUTFBytes ("From: "+pFrom+"\r\n");
writeUTFBytes ("To: "+pDest+"\r\n");
writeUTFBytes ("Date : "+new Date().toString()+"\r\n");
writeUTFBytes ("Subject: "+pSubject+"\r\n");
writeUTFBytes ("Mime-Version: 1.0\r\n");
var md5Boundary:String = MD5.hash ( String ( getTimer() ) );
writeUTFBytes ("Content-Type: multipart/mixed; boundary=------------"+md5Boundary+"\r\n");
writeUTFBytes("\r\n");
writeUTFBytes ("This is a multi-part message in MIME format.\r\n");
writeUTFBytes ("--------------"+md5Boundary+"\r\n");
writeUTFBytes ("Content-Type: text/html; charset=UTF-8; format=flowed\r\n");
writeUTFBytes("\r\n");
writeUTFBytes (pMess+"\r\n");
writeUTFBytes ("--------------"+md5Boundary+"\r\n");
writeUTFBytes ( readHeader (pByteArray, pFileName) );
writeUTFBytes ("Content-Transfer-Encoding: base64\r\n");
writeUTFBytes ("\r\n");
var base64String:String = Base64.encode64 ( pByteArray, true );
writeUTFBytes ( base64String+"\r\n");
writeUTFBytes ("--------------"+md5Boundary+"-\r\n");
writeUTFBytes (".\r\n");
flush();
} catch ( pError:Error )
{
trace("Error : Socket error, please check the sendAttachedMail() method parameters");
trace("Arguments : " + arguments );
}
}
/*
* This method is used to send HTML emails
* just pass the HTML string to pMess
*/
public function sendHTMLMail ( pFrom:String, pDest:String, pSubject:String, pMess:String ):void
{
try
{
writeUTFBytes ("HELO "+sHost+"\r\n");
writeUTFBytes ("MAIL FROM: <"+pFrom+">\r\n");
writeUTFBytes ("RCPT TO: <"+pDest+">\r\n");
writeUTFBytes ("DATA\r\n");
writeUTFBytes ("From: "+pFrom+"\r\n");
writeUTFBytes ("To: "+pDest+"\r\n");
writeUTFBytes ("Subject: "+pSubject+"\r\n");
writeUTFBytes ("Mime-Version: 1.0\r\n");
writeUTFBytes ("Content-Type: text/html; charset=UTF-8; format=flowed\r\n");
writeUTFBytes("\r\n");
writeUTFBytes (pMess+"\r\n");
writeUTFBytes (".\r\n");
flush();
} catch ( pError:Error )
{
trace("Error : Socket error, please check the sendHTMLMail() method parameters");
trace("Arguments : " + arguments );
}
}
/*
* This method automatically detects the header of the binary stream and returns appropriate headers (jpg, png)
* classic application/octet-stream content type is added for different kind of files
*/
private function readHeader ( pByteArray:ByteArray, pFileName:String ):String
{
pByteArray.position = 0;
var sOutput:String = null;
if ( pByteArray.readUnsignedInt () == SMTPMailer.PNG )
{
sOutput = "Content-Type: image/png; name="+pFileName+"\r\n";
sOutput += "Content-Disposition: attachment filename="+pFileName+"\r\n";
return sOutput;
}
pByteArray.position = 0;
if ( pByteArray.readUnsignedShort() == SMTPMailer.JPEG )
{
sOutput = "Content-Type: image/jpeg; name="+pFileName+"\r\n";
sOutput += "Content-Disposition: attachment filename="+pFileName+"\r\n";
return sOutput;
}
sOutput = "Content-Type: application/octet-stream; name="+pFileName+"\r\n";
sOutput += "Content-Disposition: attachment filename="+pFileName+"\r\n";
return sOutput;
}
// check SMTP response and dispatch proper events
// Keep in mind SMTP servers can have different result messages the detection can be modified to match some specific SMTP servers
private function socketDataHandler ( pEvt:ProgressEvent ):void
{
var response:String = pEvt.target.readUTFBytes ( pEvt.target.bytesAvailable );
buffer.length = 0;
var result:Array = reg.exec(response);
while (result != null)
{
buffer.push (result[0]);
result = reg.exec(response);
}
var smtpReturn:Number = buffer[buffer.length-1];
var smtpInfos:SMTPInfos = new SMTPInfos ( smtpReturn, response );
if ( smtpReturn == SMTPMailer.READY )
dispatchEvent ( new SMTPEvent ( SMTPEvent.CONNECTED, smtpInfos ) );
else if ( smtpReturn == SMTPMailer.ACTION_OK && (response.toLowerCase().indexOf ("queued") != -1 || response.toLowerCase().indexOf ("accepted") != -1 ||
response.toLowerCase().indexOf ("qp") != -1) ) dispatchEvent ( new SMTPEvent ( SMTPEvent.MAIL_SENT, smtpInfos ) );
else if ( smtpReturn == SMTPMailer.AUTHENTICATED )
dispatchEvent ( new SMTPEvent ( SMTPEvent.AUTHENTICATED, smtpInfos ) );
else if ( smtpReturn == SMTPMailer.DISCONNECTED )
dispatchEvent ( new SMTPEvent ( SMTPEvent.DISCONNECTED, smtpInfos ) );
else if ( smtpReturn == SMTPMailer.BAD_SEQUENCE )
dispatchEvent ( new SMTPEvent ( SMTPEvent.BAD_SEQUENCE, smtpInfos ) );
else if ( smtpReturn != SMTPMailer.DATA )
dispatchEvent ( new SMTPEvent ( SMTPEvent.MAIL_ERROR, smtpInfos ) );
}
}
}

So this suggests to me that I must change the order of sender and recipient in my SMTP module.
The order of your commands is correct. Instead, you should look at the first error message for the real cause of the problem:
error 1 550 Unauthenticated senders not allowed
This tells you that the server requires authentication from the sender, i.e. use of the SMTP AUTH command with user credentials. Attempts to send mail from an unauthenticated sender will be rejected. As such, if MAIL FROM fails, then RCPT TO fails. If RCPT TO fails, then DATA fails. And so on.
Use the SMTP EHLO command to discover the AUTH schemes (and other capabilities) that the server supports, and then send the appropriate AUTH command before sending MAIL FROM.

UPDATE: sorry, I had not examined your class carefully. You already have an authenticate method and it does what it should. So, apparently, while you think that you're calling it, either it isn't getting called, or it fails.
So check carefully that the connection you're authenticating in and the connection you're sending mail in are actually the same connection, and you are not maybe unwittingly re-creating a new, unauthenticated connection.
Also, you really need to check the results of the operations you conduct - in this case, the authentication. Is it really successful?
Old answer
From your example and what little documentation I found, you need - or at least appear to need - two authentications. Probably with the same username and password.
The first one you're doing right, it is the 'XML socket connection' that gets you to the SMTP layer.
Now you need the second authentication. Otherwise all commands will fail and the errors will pancake one into the other, while the 'real' error, as Steffen Ulrich noticed, is the first.
writeUTFBytes ("HELO "+sHost+"\r\n");
// Inner authentication
writeUTFBytes ("AUTH LOGIN\r\n");
writeUTFBytes (Base64.encode64 (username) + "\r\n");
writeUTFBytes (Base64.encode64 (password) + "\r\n");
// HERE you really should read the stream and ensure it says
// "Authentication OK" -- or something to that effect.
writeUTFBytes ("MAIL FROM: <"+pFrom+">\r\n");

This line indicates the error:
error 1 550 Unauthenticated senders not allowed
May it be that you have to register your sender email address in sendgrid first? See https://sendgrid.com/docs/User_Guide/Marketing_Campaigns/senders.html#-Create-Your-Sender-Identity
If you have already done this, what does the email activity page tells you? (See https://sendgrid.com/docs/User_Guide/email_activity.html)

Related

Jmeter Groovy JavaMail API multipart add content to sample result

Looking at answers posted in Reading Emails based on recipient email id in Jmeter using groovy I actually managed to use the recipient search term.
Using the below in a JSR223 Sampler
import javax.mail.Multipart
import javax.mail.internet.MimeMultipart
import javax.mail.Message
import javax.mail.search.RecipientStringTerm
Properties properties = new Properties()
properties.put('mail.imap.host', 'your mail server host') // i.e. imap.gmail.com
properties.put('mail.imap.port', your mail server port) // i.e. 993
properties.setProperty('mail.imap.socketFactory.class', 'javax.net.ssl.SSLSocketFactory')
properties.setProperty('mail.imap.socketFactory.fallback', 'false')
properties.setProperty('mail.imap.socketFactory.port', 'your_mail_server_port') // i.e. 993
def session = javax.mail.Session.getDefaultInstance(properties)
def store = session.getStore('imap')
store.connect('your username (usually email address)', 'your_password')
def inbox = store.getFolder('INBOX')
inbox.open(javax.mail.Folder.READ_ONLY)
def onlyToGivenUser = inbox.search(new RecipientStringTerm(Message.RecipientType.TO,'your_recipient_address')) // i.e. test+1#gmail.com
onlyFromGivenUser.each { message ->
if (message.getContent() instanceof Multipart) {
StringBuilder content = new StringBuilder()
def multipart = (Multipart) message.getContent()
multipart.eachWithIndex { Multipart entry, int i ->
def part = entry.getBodyPart(i)
if (part.isMimeType('text/plain')) {
content.append(part.getContent().toString())
}
}
SampleResult.setResponseData(content.toString(), 'UTF-8')
} else {
SampleResult.setResponseData(message.getContent().toString(), 'UTF-8')
}
}
This works perfectly, but fails when email is ContentType: multipart/MIXED as it does not drill down to multipart/RELATED, multipart/ALTERNATIVE and then to TEXT/PLAIN or TEXT/HTML, on which I like to do a regex on to extract a link from the body.
Guessing some counter on i is needed and an "if else", or something like mentioned here, but unsure how to convert to fit in the above script...
Any help would be much appreciated.
I stepped away from javax.mail.Multipart and javax.mail.internet.MimeMultipart and have implemented the below code in a While Controller
import javax.mail.Message
import javax.mail.search.RecipientStringTerm
Properties properties = new Properties();
properties.put('mail.imap.host', 'your mail server host') // i.e. imap.gmail.com
properties.put('mail.imap.port', your mail server port) // i.e. 993
properties.setProperty('mail.imap.socketFactory.class', 'javax.net.ssl.SSLSocketFactory')
properties.setProperty('mail.imap.socketFactory.fallback', 'false')
properties.setProperty('mail.imap.socketFactory.port', 'your_mail_server_port') // i.e. 993
def session = javax.mail.Session.getDefaultInstance(properties)
def store = session.getStore('imap')
store.connect('your username (usually email address)', 'your_password')
def inbox = store.getFolder('INBOX');
inbox.open(javax.mail.Folder.READ_ONLY);
def onlyToGivenUser = inbox.search(new RecipientStringTerm(Message.RecipientType.TO,'your_recipient_address')); // i.e. test+1#gmail.com
try {
onlyToGivenUser.each { message ->
ByteArrayOutputStream emailRaw = new ByteArrayOutputStream();
message.writeTo(emailRaw);
SampleResult.setResponseData(emailRaw.toString(), 'UTF-8');
}
} catch (Exception ex) {
log.warn("Something went wrong", ex);
throw ex;
}
Hope this helps someone one day.

How to validate DKIM body hash manually?

This is an auto forwarded email.
I'm trying to verify the bh field as per rfc5322.
But the sha256 hash value received is not matching this value.
The message body is: for forwarded mail
Any suggestions to validate the bh= field correctly?
I have included some sample header field data. Please do explain (relaxed) canonical procedure too
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1516798995;
s=jan2017; d=headsup.co.in; i=sender#headsup.co.in;
h=Date:From:To:Message-Id:Subject:MIME-Version:Content-Type; l=627;
bh=ODw8icD/S+4UGcXgR4ocNpxXDh4PolWtd1IUXjh0AUs=;
b=G2vTZ+uQ+krw49l+4aMnaeZjUvAJXPPRA8bvOhs3XZNbd2Ng+odB/F5PI3tRpdhr
C0CJA5KPv4VncP2V1PjNdkgKLjs1eTzLSaqmPjhhQDc8mWQRT0rzcPP3V9v6BeXF987
54Zns/QWtR+RbSacFXvUjyBEOlaWUVAmaVcqw5S8=
//Message: for forwarded mail
//Example Data
Date: Wed, 24 Jan 2018 18:33:08 +0530
From: sender <sender#headsup.co.in>
To: "receiver" <receiver#gmail.com>
Message-Id: <1612843d10d.d305604a24030.1212846966506749871#headsup.co.in>
Subject: Test Arc Seal
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_Part_76205_1329960269.1516798988558"
X-Priority: Medium
//Cannonicalization code (relaxed)
package canonicalization;
import java.io.IOException;
public class Canonicalization {
public String canonicalizeHeader(String name, String value) {
name = name.trim().toLowerCase();
value = value.replaceAll("\\s+", " ").trim();
return name + ":" + value;
}
public String canonicalizeBody(String body) {
if (body == null || "".equals(body)) {
return "\r\n";
}
body = body.replaceAll("[ \\t\\x0B\\f]+", " ");
body = body.replaceAll(" \r\n", "\r\n");
// The body must end with \r\n
if (!body.endsWith("\r\n")) {
return body + "\r\n";
}
// Remove trailing empty lines ...
while (body.endsWith("\r\n\r\n")) {
body = body.substring(0, body.length() - 2);
}
return body;
}
public static void main(String[] args) {
Canonicalization obj=new Canonicalization();
System.out.println(obj.canonicalizeHeader("Date"," Wed, 24 Jan 2018 18:33:08 +0530"));
System.out.println(obj.canonicalizeHeader("From"," sender <sender#headsup.co.in>"));
System.out.println(obj.canonicalizeHeader("To"," \"receiver\" <receiver#gmail.com>"));
System.out.println(obj.canonicalizeBody("for forwarded mail"));
}
}
You don't provide any code to illustrate how you are attempting to verify the body hash, but most likely you are incorrectly canonicalizing the input fed to the hashing function.
The code works fine for emails sent in plaintext format (Content-Type: text/plain;)
But by default, emails are sent by Rich Text Format (Content-Type: multipart/alternative;), hence the body hashes did not match

Why mail not send from my localhost php 7 in codeigniter

From my localhost, i cannot send email using gmail account. I've Used php 7.0.9 and codeigniter. I wrote the code in controller is-
$this->load->library('email');
$config['protocol'] = 'smtp';
$config['smtp_host'] = 'ssl://smtp.googlemail.com';
$config['smtp_port'] = '25';
$config['smtp_user'] = 'xxx#gmail.com';
$config['smtp_pass'] = 'password';
$config['charset'] = 'utf-8';
$config['newline'] = "\r\n";
$config['mailtype'] = 'html';
$config['wordwrap'] = TRUE;
$this->email->initialize($config);
$this->email->from('xxx#gmail.com');
$this->email->to('aaaa#yahoo.com');
$this->email->subject("Welcome to xx.");
$this->email->message('message body.....');
if ($this->email->send()) {
$data['success'] = 1;
} else {
$data['success'] = 0;
$data['error'] = $this->email->print_debugger(array('headers'));
}
echo "<pre>";
print_r($data);
echo "</pre>";
But I got the error
Array
(
[success] => 0
[error] => The following SMTP error was encountered: 0
Unable to send data: AUTH LOGIN
Failed to send AUTH LOGIN command. Error:
Unable to send data: MAIL FROM:
from:
The following SMTP error was encountered:
Unable to send data: RCPT TO:
to:
The following SMTP error was encountered:
Unable to send data: DATA
data:
The following SMTP error was encountered:
Unable to send data: User-Agent: CodeIgniter
Date: Wed, 26 Oct 2016 12:10:08 +0200
Anyone please give me suggestion about it.

Quickfix Which has higher priority: message header or sessionID?

If i send A quickfix message using
FIX::Session::sendToTarget(FIX::Message, FIX::SessionID)
Then suppose in header of message:
sender Comp ID = s1, target CompID = t1
and in the SessionID variable:
sender Comp ID = s1, target CompID = t2
Will the message go to t1 or t2
It will use the session ID you specify in the sendToTarget call. From the QF source code...
bool Session::sendToTarget( Message& message, const SessionID& sessionID )
throw( SessionNotFound )
{
message.setSessionID( sessionID );
Session* pSession = lookupSession( sessionID );
if ( !pSession ) throw SessionNotFound();
return pSession->send( message );
}
Notice that the message session ID is immediately overwritten by the sessionID passed to the function.

stream_socket_server: Client browser randomly aborting?

Below is partial code to an experimental http server app I'm building from scratch from a PHP CLI script (Why? Because I have too much time on my hands). The example below more closely matches PHP's manual page on this function. The problem I'm getting is when connecting to this server app via a browser (Firefox or IE8 from two separate systems tested so far), the browser sends an empty request payload to the server and aborts roughly every 1 in 6 page loads.
The server console displays the "Connected with [client info]" each time. However, about 1 in 6 connections will result in a "Client request is empty" error. No error is given telling the header/body response write to the socket failed. The browser will generally continue to read what I give it, but this isn't usable as I can't fulfill the client's intended request without knowing what it is.
<?php
$s_socket_uri = 'tcp://localhost:80';
// establish the server on the above socket
$s_socket = stream_socket_server($s_socket_uri, $errno, $errstr, 30) OR
trigger_error("Failed to create socket: $s_socket_uri, Err($errno) $errstr", E_USER_ERROR);
$s_name = stream_socket_get_name($s_socket, false) OR
trigger_error("Server established, yet has no name. Fail!", E_USER_ERROR);
if (!$s_socket || !$s_name) {return false;}
/*
Wait for connections, handle one client request at a time
Though to not clog up the tubes, maybe a process fork is
needed to handle each connection?
*/
while($conn = stream_socket_accept($s_socket, 60, $peer)) {
stream_set_blocking($conn, 0);
// Get the client's request headers, and all POSTed values if any
echo "Connected with $peer. Request info...\n";
$client_request = stream_get_contents($conn);
if (!$client_request) {
trigger_error("Client request is empty!");
}
echo $client_request."\n\n"; // just for debugging
/*
<Insert request handling and logging code here>
*/
// Build headers to send to client
$send_headers = "HTTP/1.0 200 OK\n"
."Server: mine\n"
."Content-Type: text/html\n"
."\n";
// Build the page for client view
$send_body = "<h1>hello world</h1>";
// Make sure the communication is still active
if ((int) fwrite($conn, $send_headers . $send_body) < 1) {
trigger_error("Write to socket failed!");
}
// Response headers and body sent, time to end this connection
stream_socket_shutdown($conn, STREAM_SHUT_WR);
}
?>
Any solution to bring down the number of unintended aborts down to 0, or any method to get more stable communication going? Is this solvable on my server's end, or just typical browser behavior?
I tested your code and it seems I got better results reading the socket with fread(). You also forgot the main loop(while(1), while(true) or for(;;).
Modifications to your code:
stream_socket_accept with #stream_socket_accept [sometimes you get warnings because "the connected party did not properly respond", which is, of course, the timeout of stream_socket_accept()]
Added the big while(1) { } loop
Changed the reading from the socket from $client_request = stream_get_contents($conn);
to while( !preg_match('/\r?\n\r?\n/', $client_request) ) { $client_request .= fread($conn, 1024); }
Check the source code below (I used 8080 port because I already had an Apache listening on 80):
<?php
$s_socket_uri = 'tcp://localhost:8080';
$s_socket = stream_socket_server($s_socket_uri, $errno, $errstr, 30) OR
trigger_error("Failed to create socket: $s_socket_uri, Err($errno) $errstr", E_USER_ERROR);
$s_name = stream_socket_get_name($s_socket, false) OR
trigger_error("Server established, yet has no name. Fail!", E_USER_ERROR);
if (!$s_socket || !$s_name) {return false;}
while(1)
{
while($conn = #stream_socket_accept($s_socket, 60, $peer))
{
stream_set_blocking($conn, 0);
echo "Connected with $peer. Request info...\n";
// $client_request = stream_get_contents($conn);
$client_request = "";
// Read until double \r
while( !preg_match('/\r?\n\r?\n/', $client_request) )
{
$client_request .= fread($conn, 1024);
}
if (!$client_request)
{
trigger_error("Client request is empty!");
}
echo $client_request."\n\n";
$headers = "HTTP/1.0 200 OK\n"
."Server: mine\n"
."Content-Type: text/html\n"
."\n";
$body = "<h1>hello world</h1><br><br>".$client_request;
if ((int) fwrite($conn, $headers . $body) < 1) {
trigger_error("Write to socket failed!");
}
stream_socket_shutdown($conn, STREAM_SHUT_WR);
}
}
Add sleep(1) after stream_set_blocking