I have a response coming from sendgrid inbound webhook, and it have some attachment, but is there any way we can separate the attachment from the full email payload.
This is the email body we receive from sendgrid below: I have shortened the email body and included important property including the X-Attachment-Id and right below is base64 attachment.
body = {
dkim: '{#gmail.com : pass}',
subject: 'Subject 1',
email: 'Received: by mx0121p1iad2.sendgrid.net with SMTP id U3SN4cHg8C Wed, 21 Sep
2022 16:36:45 +0000 (UTC)\n' +
'--000000000000de567405e9328c5d\n' +
'Content-Type: application/pdf; name="dummy copy.pdf"\n' +
'Content-Disposition: attachment; filename="dummy copy.pdf"\n' +
'Content-Transfer-Encoding: base64\n' +
'Content-ID: <f_l8buktdz0>\n' +
'X-Attachment-Id: f_l8buktdz0\n' +
'\n' +
'JVBERi0xLjQKJcOkw7zDtsOfCjIgMCBvYmoKPDwvTGVuZ3RoIDMgMCBSL0Zp\n' +
'bHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nD2OywoCMQxF9/mKu3YRk7bp\n' +
'tDAIDuh+oOAP+AAXgrOZ37etjmSTe3ISIljpDYGwwrKxRwrKGcsNlx1e31mt\n' +'\n' +
'--000000000000de567405e9328c5d\n' }
I know I can use string methods to separate the part below the X-Attachment-ID property for making the attachment which is in base64 format, but is there any other way from sendgrid so we can separate any number of attachments.
Twilio developer evangelist here.
This will most likely be resolved via code implementation with the Parse Webhook on the endpoint that you are consuming Parse payload information too.
This SO answer may be helpful.
Additionally, if you are not already, we do recommend that you select "POST the raw, full MIME message" in your Inbound Parse Settings under Settings > Inbound Parse in the SendGrid UI.
Let me know if this helps at all!
Related
I am looking to encrypt certain mail headers (Subject and Reply-To) which are being sent in an encrypted mail.
I am taking an entire MIME (Headers included) and successfully encrypting it.
I can send this S/MIME encrypted mail to my mail client (Thunderbird) successfully. It will be successfully decrypted and verified as signed.
However, any headers that are sent in the inner encrypted MIME are not being used by my mail client.
According to RFC-5751 I should be wrapping my mail in a message/rfc822 message but I am at a loss at how to achieve this.
Below are examples of my messages that I am creating.
My first question is, is the last MIME that I am creating the message/rfc822 correctly structured?
Is this possibly an issue with the mail client?
Can I event encrypt the Reply-To Header?
If I could get an example of a mesage/rfc822 encapsulated message that would be really helpful.
Mail to be encrypted
This will successfully result in a received mail that is signed and the Subject / Reply-To headers are interpreted correctly by the mail client.
Content-Type: multipart/signed; protocol="application/pkcs7-signature";
micalg=sha256; boundary="--_NmP-d017e0e3556f7bbc-Part_1"
From: sender#domain.com
Sender: senderdomain.com
To: recipient#domain.com
Reply-To: keepsecret#domain.com
Subject: A Secret Subject
Message-ID: <400b1383-362b-eed7-0719-6b2a2e231143>
Date: Mon, 24 Feb 2020 15:59:19 +0000
MIME-Version: 1.0
----_NmP-d017e0e3556f7bbc-Part_1
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
My Message that will be encrypted
----_NmP-d017e0e3556f7bbc-Part_1
Content-Type: application/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7s
MIIOCAYJKoZIhvcNAQcCoIIN+TCCDfUCAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0BBwGg
gguTMIIFCDCCA/CgAwIBAgIQVz2HAGYJcTJNsPiWLx1f/TANBgkqhkiG9w0BAQsFADCBjTELMAkG
.
.
.
17p13e02JxfyCqltdb6lkOdpRZ6ZlHHuQZyBCuRtJhRN83gvcJ4d7WCxKI349NEa2/tOb8ziFGat
gzvgu+o=
----_NmP-d017e0e3556f7bbc-Part_1--
My Encrypted Mail
This encrypted mail will be received and successfully decrypted and verified (signature verified) by my mail client. Reply-To and Subject are still working as expected as they are still visible. Note: all the headers from the unencrypted mail are all still present inside the encrypted body of this message.
Sender: sender#domain.com
From: sender#domain.com
To: recipient#domain.com
Subject: A Secret Subject
Reply-To: keepsecret#domain.com
Message-ID: <400b1383-362b-eed7-0719-6b2a2e231143>
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7m
Content-Type: application/pkcs7-mime; smime-type=enveloped-data;
name=smime.p7m
Date: Mon, 24 Feb 2020 16:03:38 +0000
MIME-Version: 1.0
MIIYbwYJKoZIhvcNAQcDoIIYYDCCGFwCAQAxggG/MIIBuwIBADCBojCBjTELMAkG
.
.
.
O+EPVCh1fGDFwiFpDtY/z1Lv8g==
My Encapsulated message/rfc822
This message will be decrypted correctly but my client does not recognise that it was an encrypted message or verify that it was signed (Not worried about that so much). The decrypted mail is interpreted as forwarded and attached as an .eml file. However, no Subject or Reply-To headers found (they are in the encrypted mail). If I add dummy values as recommended by the RFC, those dummy values will be used by my mail client, not the encrypted ones.
Content-Type: message/rfc822; forwarded=false; boundary="--_NmP-07c15c542cedfe74-Part_1"
From: sender#domain.com
Sender: sender#domain.com
To: recipient#domain.com
Date: Mon, 24 Feb 2020 15:28:07 +0000
Message-ID: <400b1383-362b-eed7-0719-6b2a2e231143>
MIME-Version: 1.0
----_NmP-07c15c542cedfe74-Part_1
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7m
Content-Type: application/pkcs7-mime; smime-type=enveloped-data;
name=smime.p7m
MIIYbwYJKoZIhvcNAQcDoIIYYDCCGFwCAQAxggG/MIIBuwIBADCBojCBjTELMAkG
.
.
.
fYU1LuhSBEyymSVRzwWr2T3lrhUe5BZBoY996epZtOPdIYrz2jqUglii1+AUBpUP
UUnpr8+cHTMk/50LHdy3MqMeYA==
----_NmP-07c15c542cedfe74-Part_1
Edit: add excerpt from RFC
In RFC-8551 it states the following
In order to protect outer, non-content-related message header fields (for instance, the "Subject", "To", "From", and "Cc" fields), the
sending client MAY wrap a full MIME message in a message/rfc822
wrapper in order to apply S/MIME security services to these header
fields. It is up to the receiving client to decide how to present
this "inner" header along with the unprotected "outer" header. Given
the security difference between headers, it is RECOMMENDED that the
receiving client provide a distinction between header fields,
depending on where they are located.
When an S/MIME message is received, if the top-level protected MIME
entity has a Content-Type of message/rfc822, it can be assumed that
the intent was to provide header protection. This entity SHOULD be
presented as the top-level message, taking into account
header-merging issues as previously discussed.
RFC 822 provides a generalized description of how message headers of an email are composed and should be treated by systems they are transmitted through. RFC 5751 S/MIME 3.2 (btw, obsoleted by it successor RFC 8551 S/MIME 4.0) describes details how to use that standard to create encrypted emails.
So your approach to encrypt an email as described under My Encrypted Mail is valid and correct.
However, your approach as described under My Encapsulated message/rfc822 is not quite correct. You have obviously misinterpreted the RFC with regard to how to apply the rfc822 wrapper. The wrapper needs to be around your message before it gets encrypted, so it's going to be inside the encrypted part.
In your example, the unencrypted message (a slightly modified version Mail to be encrypted) would have to look like this:
MIME-Version: 1.0
Content-type: message/rfc822
From: sender#domain.com
Sender: senderdomain.com
To: recipient#domain.com
Reply-To: keepsecret#domain.com
Subject: A Secret Subject
Message-ID: <400b1383-362b-eed7-0719-6b2a2e231143>
Date: Mon, 24 Feb 2020 15:59:19 +0000
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha256; boundary="--_NmP-d017e0e3556f7bbc-Part_1"
----_NmP-d017e0e3556f7bbc-Part_1
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
My Message that will be encrypted
[...]
So you basically add the message/rfc822 to the message before it gets encrypted.
I have been able to verify this approach and tested the resulting message in two receiving mail clients with different results. With the macOS Mail application, the encrypted subject was not used to replace the unprotected "outer" subject, but at least, it was displayed prominently below the original headers. This is compliant with the RFC which is not very specific about the presentation:
It is up to the receiving client to decide how to present this "inner" header along with the unprotected "outer" header. Given the security difference between headers, it is RECOMMENDED that the receiving client provide a distinction between header fields, depending on where they are located.
An encrypted Reply-To header is displayed similarly, but it's email address is not honored when replying to that email.
Client Support
The support for encrypted headers in clients is somewhere between weak and non-existent. The results of some tests:
No client supports replacing the "outer" headers by the "inner" encrypted ones
Apple Mail (macOS) displays the inner headers prominently within the message
Thunderbird displays the encrypted part including its headers as a forwarded message
Outlook does not display the encrypted part, but instead confusingly displays just an empty message with an attachment (which is the encrypted message)
Alternative approaches
There is a seemingly promising approach proposed in this draft for Protected Headers for Cryptographic E-mail (work in progress). The idea is to include the protected headers as a separate part in a multipart message. This part will be rendered inline by agnostic clients, while at the same time, it can be properly processed by supporting clients.
We do send emails from our app with xls or pdf attachments to our business clients. Although they do receive the email including the attachment, they are very often not able to open the file, neither pdf or xls (Version Excel5). The file extension is not visible in the email anymore and the clients are not able to right-click and save the attachment.
The email body including attachment is generated with ZF2:
$html = new MimePart($content.$this->getSignature());
$html->type = "text/html";
$xlsAttach = new MimePart($xls);
$xlsAttach->type = 'application/vnd.ms-excel';
$xlsAttach->filename = $excelFilename;
$xlsAttach->encoding = \Zend\Mime\Mime::ENCODING_BASE64;
$xlsAttach->disposition = \Zend\Mime\Mime::DISPOSITION_ATTACHMENT;
$body->setParts(array($html, $xlsAttach));
I did google already, but without any noteworthy success. As this seems to be a problem with the clients IT infrastructure, there maybe is an experienced IT admin out there or anybody who can take a look at the email header.
The header which arrives is the following:
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="=_691d16bb3f9c6766e19e9bcbd8ef1861"
From: XXX
To: XXX
Subject: XXX
Cc: XXX
Sender:
X-bounce-key: XXX
X-HE-SMSGID: 1dsxY8-0005qz-Hb
X-HE-Virus-Scanned: Yes
X-HE-Spam-Level: +
X-HE-Spam-Score: 1.6
X-HE-Spam-Report: Content analysis details: (1.6 points)
pts rule name description
---- ---------------------- --------------------------------------------------
0.1 HTML_MESSAGE BODY: HTML included in message
0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
0.1 MISSING_MID Missing Message-Id: header
0.6 HTML_MIME_NO_HTML_TAG HTML-only message, but there is no HTML tag
Message-ID: <XXX>
X-HE-SPF: PASSED
Envelope-to: XXX
X-AVK-Virus-Check: AVA 25.14182;7B5CFE
X-AVK-Spam-Check: 1;str=0001.0A0C0208.59BC3F12.007C:SCFSTAT44510583,ss=1,re=-4.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0;8389744
#Dmitry Streblechenko: Thanks for the hint to Gmail.
Funny thing: My/Our Outlook 2010 adds the correct extension to the attachment received, that's why we never realized the issue on the client side :(. Through sending the Emails to the Gmail Account we saw, that the file extension is actually missing, and not extended while the file was generated. (ever trust your own system)
And, another + for Gmail: We were able to optimize our SPF.
I need to get an attachment out of an email received through a pop or imap adapter and process it further. The problem is that what I get out of that mail adapter is only the message of the mail(no title, no attachment).
Attachment should be possible acc. to this: https://uacp2.hana.ondemand.com/doc/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/08af047a0c0145ab8742b946cff7bcd7.html
As you can read: To access the mail attributes, i. e. Subject, From or To, you have to set them manually as Allowed Headers in the Runtime Configuration tab page. This will add them to a Whitelist.
The headers must be set in a Content Modifier as message header. I'll guess before receiving or sending the mail.
e.g. Accept | Constant | Type java.lang.String | Text/Plain
other values possible instead of Text/Plain:
Text/CSV
Text/HTML
Application/XML
Application/JSON
Application/Octet-Stream
Requirement: To send VB file of records length 100 as attachment using TCPSMTP utility with proper message in body without using IEBGENER utility.
I am trying to send email attachment file VB dataset as attachment. Its working either message in attachment or attachment file in body. But both simultaneously not working.
My JCL is:
//IRTCPN15 EXEC PROC=TCPSMTP
//SMTPIN DD DSN=EMAIL.CODE,
// DISP=SHR
// DD DSN=FILE.TOBE.SENTAS.ATTACH.MENT,DISP=SHR
Here, I have used both datasets EMAIL.CODE and FILE.TOBE.SENTAS.ATTACH.MENT of same specification VB 100 record length. I have also tried using boundary demiliter, but still its not working both together.
Dataset EMAIL.CODE contains:
HELO *******
MAIL FROM:<*******>
RCPT TO: <********>;
DATA
FROM: <******>
TO: <*******>;
SUBJECT: subject data
MIME-VERSION: 1.0
CONTENT-TYPE: TEXT/PLAIN
---Mail Body---
CONTENT DISPOSITION: ATTACHMENT; FILENAME=FILE.TXT
Please suggest me how to send this attachment with body. I have used asterisk due to security reasons. Please feel free to ask if any more information is needed.
In the EMAIL.CODE dataset, you're specifying that content-type of your message is text/plain. However, text/plain on its own (which is the default content type anyway) is always going to appear inline.
In order for the text in the message to be seen as an attachment, you need a Content-Disposition header that specifies attachment.
I can see in your question that you have a CONTENT DISPOSITION line, but it's labeled as being part of the message body. In addition to the fact that it needs to be a header, not a part of the body, it also needs to be hyphenated. So you should have CONTENT-DISPOSITION, not CONTENT DISPOSITION.
However, what all of this gets you is a message containing nothing but the attachment, and your question specifies that you want both a message body and an attachment. In order to do that, your Content-type at the top level needs to be multipart/mixed, and the body of the message needs to contain two MIME parts, one specified simply as being text/plain, and the other also text/plain, but with Content-Disposition: attachment.
This example shows the data for a MIME message containing both a text/plain body and a text/plain attachment.
FROM: <sender#example.com>
TO: <receiver#example.com>
Subject: TESTING message with body and attachment.
Mime-Version: 1.0
Content-type: multipart/mixed; boundary=MIME_BOUNDARY
This is the non-MIME body of a multipart message in MIME format.
Unless you are using a genuinely ancient email client or viewing
the raw source of a message, you should never see this paragraph.
--MIME_BOUNDARY
Content-type: text/plain
This is the inline text section of a multipart message
in MIME format. This is what will appear as the body
of your email when using any normal email client.
--MIME_BOUNDARY
Content-type: text/plain
Content-Disposition: attachment; filename=example.txt
This is the plain-text attachment.
--MIME_BOUNDARY--
.
I'm trying to sort out an opt in mailing list system. I understand the basic principles and design required but i'm having an issue with it being picked up as spam.
If i send a html email through outlook through email#domain.com it works fine and is not treated as spam. When i use the Zend_mail object to send mail it sends but is treated as spam on the test emails accounts i'm sending it too.
This is the code im using to send an email item.
//send an email
$mail = new Zend_Mail();
$config = array('auth' => 'login','username' => 'email#domain.com','password' => 'mypassword');
$transport = new Zend_Mail_Transport_Smtp('mail.domain.com', $config);
$mail->setSubject($item->title);
$mail->setFrom("email#domain.com");
$mail->addTo($item->email, $item->forename);
//$mail->setBodyText($item->contentPlain);
$mail->setBodyHtml($item->contentHTML);
$mail->send($transport);
As you can see im using the smtp transport object to authenticate but it still seems to treat this as spam. Anyone with pointers or tips is very much appreciated!!
Header info from the email that is treated as spam:
It seems to contain a couple of client domain names in the header info that i host for people any ideas why that would be the case? I use a shared IP address with about 10 domains on it
Received: (qmail 1436 invoked from network); 14 Aug 2009 16:02:10 +0100
Received: from clientdomain1.co.uk (HELO localhost) (91.192.***.196)
by clientdomain2.info with SMTP; 14 Aug 2009 16:02:10 +0100
Subject: Manchester 2 Day Seminar: Dealing with difficult people
From: events#domain.com
To: Andi <subscriber1#domain.com>
Date: Fri, 14 Aug 2009 15:02:10 +0000
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
MIME-Version: 1.0
"Roll your own mail" is often treated as spam by large hosted email systems. When you use a paid service to send out mass emails, you are paying for those company's agreements with the major email vendors to keep them white-listed.
One thing you can do, though, is to ensure that the account you are sending from exists and the email is being sent from a matching domain (e.g. #foo.com sent from foo.com's smtp server). That is a big red flag for spam filters.
Compare your email and email from outlook. Are any headers missing? Which? Do they seem significant?
Try this to get rid of the last localhost reference:
$protocol = new Zend_Mail_Protocol_Smtp('localhost');
$protocol->connect();
$protocol->helo('mail.yourserver.com'); //**DO THIS**
$transport->setConnection($protocol);