I have a web application with email notifications.
I'm using PHPMailer to send mails over authenticated SMTP by the SMTP server given by my provider.
All messages are invalidated by SPF and DKIM settings because somehow it is expected that my webserver is included in the SPF list and PHPMailer has to sign the message.
Is there any way to let the the SMTP server to envelope and sign the email for me?
I tought an authenticated SMTP connection will define the first server as enveloped-by and not the first client. Is this a wrong tought or a misconfiguration of the SMTP server?
Whether your mail server will DKIM-sign for you is up to the mail server – and it is generally preferable to do that as you don't then have to worry about it.
However, it is definitely better to sign it yourself if you can. If you're sending through your hosting provider's mail server, they would need to have access to your private key in order to be able to sign as your domain. For this reason, if they do sign, it is often as an intermediary rather than as the original sender, and this may mean they can't vouch for the From address which you would need to be signed to pass DMARC checks. You can do both at once, because it's entirely legitimate for a message to contain multiple DKIM signatures, signed at different points of the message's journey, by different parties.
SPF does include the originating IP, but you need to be aware of the difference between "regular" SMTP and the SMTP submission protocol defined in RFC6409, and specifically mentions that you might expect a submission server to add a signature. Submission clients will often not be treated as the origin (from an SPF perspective). If that server is in your own network, then you have no issues with holding private keys.
Related
I have multiple domains pointing at nameservers of a shared hosting platform and manage all the emails of these domains using just one address called hello#domain-a.com as an example.
Because all of these domains work under one business and I don't want to manage multiple accounts, this address also receives all the emails for my other domains domain-b.com and domain-c.com using forwarders.
Sometimes I still need to change the From address, which is why I set up identities in Thunderbird using domain-a.com as the host with hello#domain-b.com and hello#domain-c.com as the From addresses. While doing that, I realized that I didn't need to create these accounts on the mail server and all emails that were sent by either hello#domain-b.com or hello#domain-c.com had a PASS on both SPF and DKIM.
This made me wonder, whether someone else on the same shared hosting platform (whois checkup shows there are more than 600 other domains) could also just use any of my domains to send emails and would get a PASS on both SPF and DKIM, basically making it useless in that specific scenario.
Can anyone on the same shared hosting send emails using my domain with a PASS on both SPF and DKIM?
Why is it possible to use domain-a.com as the host and hello#domain-b.com or hello#domain-c.com as From addresses and still send messages that way? The test messages I have sent using that method don't show domain-a.com but the domain of the From address in their headers. Is this normal behavior? Since I didn't set up any individual accounts for domain-b.com and domain-c.com, I honestly didn't expect this to work.
Thanks in advance.
If someone is on the same shared host as you, they will likely have the same IP address, and so if your SPF includes that IP, they will indeed be allowed to send messages from your domain. The simple solution to that by itself is don't do that – don't put critical content on shared hosting.
The next step is DKIM. DKIM has nothing to do with IP addresses, and sending from a shared host will have no effect, so this is your best defence against this kind of spoofing. A DKIM signature is signed by your private key (which nobody else should be able to access, even on shared hosting – though it is far more at risk there than on your own server). Because they can't see your key, they can't produce DKIM signatures for your domain, and thus messages they try to send from your domain will not contain a valid DKIM signature and they will not be able to get a DKIM pass result.
It could be that your hosting provider is also adding a DKIM signature of their own as an intermediary, though it will match their domain, not yours. You can add your own as well as messages can contain more than one DKIM signature.
To force alignment of From address with the envelope sender (which ends up in a return-path header at the receiver), you need to look at DMARC, specifically the adkim and aspf parameters.
Another approach would be to not send mail through your hosting provider but via some other mail server. This would let you have an SPF record that does not include your hosting provider's IPs. You may find however that you can't do this because low-end hosts tend to block outbound SMTP.
If I have a domain example.com that is using gsuite (DNS settings at registrar has gmail cnames, spf & txt records etc) and I have another service sending on behalf of the domain (Klaviyo). Do the gmail DKIM and DMARC settings help to strengthen the deliverability of those emails sent by the other service (Klaviyo)?
To answer your question: A DMARC reject or quarantine policy helps improve deliverability for all parties that send on behalf of your domain AND properly authenticate by SPF or DKIM, in alignment with your domain.
DKIM consists of a cryptographic key pair. You publish the public key on the Internet and you use the private key to sign headers of your outbound emails. This signing is done on the sending server. So unless Klaviyo is using Google servers to relay your messages, those messages are not being DKIM signed by Google.
You should follow the instructions provided by Klaviyo here, so that the emails you send from their platform, using your email domain, will authenticate properly and will NOT fail DMARC.
Update:
Say you own the domain myexample.com, then you should publish a TXT record at the root of that domain that looks like "v=spf1 include:_spf.google.com ~all". Additionally you can add any other services or servers to this record as you see fit. You don't need to add Klaviyo to your SPF record as they will try to authenticate from the send.myexample.com domain used in the bounce address. That is what you created the first CNAME for. It redirects to an SPF (and MX) record hosted at Sendgrid. Additionally, Klaviyo will authenticate those emails using DKIM.
In order to make DMARC work, you need to publish another TXT record at _dmarc.myexample.com, if you haven't already, looking like: "v=DMARC1;p=none;rua=mailto:DMARC#myexample.com;". Then you'll start receiving aggregate reports at the mailbox you supplied. Once you're confident you've included all required parties in your authentication scheme, you can move to a p=reject policy in order to protect your domain.
Yes, DKIM and DMARC settings do help deliverability.
I assume that Klaviyo does what my company Autoklose is doing as well, and that's using Gmail API to send the email in your name. That means that they only indirectly affect the sending process and the email itself is sent from Google servers and not Klaviyo's servers.
Also, you have to be aware that DKIM & DMARC are only two of the factors in successfully delivering your email. For example, having DKIM & DMARC correctly set gets you positive points but if your domain is blacklisted, it still might not get delivered.
I have an email address forwarding to a gmail account. I then use SMTP to send a response from gmail via the domains SMTP server. This is all set up fine. However some recipients are not receiving the emails? Is there further configuration I need to do on the domain side?
I am told I need to configure the SPF, DKIM and DMARC records but I have no idea what the configuration/values should be?
Having SPF, DKIM and DMARC set up is seldom a prerequisite for having your email delivered. If your email domain and servers have a decent reputation, you won't, generally, run into to much trouble.
However, it is best practice to set up all three, to start authenticating your emails and making it harder for others to impersonate your email domain without authorization. I'll outline the basics for you:
Why Authenticate
Phishing: Email Authentication will make it harder to impersonate your email
domain, without authorization. It (somewhat) protects your colleagues, partners and customers against phishing.
Brand Reputation Protection: Phishing from your domain can harm the reputation of your brand.
Deliverability: Authentication improves deliverability because it's weighed heavily in determining whether or not the email is legit.
DMARC
DMARC will try to find successful authentication for servers sending on your behalf. Specifically, it will look for a Pass on SPF or DKIM, in alignment with the email address (domain) that is being showed to the recipient in his email client. This is known as the Header.From field. (Not to be mistaken with the Sender field, the Reply-To field or Return-Path).
SPF
SPF is basically a list of IP addresses, published as a TXT DNS resource record, listing all servers that are authorized to send email for the domain the record lives in. This does not include subdomains, those require additional SPF records. One of the (many) problems with SPF: Receiving servers need to check the Return-Path email address to lookup the SPF record, instead of the Header.From domain. There is no need for the Header.From email address and the Return-Path address to share any of the domain part, according to the SMTP RFC. Thus where DMARC comes in.
DKIM
Signing an email message with a DKIM private key, requires you to publish a matching public key in the subdomain _domainkey for the domain your signing for. The receiving server will look for d= value and the s= value in the DKIM signature to construct the correct DNS TXT resource record to query, holding the public key. Example d=stackexchange.email s=s1 will result in a DNS query for the TXT record s1._domainkey.stackexchange.email. The same applies here as with SPF: The d= value does not have to match with the domain portion of the Header.From email address.
Unfortunately the configuration and values are very specific, depending on which parties are allowed to send on behalf of your domains, the subdomains you use and how you use them, etc. Especially SPF has a few limits that will make the setup harder.
I am using AWS SES and PHPMAILER to handle email sending in my system.
My question is, does the SES SMTP server automatically sign my emails with my DomainKey (DKIM) or do I have to do it manually using PHPMAILER?
On one hand, Amazon declares that they sign emails with DKIM, but on the other hand I see that you can add headers and sign emails with DKIM in PHPMAILER, so I don't really understand whose responsibility it is to sign my emails with it.
If you set up SES's Easy DKIM, all messages sent through SES will be automatically signed. You don't need to bother with having PHP handle that.
If you're sending email from a single domain (or small number of them) this is the way to go. If you are doing something tricky (like sending email on behalf of hundreds or thousands of domains) it might be easier to handle the DKIM signing in PHP.
When setting-up my mobile email client, I usually need to enter the SMTP relay server domain name, but why my email client need a SMTP relay server at all?
I think it can directly talk to the smtpd server (e.g. postfix) from which domain the destination Email address is, by looking up the DNS MX records. And send directly to it, why not?
Actually I have another confusion, if a smtpd server use SMTP AUTH, then how can a rely server elsewhere rely a mail to it? The rely server simply don't have the username and password necessary to transfer that mail.
Email clients use SMTP relay server to
simplify email sending (e.g. to avoid repeating delivery attempts by email client)
avoid being mistaken for spammers.
Direct email sending might be a reasonably simple option in ancient pre spam past.
SMTP AUTH is used (mainly) with email client to email server communication. SMTP server to SMTP server communication could not require SMTP AUTH without prohibiting email from "strangers". SMTP AUTH allows to accept messages from trusted clients (customers) even from IP addresses that otherwise would be blocked by DNSBL (e.g. DUL ranges).
A SMTP server these days does many more things:
Queues the emails and works on delivering them: this may not matter if you just send a single email here and there, but it's a different thing for a 500k recipients newsletter.
Retries messages than can not be delivered immediately (e.g., slow receiving server), and eventually bounces them if they cannot be delivered. According to RFC 5321 once a SMTP server accepts responsibility of an email message, it must not lose it but either deliver or return (bounce) it.
DKIM sign the message.
Route emails to go out from different sending IP addresses and host names, for deliverability purposes. (Email reputation is based a lot on sending IP addresses.)
Throttle delivery of large amounts of emails to avoid being seen as an aggressive sender and getting blocked.
Optionally, archive (or bcc) all outgoing email, for documentation or compliance purposes.
Of course you can also do all these things in your email sending application and then you don't need a SMTP server, then you wrote an SMTP server.