Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
I use SendGrid and I have 2 dedicated IPs... My website sends its emails using SendGrid.
At the same time I use G Suite and I personally send my emails using G Suite (Google Workspace)
I want to setup DKIM and SPF records. SendGrid documentation has an option called Automated Security:
Automated security allows SendGrid to handle the signing of your DKIM
and authentication of your SPF for your outbound email with CNAME
records. This allows you to add a dedicated IP address or update your
account without having to update your SPF record.
and later on...
When Automated Security is On, SendGrid generates 3 different CNAME
records. In a later step of setting up domain authentication, you give
these records to your DNS provider, and then you verify that they
upload correctly.
If you select Off, we generate 1 MX record and 2 TXT records. In a
later step of setting up domain authentication, you give these records
to your DNS provider, and then you verify that they upload correctly.
So When Automated Security is ON I won't need to add any SPF and DKIM record, instead I have added the 3 CNAME records that is required for Automated Security...
On the other hand I am also using G Suite and G Suite wants me to add DKIM and the following SPF record:
v=spf1 include:_spf.google.com ~all
From what I understand, the above SPF record means that _spf.google.com is the only server that should be sending my emails...
So my question is: Is adding G Suite SPF record conflicting with SendGrid Automated Security? My SPF record is saying the My SendGrid's dedicated IPs are not allowed to send email?
Imagine Yahoo mail server receives an email from one of my dedicated IPs (SendGrid)... how would Yahoo know that I am using Automated Security and the SPF record is handled by SendGrid? Because If Yahoo checked the SPF record, the SPF record is telling it that only _spf.google.com is allowed to send my emails?
Just a note: Your question would probably do better at the Server Fault forum.
To answer you questions:
No the SPF record where you list G Suite is not conflicting with your SendGrid setup.
What is important to understand is: SPF says absolutely nothing about who is allowed to send emails FROM your domain. SPF authenticates the Return-Path address instead, and you're creating a subdomain for the bounce messages (what the Return-Path header is used for) with the CNAME delegation records in "Automated Security".
So, basically, Sendgrid is asking you to create a sub domain for them, e.g. em123.yourdomain.com by means of CNAME record. You delegate that subdomain over to them, targeting your personal Sendgrid tenant zone in DNS, where they create an SPF record and MX record at the root of that zone. A typical email from Sendgrid will then look like this:
from: you#yourdomain.com, Return-Path: bounces#em123.yourdomain.com
Thus, the receiving server will check the SPF record at em123.yourdomain.com to see if your IPs are allowed.
The other two CNAMEs are for the DKIM selector records. They point to TXT records that Sendgrid manages and rotates periodically.
So why not have Sendgrid use their domain in the Return-Path and not bother with the CNAME setup at all?
DMARC demands that your Return-Path domain aligns with your FROM domain, OR that the domain used in your DKIM signatures aligns with your FROM domain. Preferably both.
Related
If I have a domain example.com that is using gsuite (DNS settings at registrar has gmail cnames, spf & txt records etc) and I have another service sending on behalf of the domain (Klaviyo). Do the gmail DKIM and DMARC settings help to strengthen the deliverability of those emails sent by the other service (Klaviyo)?
To answer your question: A DMARC reject or quarantine policy helps improve deliverability for all parties that send on behalf of your domain AND properly authenticate by SPF or DKIM, in alignment with your domain.
DKIM consists of a cryptographic key pair. You publish the public key on the Internet and you use the private key to sign headers of your outbound emails. This signing is done on the sending server. So unless Klaviyo is using Google servers to relay your messages, those messages are not being DKIM signed by Google.
You should follow the instructions provided by Klaviyo here, so that the emails you send from their platform, using your email domain, will authenticate properly and will NOT fail DMARC.
Update:
Say you own the domain myexample.com, then you should publish a TXT record at the root of that domain that looks like "v=spf1 include:_spf.google.com ~all". Additionally you can add any other services or servers to this record as you see fit. You don't need to add Klaviyo to your SPF record as they will try to authenticate from the send.myexample.com domain used in the bounce address. That is what you created the first CNAME for. It redirects to an SPF (and MX) record hosted at Sendgrid. Additionally, Klaviyo will authenticate those emails using DKIM.
In order to make DMARC work, you need to publish another TXT record at _dmarc.myexample.com, if you haven't already, looking like: "v=DMARC1;p=none;rua=mailto:DMARC#myexample.com;". Then you'll start receiving aggregate reports at the mailbox you supplied. Once you're confident you've included all required parties in your authentication scheme, you can move to a p=reject policy in order to protect your domain.
Yes, DKIM and DMARC settings do help deliverability.
I assume that Klaviyo does what my company Autoklose is doing as well, and that's using Gmail API to send the email in your name. That means that they only indirectly affect the sending process and the email itself is sent from Google servers and not Klaviyo's servers.
Also, you have to be aware that DKIM & DMARC are only two of the factors in successfully delivering your email. For example, having DKIM & DMARC correctly set gets you positive points but if your domain is blacklisted, it still might not get delivered.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm using a fair bit of email forwarding on a number of domains and the latest p=reject policy of AOL is causing me some problems and also a lot of confusion. My understanding of DMARC is that it's based on DKIM & SPF with a reporting layer. I understand that SPF is a problem with forwarding but as long as the SPF is set to ~all soft fail then that isn't a show stopper. I also thought DKIM could pass through forwarding without problems as long as you don't mess with the headers much. However I'm finding that certain emails from AOL being forwarded by MailGun are failing DMARC when they land at GMail. MailGun say its due to a sender/from mismatch error. Can anyone elaborate on whether email forwarding is doomed as DMARC takes hold or are MailGun just not forwarding properly?
I think the problem you are coming accross is not the forwarding, but the DKIM alignment. Yes, SPF will Fail, but DKIM should Pass, but the DKIM Alignment might not pass. DMARC requires alignment to be strict or relaxed, based on your policy. You can read up more on alignments here: Identifier Alignments
There are testing tools like sending an email to mailtest#unlocktheinbox.com that will show the alignment on the results, but it's a paid feature.
Since this is a problem, there is a new specification in the works called "ARC" the intent of this is to solve this very problem, you can read about that here: ARC Spec
I asked Mailgun support about this problem and was not given any helpful answers. However, the solution is actually in their documentation:
Note If you forward messages to another email address, then you should
disable click tracking, open tracking and unsubscribes, by editing
your domain settings in the Control Panel. If these features are
enabled, the content of each message is modified by Mailgun before
forwarding, which invalidates the DKIM signature. If the message comes
from a domain publishing a DMARC policy (like Yahoo! Mail), the
message will be rejected as spam by the forwarding destination.
https://documentation.mailgun.com/en/latest/user_manual.html#routes
So, I turned off all of those features on my main domain which I use to receive mail and that solved the DMARC bouncing. If you want to use any of those features, you will need to setup a sub-domain for outgoing mail.
As mentioned above, the issue is that upon sending an email somedomain.com stamps the email with a DKIM (a unique hash of the email's content). When Mailgun forwards the message to Gmail, it inserts it's open/click/unsubscribe modifications which then invalidates the DKIM hash.
Because somedomain.com DMARC says "reject anything that has had the DKIM invalidated" Gmail or other services reject the email.
The workaround I found was to to use a native mailbox solution. WhoIs comes with one which is easy to setup for example: https://manage.whois.com/kb/servlet/KBServlet/faq579.html
Afterwards I simply added the POP mailbox in Settings of Gmail. (As of Gmail December 24, 2017). I was able to re-enable Open and Click tracking and now everything works and I happily get my mails in Gmail still.
As a bonus, we use the email as a shared inbox so now everyone can see the sent emails in their inbox instead of just the forwarded responses.
[Edit] After doing this I realized I was now only getting my emails in the native mailbox solution but no longer in Mailgun where I also needed them.
To work around this, I added MX records to point to mxa.mailgun.org using a subdomain, so mg.exampledomain.com
Then I used a forwarding rule in WhoIs to send all emails that are delivered to info#exampledomain.com to also be delivered to info#mg.exampledomain.com. Now I get the messages in Mailgun and am still able to see it in Gmail through the POP mailbox.
In my case it turns out this was simply caused by invalid DKIM signatures from reputable senders, despite the fact that the forwarders are NOT modifying the messages.
Without a forwarder in between, SPF might have passed and allowed the email to go through anyway, but given SPF won't pass here either (since you're not an authenticated sender), DMARC tells the the receiver to hard-fail, and then your forwarder might receive an error, such as this one from Gmail:
5.7.1 Unauthenticated email from example.com is not accepted due to domain's DMARC policy.
Please contact the administrator of example.com domain if this was a legitimate mail.
Please visit https://support.google.com/mail/answer/2451690 to learn about the DMARC initiative.
- gsmtp
The key phrase here is unauthenticated email, meaning email that doesn't have a valid signature. Authenticated email, on the other hand, could have been allowed from your domain.
You can verify that the issue is an invalid DKIM signature as follows:
$ pip install dkimpy # dkimpy supercedes pydkim
$ dkimpy -v < message.eml
Traceback (most recent call last):
File "/usr/sbin/dkimverify", line 10, in <module>
sys.exit(main())
File "/usr/lib/python/site-packages/dkim/dkimverify.py", line 41, in main
res = d.verify()
File "/usr/lib/python/site-packages/dkim/__init__.py", line 869, in verify
return self.verify_sig(sig, include_headers, sigheaders[idx], dnsfunc)
File "/usr/lib/python/site-packages/dkim/__init__.py", line 696, in verify_sig
(base64.b64encode(bodyhash), sig[b'bh']))
dkim.ValidationError: body hash mismatch (got b'...', expected b'...')
Unfortunately, if this happens, you don't have great options at your own disposal:
Send a separate delivery failure message from your own domain, and include the original email as an attachment. If you do it properly, Gmail (and possibly other receivers) can actually display the attachment inline as a forwarded message.
Replace the "From" address with your own, then sign with your own DKIM key.
Perhaps include the original email as an attachment so the receiver knows what it was.
If the receiver is yourself, then write a script to download the email and shove it directly into your inbox (say, either via IMAP, or directly with an API).
Tell the domain owner and hope that they fix it.
(Though, if they haven't already, good luck convincing them the problem is on their side.)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
I have a domain mydomain.com
I need to use this domain on multiple servers.
I have greengeeks shared hosting as well as AWS with few EC2 machines for my web applications.
mydomain.com is hosting a marketing site via greengeeks hosting and it also provides my email server.
I want one sub domain app.mydomain.com that will point to one of my ec2 instance, another subdomain appadmin.mydomain.com point to another ec2 instance, What is the best way to setup domains and sub-domains like this. (currently i am using different domain (mydomain.org) for my aws instance, which isn't elegant)
As my mail server is on greengeeks, and I programatically send email from my ec2 app server using an email id app#mydomain.com, some email clients shows warning as "cannot verify email origin" and some pushes it in spam folder, is there a solution to it, I read about MX and SPF record but I am confused how to create them and where to put them.
Thanks,
K
I want one sub domain app.mydomain.com that will point to one of my ec2 instance, another subdomain appadmin.mydomain.com point to another ec2 instance, What is the best way to setup domains and sub-domains like this. (currently i am using different domain (mydomain.org) for my aws instance, which isn't elegant)
I'm not sure what the question is about. You should be able to make an a-record for each site. One for app.mydomain.com and another appadmin.mydomain.com.
As my mail server is on greengeeks, and I programatically send email from my ec2 app server using an email id app#mydomain.com, some email clients shows warning as "cannot verify email origin" and some pushes it in spam folder, is there a solution to it, I read about MX and SPF record but I am confused how to create them and where to put them.
SPF records are made typically made in either TXT-records, or SPF-records if your DNS provider, has a specific section for this. Both should work.
TXT records are nothing more than a record, with some text in it, but it can be interpeted as a SPF record, if specified as one (see part 1).
SPF entries are a collection of trusted IPs/hosts, that a spamfilter or mailserver can use to validate a the IP adress of the sender.
In the email message header, it will always say where the email was sent from(cant be spoofed to my knowledge). A spamfilter will compare the sender to the SPF entries of the specific domain and decide weather to reject the message or accept it.
Lets look at one example.
v=spf1 include:1.mailserver.com include:2.mailserver.com ip4:99.99.99.99 include:relay.anothermailserver.com ?all
Part 1
Specifies the version of SPF that you're using. SPFv1 should be able to do the trick for you.
v=spf1
Part 2
You specify the mailservers/IP subnets that should be allowed to send from your domain.
include:1.mailserver.com
include:2.mailserver.com
ip4:99.99.99.99
include:relay.anothermailserver.com
Part 3
Specifies weather to mark message as (~)SoftFail, (-)HardFail, (+)Pass or (?)Neutral, when the sender is not specified in your entries.
The receiving mailserver decides, weather to reject or accept a message based on their own policy. Meaning that some spamproviders/mailservers will reject a softfail, and some still may accept it.
Generally a hardfail should get rejected.
?all
If you need some material on this subject, id sugest www.openspf.org/SPF_Record_Syntax for the syntax.
Also you can use mxtoolbox.com which has a great tool, that specifies the different parts of a record.
As for the MX records. The MX specifies the receiving mailserver and in which order it should try to send the messages to them.
Example
Sending a mail to administrator#mydomain.com
MX record for mydomain.com
Priority = 10 / 99.99.99.99
Priority = 20 / server1.mymailserver.com
Priority = 30 / server2.mymailserver.com
Priority = 5 / myloadbalancer.mymailserver.com
Here it will look up the MX-records for mydomain.com, find the MX with the lowest priority and send the message to that IP/Host. If it doesnt get an answear from the server, it will proceed to try and send to the next MX-entry. If the message is accepted by receiving mailserver, then the mail goes through. If no answer is given by any of the MX-entries, it will notify the sender address, that the message could not be sent.
Please let me know if i missed something.
/AG
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 4 years ago.
Improve this question
I have recently acquired a domain name via Google Domains. I have set some configuration to have it point at an OpenShift application via Cloudflare. Cloudflare requires me to set their DNS servers, which I did in Google Domain.
At Cloudflare, I have created two CNAME records (and nothing else). One is an alias from my mydomain.com to some.url.at.openfshit.com, and the other is from www to mydomain.com.
Yet, within Gmail Domain, I have also set an email using my domain name which is to be forwarded to a private email. But, I don't receive any emails when testing.
I am wondering whether I could have my emails forwarded properly. Is it a matter of creating a MX record at Cloudflare? If yes, with what configuration?
P.S.: I have set a MX record using instructions available here, but I get:
Delivery to the following recipient failed permanently:
contact#mydomain.com
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain chartvibes.com by aspmx.l.google.com. [2607:f8b0:4001:c20::1b].
The error that the other server returned was:
550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient's email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 https://support.google.com/mail/answer/6596 p123si522326ioe.111 - gsmtp
The MX records you're using are for G Suite accounts. You can still forward emails with Cloudflare and Google Domains, but you'll need different MX records. As Overdrivr pointed out in a comment below, you can find your MX records in the DNS settings in Google Domains. Once you're in the DNS settings page, look for a collapsible panel called "Email forward" under the "Synthetic records" section. You should see something like this
Then, make a backup of your Cloudflare DNS setup, erase all MX records and add the ones listed in your account using the number right before the mail server (e.g., 5, 10, etc.) as its priority.
It might take a few minutes for the changes to take effect. If you try to send an email right after changing the records, it's likely that you'll get a message saying that the address could not be found, but it'll have the G Suite mail server in the Remote-MTA field (aspmx.l.google.com) instead of gmr-smtp-in.l.google.com. If this is the case, just wait for a few more minutes and try again
I'm not sure if you already have a solution to this, but if you do, I'm interested in how to do it too. Could you please post your solution here if you find one ?
The bad news is, it cannot be done because the way Google Domains work. Google Domains has email forwarding, but it works only when you're using Google's DNS servers. It's the same with all hosting services or whatever they're called.
I think Google just has an email forwarding service that can forward upto 100 alias email addresses per domain to an actual email address. But the actual email address has to exist somewhere. The ones you set up in the Domains console are just aliases or forwarding instructions.
For Cloudflare email forwarding to work, you need to use the SMTP servers where the actual email addresses exist, but since Domains has no actual email service servers, the emails sent out are failing with email account does not exist. The instructions you mentioned are for the Google Apps, which have actual email/gmail addresses set up, but they cost $5/user/month.
The only solution that I can think of to get around this issue is to have our own mail server, and have cloudflare point to those, and then forward/deliver the emails from that mail server.
Hope this helps.
EDIT :
I probably didn't research this well enough before, but looks like people are getting around this issue by using a third party email forwarding service called mailgun
The actual article describing how to use it is on lowendtalk
Some discussion surrounding it is here
I am trying to help out a little non-profit organization, who has decided to let One.com host their domain, including website and e-mail. Now, my issue is that One.com does not add SPF-records or DKIM-keys to your domain and I believe that is the reason why a large number of mails sent from the domain, end up in spam.
I've been in touch with their support, who kindly answered:
You are already using our mail servers, there is no need to use SPF for that.
Our mail servers already have SPF installed, and if you are using our mail servers, SPF will not be question since domain is hosted here and it is using One.com's mail server. SPF will only be required if your domain is hosted here but is using a different MX record or mail server
I've tried to figure out if you can leave out SPF, but all I've been able to conclude is that proper SPF on each domain is definitely the proper way, instead of just the hosting companys main domain. I mean, if it was that simple, how come even Google Apps, Zoho, Rackspace etc. recommends adding SPF, if it worked just as well leaving it out - you'd be using their MX as well, so isn't that the same? And wouldn't leaving SPF out leave us with the same issues as before SPF, namedly that you'd have no way to validate if mail was truly being sent from the owners of the domain or just somebody imposing.
So what it comes down to: Can One.com really leave out SPF records on their clients domains, send mail on the clients behalfs and still expect mail to come through without ending up in spam more often?
Thank you very much for your time!
The short answer is "No, they can't". The longer answer is a little more complicated.
SPF uses either the EHLO domain of the sending server or the domain in the Return-Path to look up SPF records in DNS. Most systems that handle multiple domains do not use SPF records on the EHLO domains of the sending servers, so the SPF domain is taken from the email's Return-Path. You should take a look at the Return-Path for one of the emails that this non-profit has sent through One.com to determine whether the Return-Path is on a subdomain of one.com, or is using the non-profit's domain. The latter is definitely preferred.
If the Return-Path is on a subdomain of one.com, then that's the domain that will be used to look up SPF records. So adding SPF records to your non-profit's DNS won't do anything. While this may seem the easier path, it causes problems with DMARC and may cause the email to be flagged as spam even if it passes SPF, as the address in the 'From' header will have a domain that doesn't match the Return-Path
If the Return-Path is on a subdomain of your non-profit's domain, then you should definitely add an SPF record to your non-profit's DNS. Looking at one.com's current records, something like:
v=spf1 include:_spf.one.com ~all
should do it.
By the way, you should be able to see whether an email has been SPF or DKIM authorized by looking at the headers of the received email. That's the best way to understand the actual behavior.