I have multiple domains pointing at nameservers of a shared hosting platform and manage all the emails of these domains using just one address called hello#domain-a.com as an example.
Because all of these domains work under one business and I don't want to manage multiple accounts, this address also receives all the emails for my other domains domain-b.com and domain-c.com using forwarders.
Sometimes I still need to change the From address, which is why I set up identities in Thunderbird using domain-a.com as the host with hello#domain-b.com and hello#domain-c.com as the From addresses. While doing that, I realized that I didn't need to create these accounts on the mail server and all emails that were sent by either hello#domain-b.com or hello#domain-c.com had a PASS on both SPF and DKIM.
This made me wonder, whether someone else on the same shared hosting platform (whois checkup shows there are more than 600 other domains) could also just use any of my domains to send emails and would get a PASS on both SPF and DKIM, basically making it useless in that specific scenario.
Can anyone on the same shared hosting send emails using my domain with a PASS on both SPF and DKIM?
Why is it possible to use domain-a.com as the host and hello#domain-b.com or hello#domain-c.com as From addresses and still send messages that way? The test messages I have sent using that method don't show domain-a.com but the domain of the From address in their headers. Is this normal behavior? Since I didn't set up any individual accounts for domain-b.com and domain-c.com, I honestly didn't expect this to work.
Thanks in advance.
If someone is on the same shared host as you, they will likely have the same IP address, and so if your SPF includes that IP, they will indeed be allowed to send messages from your domain. The simple solution to that by itself is don't do that – don't put critical content on shared hosting.
The next step is DKIM. DKIM has nothing to do with IP addresses, and sending from a shared host will have no effect, so this is your best defence against this kind of spoofing. A DKIM signature is signed by your private key (which nobody else should be able to access, even on shared hosting – though it is far more at risk there than on your own server). Because they can't see your key, they can't produce DKIM signatures for your domain, and thus messages they try to send from your domain will not contain a valid DKIM signature and they will not be able to get a DKIM pass result.
It could be that your hosting provider is also adding a DKIM signature of their own as an intermediary, though it will match their domain, not yours. You can add your own as well as messages can contain more than one DKIM signature.
To force alignment of From address with the envelope sender (which ends up in a return-path header at the receiver), you need to look at DMARC, specifically the adkim and aspf parameters.
Another approach would be to not send mail through your hosting provider but via some other mail server. This would let you have an SPF record that does not include your hosting provider's IPs. You may find however that you can't do this because low-end hosts tend to block outbound SMTP.
Related
I'm using PHP mail() to send an email from my server to two different accounts, one which is my Gmail account, which SPF passes with, and one to an account hosted by my domain provider, which is then forwarded to my Gmail account. That causes SPF to fail because the originating IP is different.
But, there's no way to tell if the email address you send emails to is the recipient, or forwards them elsewhere. So is there any way to allow SPF to pass through any (unknown) relay?
It's unclear exactly what you mean here. Are you using arbitrary From addresses?
Generally, SPF control over email sources is handled in a few different ways:
Authorise your domain provider's servers to send from your domain (i.e. add them to your SPF record)
Hope that your hosting provider's mail servers support SRS, the Sender Rewriting Scheme, which they should
Allow any IP to be a source of email for your domain by adding +all to your SPF record (clearly a bad idea!)
I'm currently testing some complicated functionality in a web app that involves multiple accounts within the app; each with its own unique email address. I have a few temp gmail accounts that I'm using for this purpose, but keep having to update email addresses in the database to something else for some accounts in order to reuse email addresses. It gets meesy very quickly.
My question is; is there a way I can setup an email server on my VPS (Ubuntu 16.04) that is able to accept all email addresses on a given domain without me having to define each one in advance? Ie; I'd like to be able to make up email addresses as I go; test1#mydomain.com, test2#mydomain.com, etc, and have the server receive AND store any incoming emails to those addresses. Emails include account verifications, notifications, etc, and I need to know what is being sent to each participant at each point in each workflow as I test.
Ideally I'd like some way to view all of these emails on the domain regardless of address. Perhaps setting up an alias using some kind of wildcard so that all incoming email go to a single address that I can configure in my email client on my dev machine.
Consider rewriting envelope recipient x#subdomain.example.net to handler+x#example.net
AFAIR sendmail, postfix and exim support such rewrites. AFAIR all three support passing +detail as parameter to procmail script (to ~handler/.procmailrc as $1).
WARNING: test handling message to multiple special recipients.
Post which specific SMTP/MTA server do you use if you want more hints.
Gmail also supports plussed addresses BUT you get reliable hint about original recipient only for message to single special/plussed recipient.
After a lot of research and messing about with different configurations, I found the best solution for me was PostFix with Dovecot.
I was able to set up virtual mailboxes in PostFix with virtual_mailbox_maps pointing to a file that contained this; #mydomain.com mydomain.com/catchall/
This allows mail to any email address within the domain to go to one mailbox. The mydomain.com/catchall/ is a location on the server where the mailbox is located. By convention, the domain is used in the path. Useful if you are hosting email on multiple domains. Dovecot supports variables in the config which allows you to point it to the correct mailbox.
I also found that the PostFix main.cf file must have mydestination = localhost for this configuration to work.
the following posts helped me getting this up and running;
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot
https://www.digitalocean.com/community/tutorials/how-to-install-and-setup-postfix-on-ubuntu-14-04
We are building a system that is, effectively, an email/calendar/contact client.
Users will provide us with their email address password (or other auth, eg oauth) and we will connect to their underlying email system.
"underlying systems" include:
Microsoft Exchange / Office365
GMail
Yahoo Mail
Apple email
Generic IMAP
Each of these systems have subtle (and not so subtle) differences in their APIs, especially to access calendar & contact data. Thus we need to know what provider the user is using.
But we would rather not ASK the user. We would like to figure it out (and at, least reduce the choices) automatically.
I've looked around for something that already implements this but have not found anything. I know it's mostly possible because Windows Phone does it pretty well (just enter username/pw and it does the right thing).
Before I dive in and start writing my own I want to ensure I'm not wasting my time if someone's already done it in an excellent way.
Know of anything like this? For this project prefer C#/.NET.
[EDIT: Adding potential algo]
Potential Algorithm:
Given email address & password
Extract domain name from email address
Try Exchange autodiscover. If successful done.
Use DNS MX records to find smtp host.
Do SMTP EHLO
Gmail responds with "250-mx.google.com at your service"
So if we see a "google.com" we are done.
Yahoo responds with 250-mta1257.mail.sk1.yahoo.com
So if we see a "yahoo.com" in the response we are done.
Apple responds with 250-xxxxxx-mac.com
So if we see a "mac.com" in the response we are done.
If none of the above
IMAP?
...
[EDIT: 5/18]
I built a prototype that uses methods that don't require auth (e.g. just MX/SMTP sluthing). Give it a try: http://bit.ly/KLZKxD
Algorithm seems reasonable. You will get best results running from unfiltered server (meaning it has direct SMTP outbound / doesn't run through a proxy). If running from client (mobile/tablet/desktop), then no guarantees as some ISP's pass SMTP through a proxy relay hence EHLO response is only for proxy.
You may want to do a port check for servers to verify expected protocol support (just a TCP connect may be sufficient but protocol handshake is preferable). Additionally, best to build up a database of verified SMTP and IMAP server mappings as there can be split names (e.g. smtp.domain.com and mail.domain.com) - discovering SMTP is easy, discovering outbound server(s) which usually also means calendar/contacts server, not so much (except for Exchange but only if autodiscovery is configured correctly).
If you can get your users to approve and assuming you have their username/password, you could try connecting to SMTP via MX record and sending an email back to your own address then checking through headers for useful info about the server (needs to be authenticated to relay). Users could alternatively reply to an email you generate in order to get the same server info.
Also ensure that you do your own DNS query and try each MX record or all primaries -- if the principal MX is down or DNS is poorly configured for equal weighting, you could end up hitting a smarthost / backup which may just be a dumb SMTP relay / store-and-forward and not give you the correct response.
TL;DR: No quick solution but a cascaded algorithm that trys and fails different solutions until one works / gives an expected result should work.
Most devices can auto detect the service by parsing the Whole e-mail address. Xyz#gmail.com would obviously be a gmail account. So for Apple, Gmail, Yahoo, Live, Hotmail etc you can easily program for.
For other domains, including custom, you can try this: http://www.exclamationsoft.com/exclamationsoft/netmailbot/help/website/HowToFindTheSMTPMailServerForAnEmailAddress.html
You can detect Google Apps For Your Domain accounts by examining the domain's MX records. If the primary MX record is ASPMX.L.GOOGLE.com, then it's GMail.
I created an implementation of this that has been used widely with some success: https://github.com/tig/Email2Calendar
This is used by both milelogr.com and freebusy.io.
From the definition of SPF, SPF only authorizes IP address. For one of our domain name, we have created an SPF record to allow only A and MX IPs as genuine sender. This domain is hosted in a shared-hosting environment along with many other customers.
In such setup, owners of other domains on same host can spoof my emails easily. Is there any way SPF still work?
(correct me if my understanding abt SPF is wrong)
Yes they can spoof them but it would be very very unlikly.
If you are concerned about your personal mails that you send out or automatic mails from any system you provide on said host, you might consider signing them cryptographicaly to enable recipients to check if they are genuine.
I think there techniquies implemented in some mailservers to sign mails automaticly AND there is of course DNS signatures but what the status there is is bejond my knowledge.
When developing an application that sends out notification email messages, what are the best practices for
not getting flagged as a spammer by your hosting company. (Cover any of:)
best technique for not flooding a mail server
best mail server products, if you were to set up your own
sending messages as if from a specific user but still clearly from your application (to ensure complaints, etc come back to you) without breaking good email etiquette
any other lessons learned
not getting flagged as spam by the receiver's client? (Cover any of:)
configuring and using sender-id, domain-keys, SPF, reverse-dns, etc to make sure your emails are properly identified
best SMTP header techniques to avoid getting flagged as spam when sending emails for users (for example, using Sender and From headers together)
any other lessons learned
An additional requirement: this application would be sending a single message to a single recipient based upon an event. So, techniques for sending the same messages to multiple recipients will not apply.
best technique for not flooding a mail server
not a lot you can do about this beyond checking with your mail server admin (if it's a shared hosting account / not in your control). but if the requirement is one email to a single recipient per event, that shouldn't be too much of an issue. the things that tend to clog mail systems are emails with hundreds (or more) of recipients.
if you have events firing off all the time, perhaps consider consolidating them and having an email sent that summarizes them periodically.
sending messages as if from a specific user but still clearly from your application (to ensure complaints, etc come back to you) without breaking good email etiquette
you can accomplish this by using the "Reply-To" header, which will then have clients use that address instead of the From address when an email message is being composed.
you should also set the "Return-Path" header of any email, as email without this will often get filtered off.
ex.
From: me#me.com
Return-Path: me#me.com
Reply-To: auto#myapp.com
configuring and using sender-id, domain-keys, SPF, reverse-dns, etc to make sure your emails are properly identified
this is all highly dependent on how much ownership you have of your mail and DNS servers. spf/sender-id etc... are all DNS issues, so you would need to have access to DNS.
in your example this could present quite the problem. as you are setting mail to be from a specific user, that user would have to have SPF (for example) set in their DNS to allow your mail server as a valid sender. you can imagine how messy (if not outright impossible) this would get with a number of users with various domain names.
as for reverse DNS and the like, it really depends. most client ISP's, etc... will just check to see that reverse DNS is set. (ie, 1.2.3.4 resolves to host.here.domain.com, even if host.here.domain.com doesn't resolve back to 1.2.3.4). this is due to the amount of shared hosting out there (where mail servers will often report themselves as the client's domain name, and not the real mail server).
there are a few stringent networks that require matching reverse DNS, but this requires that you have control over the mail server if it doesn't match in the first place.
if you can be a bit more specific i may be able to provide a bit more advice, but generally, for people who need to send application mail, and don't have a pile of control over their environment, i'd suggest the following:
make sure to set a "Return-Path"
it's nice to add your app and abuse info as well in headers ie: "X-Mailer" and "X-Abuse-To" (these are custom headers, for informational purposes only really)
make sure reverse DNS is set for the IP address of your outgoing mail server
first a quick correction to the previous
return-path: is a header added by recieving system based on the envelope-sender of the incomming message
for spf to work the return-path/envelope-sender needs to be yourapp#yourdomain.com
and ensure the spf record for yourdomain.com {or if per-user spf} for yourapp#yourdomain.com allows mails to originate on the server that hosts the app/sends the email
this envelope-sender is the address that will recieve all bounces/errors
now sender-id is different entirely it checks the return-path/envelope-sender
and the
from: address {stored inside the message}
if sending
from: hisname yourapp#yourdomain.com
reply-to: hisname hisaddres#hisdomain.com
this will be a non-issue
if sending
from: hisname hisaddres#hisdomain.com
it will be and you must add a
Resent-From: hisname yourapp#yourdomain.com
as this specifies to ignore the from: for sender-id checks use this instead as it has been sent by you on his behalf
now for the other bits that are worthwhile
ip's mentioned are your mailservers
a have your ip's ptr point to a name that also resolves to the same ip
FQDNS
b have your server helo/ehlo with whatever.domain.com where domain.com is the same as the domain of the name in step A {not the same name for resons below}
c have that helo/ehlo servername also resolve to the ip of your server
d add the following spf record to that helo/ehlo name "v=spf1 a -all"
{meaning allow helo/ehlo with this name from ip's this name points to only}
e add the following sender-id lines to the helo/ehlo name {purely for completeness
"spf2.0/mfrom,pra -all" {ie there are no users#this-domain}
f add the following spf to the FQDNS-name and any other hostnames for your server
"v=spf1 -all" {ie no machines will ever helo/ehlo as this name and no users#this-domain}
{as the fqdns name can be determined by bots/infections its better to never allow this name to be used in helo/ehlo greetings directly it is enough that it be from the same domain as the helo/ehlo identity to prove the validity of both}