I'm implementing an application in ruby on rails (although this is a minor detail) that needs to send emails. Everything works fine, but I'm surprised cause when the app sends a mail, as long as it's authenticated against the smtp server, the sender it's going to be whatever it's specified in mail from. My point is, I can authenticate in a smtp server with "myuser#mydomain.com" and "mypassword" and be able to send a mail in which the recipient would see whichever address I want as sender.
This doesn't make any sense for me, and I'm sure it can't be so easy. I know it's surely a completely silly question for most of you, but would really appreciate anyone could make it clear for me.
Cheers!!
You authenticate as a generic user, it is only an accident that your login name is similar to a mail address. And yes, usually you can send mail with any reverse-path (reverse-path is the address in the MAIL FROM SMTP command). This may be necessary, for example a list manager uses VERP and sends mails with many unique reverse-paths.
However, some mail servers and mail administrators do place restrictions on the reverse path and the address in the From header. Especially, if the server signs the message.
Moreover, if the receiving server checks SPF records, it is useless to impersonate a remote domain, even if it is allowed, because the mail will be rejected anyway (if the domain publishes an SPF DNS record).
Related
My team supports a website for a client, and we use SendGrid to send email related to the site on their behalf.
We do not have anything to do with their own email server and I don't at present know anything about it.
So far as I can work out, SendGrid has proper authentication and is an authorised sender for their domain, and almost 98% of email is delivered successfully.
However, we have had a handful of bounces with the reason "550 relaying denied" and all of these were to addresses at our client's domain (the same one as their website and the from address of the emails.)
Most emails to their domain were delivered successfully.
Unfortunately I don't have access to the full headers of the bounce emails, only the reason.
I understand that in general this error can either be caused by
the sender not being authenticated correctly. I am very far from being an expert in this but so far as I can tell, there is nothing wrong there. Or
a DNS or similar misconfiguration on the part of the recipient's email domain. I have even less understanding about this and I have no access or responsibility for the client's email server.
My main question is, is there any way the domain being the same as the from address could be related? Being as the email is claiming to be from the same place it's sent to, is it possible for that to affect how it's handled by relays?
If not, I'd also appreciate any pointers on where to look for the issue (or what to advise the client to look at if the problem is likely to be from their end.) I have been trying to research issues with email configuration and authentication but I am very much a novice in this area.
Thanks in advance.
The domain being the same could very well be related, but normally when that happens, the receiving server refuses all mail purporting to be from itself.
Separate from DKIM & SPF, most mail servers believe they alone are responsible for the mail from their domain.com. As such, a lot of them have anti-phishing filters that reject "outside" mail that claims to be from themselves. It's like "You can't be Carrie, I'm Carrie! Go Away!"
The fact that it's only some mail is interesting. The error being relay denied may also be key, though these anti-phishing filters often use "fake" errors to not give away the game.
Do the recipients of the messages that are being rejected have some kind of internal forwarding applied? That may be the cause, in which case that bounce reason is honest.
Or they may have a more defined anti-phishing feature, only rejecting mail From or For certain addresses. You can try testing certain combinations, and see if anything is repeatable.
Ultimately however, it will come down to working with the receiving mail domain's admin, and either updating those rules, or whitelisting the SendGrid IPs that are sending the mail to them.
I have a server on OVH and I'm trying to send some mail to my Gmail address using sendmail.
I installed sendmail with apt-get on debian, and echo "Subject: test" | /usr/sbin/sendmail -v mymail#gmail.com works. However Gmail puts the mail in the spam folder and says the mail is not authenticated.
I have no knowledge of how email works and from what I've seen I could use SPF or DKIM to provide authentication, but it seems it requires admin access to ovh DNS servers.
What would be the easiest way, using only admin access to the server that sends mails, to make sure emails sent from it are not marked as spam ?
Thanks
There are no easy way. That's the simple answer.
Google Mail is fighting spam every second, every day, all year. To get mail delivered directly to the inbox requires time and patience and there are no shortcuts. If there where, spammers would have it way to easy!
You are very correct that Google's SMTP servers will ask you to beef up your SMTP mail headers to include better authentication and security. There's no way around it if you want to be on good terms with Google's SMTP servers.
Here is a list of things to consider
SPF (Must have, also to defend spam sent in your name)
DKIM (Must have, this is somewhat a step up from SPF)
ESMTP (Google likes it when you talk to them encrypted)
Bulk headers (Use them if it's bulk, no reason to lie!)
Unsubscribe headers (Use them if you are sending out maillist content)
SMTP relay's with nice Senderscores (Easy access to the inbox, but cost)
SMTP relay's with good reputation (Easy access to the inbox, but cost)
Feedback loop headers/setup (If you send large amount of mails)
Only use "warmed up" SMTP relay servers (Mostly used by bulk senders)
Reverse DNS to match HELO/HELO (Mismatch can make problems)
Static IP (It's a given)
As you can see it's no simple task to "just" send an email to Google and expect it to be passed along to the users Inbox without getting targeted as spam.
Most of the options above requires extensive server knowlagde or that your hosting provider supports it. You need to have access to change your own DNS records but also the SMTP server that you send your mail from needs to be setup with the things in advance.
Now to the:
"How to send emails to Google Mail that always land in the inbox for dummies"
Well you basically buy your way into the inbox. Use SMTP services like Mailchimp, Amazon AWS-SMS (Simple Mail Service) many if not all of these services have already setup DKIM, SPF and are on good terms with Google. They do however have many rules and what and what not to do so if you don't follow the rules they will close your account right away.
In my project when the user stored a new record to database, sent to him email with to perform afterSave() Method.
How to make sure that the email was sent?
I don't think this is something to do with yii2 or the afterSave() event (as long as the afterSave event is triggered, which you can verify by Runtime Logging for example). When using PHPMailer class you can see this discussion about making sure an email has been sent.
$mail->send() will not always return true. It returns true if the part of the sending process it was involved with works. So if you send to an unknown address, but do so via gmail, gmail's servers don't know whether the address exists or not at the time, so it will be accepted and bounced later. If you were sending to a gmail address when sending through gmail, then it would fail immediately.
If an account does not exist at all, most servers (including gmail) will still give a 5.1.1 "Unknown user" response, and that will be reported correctly by PHPMailer if you send by SMTP directly to the recipient's supposed mail server (but not if you send via an intermediate server (like gmail) or using mail()). PHPMailer doesn't have built-in support for doing that, but doing it yourself only involves a call to getmxrr and setting Host manually. Also you won't need to use authentication if you send that way.
You can do various things like check if a domain exists at all - if it doesn't, mail delivery won't work. Some servers will accept all addresses and send bounces later (e.g. if they have a spam filter with a long processing queue), but if you get rejected up-front, it's a pretty sure indication that the address doesn't exist.
You need to look into bounce handling too which will allow you to remove addresses that looked ok but later proved not to be, which is an entirely separate thing from anything that PHPMailer does. I will warn you now - bounce handling is extremely unpleasant!
You should also send using tls on port 587, not ssl on 465; see the gmail example provided with PHPMailer.
I would also recommend you to send mails via an SMTP auth connection trough PHPMailer.
I am sending email from application. My problem is when your type email address means format is correct but it may be not available anywhere.so when I am trying send such email address using SMTP through application, so how to track fail-over notification if intended recipient not found.
please suggest any approach.
There is no standard way of doing this. You might be able to make it work in some cases by parsing incoming email looking for delivery failure notices but it'll be a lot of work to keep that parsing code up to date. So at best you'll only get some of the notices. If you get a notice at all. The recipient's SMTP server may not send you anything, or the notice might get lost on it's way back to you.
The only way you are reliably get this sort of information is to host the SMTP server yourself, and deliver directly to the recipients SMTP server. In that case the SMTP server knows the status of the email's delivery and could report that to your email client via some non-standard back channel.
Note: Hosting your own SMTP server is non-trivial. And you will probably need to be on a fixed IP otherwise external SMTP servers will treat your email as spam. In case you're thinking of running one over a home connection or something.
We are building a system that is, effectively, an email/calendar/contact client.
Users will provide us with their email address password (or other auth, eg oauth) and we will connect to their underlying email system.
"underlying systems" include:
Microsoft Exchange / Office365
GMail
Yahoo Mail
Apple email
Generic IMAP
Each of these systems have subtle (and not so subtle) differences in their APIs, especially to access calendar & contact data. Thus we need to know what provider the user is using.
But we would rather not ASK the user. We would like to figure it out (and at, least reduce the choices) automatically.
I've looked around for something that already implements this but have not found anything. I know it's mostly possible because Windows Phone does it pretty well (just enter username/pw and it does the right thing).
Before I dive in and start writing my own I want to ensure I'm not wasting my time if someone's already done it in an excellent way.
Know of anything like this? For this project prefer C#/.NET.
[EDIT: Adding potential algo]
Potential Algorithm:
Given email address & password
Extract domain name from email address
Try Exchange autodiscover. If successful done.
Use DNS MX records to find smtp host.
Do SMTP EHLO
Gmail responds with "250-mx.google.com at your service"
So if we see a "google.com" we are done.
Yahoo responds with 250-mta1257.mail.sk1.yahoo.com
So if we see a "yahoo.com" in the response we are done.
Apple responds with 250-xxxxxx-mac.com
So if we see a "mac.com" in the response we are done.
If none of the above
IMAP?
...
[EDIT: 5/18]
I built a prototype that uses methods that don't require auth (e.g. just MX/SMTP sluthing). Give it a try: http://bit.ly/KLZKxD
Algorithm seems reasonable. You will get best results running from unfiltered server (meaning it has direct SMTP outbound / doesn't run through a proxy). If running from client (mobile/tablet/desktop), then no guarantees as some ISP's pass SMTP through a proxy relay hence EHLO response is only for proxy.
You may want to do a port check for servers to verify expected protocol support (just a TCP connect may be sufficient but protocol handshake is preferable). Additionally, best to build up a database of verified SMTP and IMAP server mappings as there can be split names (e.g. smtp.domain.com and mail.domain.com) - discovering SMTP is easy, discovering outbound server(s) which usually also means calendar/contacts server, not so much (except for Exchange but only if autodiscovery is configured correctly).
If you can get your users to approve and assuming you have their username/password, you could try connecting to SMTP via MX record and sending an email back to your own address then checking through headers for useful info about the server (needs to be authenticated to relay). Users could alternatively reply to an email you generate in order to get the same server info.
Also ensure that you do your own DNS query and try each MX record or all primaries -- if the principal MX is down or DNS is poorly configured for equal weighting, you could end up hitting a smarthost / backup which may just be a dumb SMTP relay / store-and-forward and not give you the correct response.
TL;DR: No quick solution but a cascaded algorithm that trys and fails different solutions until one works / gives an expected result should work.
Most devices can auto detect the service by parsing the Whole e-mail address. Xyz#gmail.com would obviously be a gmail account. So for Apple, Gmail, Yahoo, Live, Hotmail etc you can easily program for.
For other domains, including custom, you can try this: http://www.exclamationsoft.com/exclamationsoft/netmailbot/help/website/HowToFindTheSMTPMailServerForAnEmailAddress.html
You can detect Google Apps For Your Domain accounts by examining the domain's MX records. If the primary MX record is ASPMX.L.GOOGLE.com, then it's GMail.
I created an implementation of this that has been used widely with some success: https://github.com/tig/Email2Calendar
This is used by both milelogr.com and freebusy.io.