Encrypted email - sounds like a great thing, right? Problem already solved, right? Well... I don't think so and I'm hoping I'm wrong!
To understand what I'm asking, please understand what I'm NOT asking: I am not asking how I encrypt and sign messages sent over public networks. This is a bit different.
I want to set up a mail server where email clients can both read and post and their messages to the mail server encrypted in both directions, preferably using standard clients available at internet cafes, etc, without requirement for specialized software or encrypted port-forwarding a-la SSH.
The key here is that there is a trusted mail server which can deliver email to a community using a local-delivery-agent. One can then send email to and receive from anyone and everyone who also uses that same system without concern for the security of their communications. There's no need to encrypt every message using the public key of every recipient - what pain THAT would be! - and instead all we're talking about here is a bidirectional communication that's encrypted from clients to this system.
Of course, public messages come in unencrypted via the usual port 25 process to all participants of the email server. They may or may not be encrypted as they were - we're not worried about them. Email Clients connect from anywhere and the responding code on the server system encrypts those messages for reading, even though they were sent through the public network in plain text already... This much I can already get fairly easily with encrypting IMAP servers like Dovecot.
What I want to add to this is that connected email clients can send email, encrypted, back to the system of which they are a client wherein that system forwards externally unencrypted, wherever it needs to go. If it's a local mailbox, the message is delivered via a local delivery agent. No keys involved there. The advantage of this design is that there's NO exposure of the email to external, untrusted systems or networks, and if the delivery is local, it's effectively protected end-to-end WITHOUT having a point-to-point hassle of encrypting individual messages in the more typical use.
This would be "god send" because as it is now, it's impossible to send secured mail through clients on the public internet to groups of people within an internal network.
I guess another way to phrase what I’m asking for is: Has someone created a package that gives us the other half of the encryption that IMAP (and POP?) servers already do – that distant clients on untrusted networks can hand-off out-bound unencrypted email through an encrypted link to the server on the other end?
ANOTHER ALTERNATIVE occurred to me: Encrypting SMTP / ESMTP servers that talk mail server to mail server in an encrypted form. (Similarly, clients should be able to hand off unencrypted email through an encrypted link, much as https works.) Anybody know of such a package? This is not quite as good, but an important part of email architecture...
If this doesn’t exist today, it should!
Thanks for your thoughts, pointers, etc.
Most existing email clients support message submission (either using SMTP or MSA) via TLS/SSL. I just checked in Outlook and Opera and both support it.
And I know for a fact that Courier's email suite supports TLS/SSL for both SMTP and MSA (and IMAP), so it's not an obscure setup; just a little uncommon. And it supports requiring TLS/SSL for any or all protocols.
Here is an example client SSL config screen.
Outlook Express:
Related
I wondered how email is secured (encryption and signing), then googled this and got the following knowledge.
Normally, end-to-end encryption is not performed. If it should be encrypted, another protocol such as S/MIME is needed.
Inter mail server connection mostly be encrypted using StartTLS.
Though, I couldn't find out whether inter mail server connection uses mutual TLS or not.
In other words, when the sender's mail server (e.g., owned by Google) sends emails to the receiver's mail server (e.g., owned by Microsoft), the latter check the former's certificate and verify the sender is actually who he says he is?
First of all, email is much less secure than one would think and hope. Because I didn't understand email and had similar questions as you do, I spent several months researching all technical aspects of modern email, which resulted in this article.
As you have already figured out, end-to-end encryption is totally independent from everything else. Encrypting email in transit is optional, and the percentage of TLS-secured connections increased from around 40% in 2013 to around 90% in 2020 as reported by Google/Gmail.
ESMTP itself provides only opportunistic security, which is useful only against passive attackers. As a result, the identity of the receiving server is often not even verified.
There are two standards to provide confidentiality even in the presence of an active attacker, which aren't widely deployed yet:
DNS-Based Authentication of Named Entities (DANE)
Mail Transfer Agent Strict Transport Security (MTA-STS)
Authenticating that an email is coming from the sender's domain is solved separately with the following standards:
Sender Policy Framework (SPF)
DomainKeys Identified Mail (DKIM)
Domain-based Message Authentication, Reporting, and Conformance (DMARC)
These standards are more widely deployed than the previous ones, but we're still far away from "universal" adoption and enforcement. One of the reasons for this is that not everyone sees email spoofing as a bug. Mailing lists use this as a feature.
Nope, no mutual SSL is used by default
emails totally can't be trusted
You are right. The same way as you cannot trust the name of the sender. However there are additional measures (not always used or enforced unfortunately) in the email header
routing information (can be spoofed
Domain Keys Identified Mail
see: https://blog.mailfence.com/email-header/
The fact that I access a certain website is certainly visible to the admin in my office - i.e. at work - and to the ISPs at work, home or wherever I access that website from. But if I send an e-mail from Yahoo/Google/etc. are the sender and receiver visible to the said admin, considering the fact that the connection is thru https: ?
There's a few things to consider here:
Connecting to your service prover If you log on to https://mail.google.com or https://www.hotmail.com then you are over https and so should be secure.
However most corporates only have internet access through a proxy and like to virus scan traffic. As HTTPS traffic is encrypted they use so called man in the middle (MITM) proxies so when you connect to Gmail over https you actual connect to the proxy over https (and it provides a HTTPS certificate in Google's name) and then that proxy connects to Google.
This is allowed by web browsers if the proxy certificate is set up as a local CA on your computer (which if in such a corporate environment they will be able to install this), but is very difficult to do without access to your machine (e.g.on coffee shop wifi as you will get a HTTPS certificate error). While this sounds dodgy (and is not well liked by security professionals as can introduce more risks), it is quite common in larger companies - though often companies that do this also block access to web mail providers as well. Virus scanners installed on your machine work in the exact same way.
You can see if this is the case by looking at the HTTPS certificate change in your browser and see if it is the same as at home and chains up to a real certificate authority (CA) or to your company name.
Once a party is performing a MITM and, while most companies have neither the resources nor the inclination to read this traffic, other than to run it through an automatic virus scanner, in theory they can.
Encryption to the sender address. Once you are on your mail provider (via web mail or from an email client on your device) and send a mail, for example from your gmail account to a hotmail account you've got to consider whether that connection is encrypted. In 2026 Gmail started warning if the sender address is for a service that doesn't allow Gmail to use HTTPS to send it the message. Unfortunately that was pretty much the norm for a long time and in 2014 Google said 50% of mail it sends is unencrypted, though (partially in response to Gmail marking this as insecure I guess) that has increased to 80% at the time of writing.
Either way this is from Gmail servers so local admins (e.g. in your office) should not be able to intercept this traffic as it's not sent on the local network (unless you work for a network telco or a spy agency!), but in theory unencrypted messages could be read here.
User encrypted mail only encrypted the body. Technologies like PGP or SMIME allow you to Encrypt your email message separately before you send it via a provider but this is only for the message contents so the To Address, From Address and Subject are still visible over plain text SMIME does allow you to have different subjects but that has its own problems - see the answer here for more details). And it's a massive of a pain to use these to be honest.
Authentication technologies like do not help here When researching encrypted mail you may see mention of DMARC, DKIM and SPF but they are authentication methods (to ensure you sent the message) rather than tools to hide data so ignore any mention of those when asking about hiding data. For example the earlier Google Blog link talks about authentication as the other thing they will warn against.
Quote a broad topic, and not a great fit for a Stack Overflow question to be honest (perhaps would be OK on the sister site http://security.stackexchange.com), but hopefully that gives you some understanding anyway.
Another, more detailed answer, can be found here: Using SSL/TSL to secure your communications. What I understand summing up both answers is that the e-mail content should be encrypted separately to be completely safe. I still am not sure what the answer is re: e-mail adresses of the sender and the receiver(s).
We are building a system that is, effectively, an email/calendar/contact client.
Users will provide us with their email address password (or other auth, eg oauth) and we will connect to their underlying email system.
"underlying systems" include:
Microsoft Exchange / Office365
GMail
Yahoo Mail
Apple email
Generic IMAP
Each of these systems have subtle (and not so subtle) differences in their APIs, especially to access calendar & contact data. Thus we need to know what provider the user is using.
But we would rather not ASK the user. We would like to figure it out (and at, least reduce the choices) automatically.
I've looked around for something that already implements this but have not found anything. I know it's mostly possible because Windows Phone does it pretty well (just enter username/pw and it does the right thing).
Before I dive in and start writing my own I want to ensure I'm not wasting my time if someone's already done it in an excellent way.
Know of anything like this? For this project prefer C#/.NET.
[EDIT: Adding potential algo]
Potential Algorithm:
Given email address & password
Extract domain name from email address
Try Exchange autodiscover. If successful done.
Use DNS MX records to find smtp host.
Do SMTP EHLO
Gmail responds with "250-mx.google.com at your service"
So if we see a "google.com" we are done.
Yahoo responds with 250-mta1257.mail.sk1.yahoo.com
So if we see a "yahoo.com" in the response we are done.
Apple responds with 250-xxxxxx-mac.com
So if we see a "mac.com" in the response we are done.
If none of the above
IMAP?
...
[EDIT: 5/18]
I built a prototype that uses methods that don't require auth (e.g. just MX/SMTP sluthing). Give it a try: http://bit.ly/KLZKxD
Algorithm seems reasonable. You will get best results running from unfiltered server (meaning it has direct SMTP outbound / doesn't run through a proxy). If running from client (mobile/tablet/desktop), then no guarantees as some ISP's pass SMTP through a proxy relay hence EHLO response is only for proxy.
You may want to do a port check for servers to verify expected protocol support (just a TCP connect may be sufficient but protocol handshake is preferable). Additionally, best to build up a database of verified SMTP and IMAP server mappings as there can be split names (e.g. smtp.domain.com and mail.domain.com) - discovering SMTP is easy, discovering outbound server(s) which usually also means calendar/contacts server, not so much (except for Exchange but only if autodiscovery is configured correctly).
If you can get your users to approve and assuming you have their username/password, you could try connecting to SMTP via MX record and sending an email back to your own address then checking through headers for useful info about the server (needs to be authenticated to relay). Users could alternatively reply to an email you generate in order to get the same server info.
Also ensure that you do your own DNS query and try each MX record or all primaries -- if the principal MX is down or DNS is poorly configured for equal weighting, you could end up hitting a smarthost / backup which may just be a dumb SMTP relay / store-and-forward and not give you the correct response.
TL;DR: No quick solution but a cascaded algorithm that trys and fails different solutions until one works / gives an expected result should work.
Most devices can auto detect the service by parsing the Whole e-mail address. Xyz#gmail.com would obviously be a gmail account. So for Apple, Gmail, Yahoo, Live, Hotmail etc you can easily program for.
For other domains, including custom, you can try this: http://www.exclamationsoft.com/exclamationsoft/netmailbot/help/website/HowToFindTheSMTPMailServerForAnEmailAddress.html
You can detect Google Apps For Your Domain accounts by examining the domain's MX records. If the primary MX record is ASPMX.L.GOOGLE.com, then it's GMail.
I created an implementation of this that has been used widely with some success: https://github.com/tig/Email2Calendar
This is used by both milelogr.com and freebusy.io.
My application will be used in a manner that the user is remote from his computer running the application, receiving data via text messages sent periodically from the PC. Im sending the text message via email, using the number#carrierdomain.com. To simplify determining the user's smtp server, I've been sending the message using the destination phone carrier's smtp, instead of the whatever may or may not be available at the PC. This has worked so far with AT&T and T-Mobile, but not Verizon, as they have discontinued their smtp service.
I'm using mailsend v1.15 http://www.muquit.com/muquit/software/mailsend/mailsend.html
but Im open to alternatives.
I do have a less than ideal workaround, which is to use google or hotmail's smtp, but that requires a login/pass.
If I can send the text through regular email, using outlook, it works. I've looked at outlooks smtp logs, but that doesn't really help.
If its not being sent to a Verizon smtp server, how does it get routed, and is it something that my application can use?
Is there a reliable way to send email without having the senders smtp?
Theoretically, any smtp server can send your e-mail. However, due to the overwhelming amount of spaming on the internet, pretty much every isp has locked down their smtp server and will not allow anonymous relaying of e-mail.
If you do have the outlook client on the box you are sending the e-mail you can look at using one of the Outlook APIs:
http://msdn.microsoft.com/en-us/library/gg608200.aspx
to send the e-mail message. You are using the outlook configuration then and don't have to worry about it your code.
As long as an SMTP server is configured to allow relay from a remote domain then you can use it send email from/to virtually any address. There are downsides, however, to using any SMTP server to send an email from a specific domain - the most notable of which is that the recipient's domain will likely flag the message as spam because the domain of the SMTP server is different from that of the sender's email address.
There are other non-email APIs that can be used to send text messages, if that is something you're willing to consider. Check out:
Tropo (http://www.tropo.com)
Zeep Mobile (http://www.zeepmobile.com)
Best of luck to you.
Impossible. SMTP is the Simple Mail Transfer Protocol, without this is impossible to access to another account with Email format. You can make a php mail where you don't need to have smtp, but your access will be limited, not all ESP / ISP will allow you to delivery those emails, because they need to identify you using your SMTP and MX Lookup (Reverse MX A records) from your server. It exists more ways, but are 100% SPAMMERS and HACKERS methods and I don't recommend to use them.
Am developing and newsletter email software in vb.net for a friend, and he has asked me to add option to send email using dns servers,
How do i send an emai using dns server without smtp or imap?
Send email "using DNS servers" still uses SMTP. And it's generally a bad idea. Here's what it's about:
Usually, an email app (your own, Outlook, etc.) send mail using SMTP or similar protocol to a configured mail server. It could be a company's own server, or an ISP's server, or something like that. The mail may pass through various outgoing mail servers on its way out of an organisation, but at some point it hits a "boundary server" (There are pretty names for all these different types of server, but they're not important right now.)
This boundary server uses DNS to find the recipient's mail server and then sends the message (using SMTP) to that server. (I have omitted the details on purpose)
Sending email "using DNS" means that your app acts as a boundary server. It uses DNS to find the recipient's mail server and then sends the mail directly - rather than using any outgoing servers.
It's a bad idea for two reasons.
More and more ISPs block SMTP traffic that doesn't pass through their outgoing servers
More and more incoming servers validate the boundary server using various techniques. Your app is not likely to be accepted by these servers
Both of these issues are anti-spam countermeasures, so using this technique makes you look like a spammer, leaves many of your messages blocked, and is likely to get your IP address blacklisted.
Hopefully you can explain this to your friend so that they will understand that the "normal" way of sending mail is the only way to ensure that they don't get listed as a spammer.