When a SMTP relay A forwards a message, it looks up the MX record of the target address, then forwards the email to the SMTP server B that this record points to.
But how does A prove its identity to B?
(I would like to send emails from my domain name)
There is no authentication for general SMTP server to SMTP server communication.
SMTP was designed long time ago in "small & friendly internet" era.
There are later added "tricks" to reject spam anyway e.g.
rejecting "bad" IP addresses based on locally selected set of DNSBLs, some DNSBLs list DUL ranges
rejecting based on locally gathered non published reputation (especially by big mail services)
…
When setting-up my mobile email client, I usually need to enter the SMTP relay server domain name, but why my email client need a SMTP relay server at all?
I think it can directly talk to the smtpd server (e.g. postfix) from which domain the destination Email address is, by looking up the DNS MX records. And send directly to it, why not?
Actually I have another confusion, if a smtpd server use SMTP AUTH, then how can a rely server elsewhere rely a mail to it? The rely server simply don't have the username and password necessary to transfer that mail.
Email clients use SMTP relay server to
simplify email sending (e.g. to avoid repeating delivery attempts by email client)
avoid being mistaken for spammers.
Direct email sending might be a reasonably simple option in ancient pre spam past.
SMTP AUTH is used (mainly) with email client to email server communication. SMTP server to SMTP server communication could not require SMTP AUTH without prohibiting email from "strangers". SMTP AUTH allows to accept messages from trusted clients (customers) even from IP addresses that otherwise would be blocked by DNSBL (e.g. DUL ranges).
A SMTP server these days does many more things:
Queues the emails and works on delivering them: this may not matter if you just send a single email here and there, but it's a different thing for a 500k recipients newsletter.
Retries messages than can not be delivered immediately (e.g., slow receiving server), and eventually bounces them if they cannot be delivered. According to RFC 5321 once a SMTP server accepts responsibility of an email message, it must not lose it but either deliver or return (bounce) it.
DKIM sign the message.
Route emails to go out from different sending IP addresses and host names, for deliverability purposes. (Email reputation is based a lot on sending IP addresses.)
Throttle delivery of large amounts of emails to avoid being seen as an aggressive sender and getting blocked.
Optionally, archive (or bcc) all outgoing email, for documentation or compliance purposes.
Of course you can also do all these things in your email sending application and then you don't need a SMTP server, then you wrote an SMTP server.
Am developing and newsletter email software in vb.net for a friend, and he has asked me to add option to send email using dns servers,
How do i send an emai using dns server without smtp or imap?
Send email "using DNS servers" still uses SMTP. And it's generally a bad idea. Here's what it's about:
Usually, an email app (your own, Outlook, etc.) send mail using SMTP or similar protocol to a configured mail server. It could be a company's own server, or an ISP's server, or something like that. The mail may pass through various outgoing mail servers on its way out of an organisation, but at some point it hits a "boundary server" (There are pretty names for all these different types of server, but they're not important right now.)
This boundary server uses DNS to find the recipient's mail server and then sends the message (using SMTP) to that server. (I have omitted the details on purpose)
Sending email "using DNS" means that your app acts as a boundary server. It uses DNS to find the recipient's mail server and then sends the mail directly - rather than using any outgoing servers.
It's a bad idea for two reasons.
More and more ISPs block SMTP traffic that doesn't pass through their outgoing servers
More and more incoming servers validate the boundary server using various techniques. Your app is not likely to be accepted by these servers
Both of these issues are anti-spam countermeasures, so using this technique makes you look like a spammer, leaves many of your messages blocked, and is likely to get your IP address blacklisted.
Hopefully you can explain this to your friend so that they will understand that the "normal" way of sending mail is the only way to ensure that they don't get listed as a spammer.
Need to implement the email validation to find if the email really exists. After some search found that the this is not possible due to the spam control techniques used (e.g. grey listing).
So moved with MX and A attribute check (as we are not allowed to send confirmation mails directly):
DirContext ictx = new InitialDirContext();
Attributes attrs = ictx.getAttributes("gmail.com",
new String[] {"MX"});
I am getting the results but not sure how the dns lookup works here (the complete background of the request and response i.e how the request is forwarded to the mail exchange server and also which port is used for the contacting the Mail Exchange server)
This will immensely help in moving into the production env without any doubts about the network behavior as we are having strick firewall restrictions.
Production Env : Weblogic Server on Solaris OS.
Similar dicussion is at Email validation MX Lookup but it doesnt have enough detail
It sounds like you're confusing two different techniques:
1) Perform a DNS lookup to ensure that an MX record exists for the domain. This will eliminate all email addresses at fake (i.e., not registered) domains. Note however that having an MX record is not a requirement for legitimate mail reception. In the absence of an MX record, the mail software will simply use the A record for the base domain.
2) Make a TCP connection to the recipient mail server to see if anything is listening. This is always port 25. You don't have to send a test message, you can just connect to the port and then immediately disconnect. A successful connection implies a valid email address. (Or at least, a valid domain, which is as much as you can verify.)
For the past few weeks, I've been trying to learn about just how email works. I understand the process of a client receiving mail from a server using POP pretty well. I also understand how a client computer can use SMTP to ask an SMTP server to send a message. However, I'm still missing something...
The way I understand it, outgoing mail has to make three trips:
Client (gmail user using Thunderbird) to a server (Gmail)
First server (Gmail) to second server (Hotmail)
Second server (Hotmail) to second client (hotmail user using OS X Mail)
As I understand it, step one uses SMTP for the client to communicate. The client authenticates itself somehow (say, with USER and PASS), and then sends a message to the gmail server.
However, I don't understand how gmail server transfers the message to the hotmail server.
For step three, I'm pretty sure, the hotmail server uses POP to send the message to the hotmail client (using authentication, again).
So, the big question is: when I click send Mail sends my message to my gmail server, how does my gmail server forward the message to, say, a hotmail server so my friend can recieve it?
Thank you so much!
~Jason
Thanks, that's been helpful so far.
As I understand it, the first client sends the message to the first server using SMTP, often to an address such as smtp.mail.SOMESERVER.com on port 25 (usually).
Then, SOMESERVER uses SMTP again to send the message to RECEIVESERVER.com on port 25 (not smtp.mail.RECEIVESERVER.com or anything fancy).
Then, when the recipient asks RECEIVESERVER for its mail, using POP, s/he recieves the message... right?
Thanks again (especially to dr-jan),
Jason
The SMTP server at Gmail (which accepted the message from Thunderbird) will route the message to the final recipient.
It does this by using DNS to find the MX (mail exchanger) record for the domain name part of the destination email address (hotmail.com in this example). The DNS server will return an IP address which the message should be sent to. The server at the destination IP address will hopefully be running SMTP (on the standard port 25) so it can receive the incoming messages.
Once the message has been received by the hotmail server, it is stored until the appropriate user logs in and retrieves their messages using POP (or IMAP).
Jason - to answer your follow up...
Then, SOMESERVER uses SMTP again to send the message to RECEIVESERVER.com on port 25 (not smtp.mail.RECEIVESERVER.com or anything fancy).
That's correct - the domain name to send to is taken as everything after the '#' in the email address of the recipient. Often, RECEIVESERVER.com is an alias for something more specific, say something like incoming.RECEIVESERVER.com, (or, indeed, smtp.mail.RECEIVESERVER.com).
You can use nslookup to query your local DNS servers (this works in Linux and in a Windows cmd window):
nslookup
> set type=mx
> stackoverflow.com
Server: 158.155.25.16
Address: 158.155.25.16#53
Non-authoritative answer:
stackoverflow.com mail exchanger = 10 aspmx.l.google.com.
stackoverflow.com mail exchanger = 20 alt1.aspmx.l.google.com.
stackoverflow.com mail exchanger = 30 alt2.aspmx.l.google.com.
stackoverflow.com mail exchanger = 40 aspmx2.googlemail.com.
stackoverflow.com mail exchanger = 50 aspmx3.googlemail.com.
Authoritative answers can be found from:
aspmx.l.google.com internet address = 64.233.183.114
aspmx.l.google.com internet address = 64.233.183.27
>
This shows us that email to anyone at stackoverflow.com should be sent to one of the gmail servers shown above.
The Wikipedia article mentioned (http://en.wikipedia.org/wiki/Mx_record) discusses the priority numbers shown above (10, 20, ..., 50).
You're looking for the Mail Transfer Agent, Wikipedia has a nice article on the topic.
Within Internet message handling services (MHS), a message transfer agent or mail transfer agent (MTA) or mail relay is software that transfers electronic mail messages from one computer to another using a client–server application architecture. An MTA implements both the client (sending) and server (receiving) portions of the Simple Mail Transfer Protocol.
The terms mail server, mail exchanger, and MX host may also refer to a computer performing the MTA function. The Domain Name System (DNS) associates a mail server to a domain with mail exchanger (MX) resource records containing the domain name of a host providing MTA services.
You might also be interested to know why the GMail to HotMail link uses SMTP, just like your Thunderbird client. In other words, since your client can send email via SMTP, and it can use DNS to get the MX record for hotmail.com, why doesn't it just send it there directly, skipping gmail.com altogether?
There are a couple of reasons, some historical and some for security. In the original question, it was assumed that your Thunderbird client logs in with a user name and password. This is often not the case. SMTP doesn't actually require a login to send a mail. And SMTP has no way to tell who's really sending the mail. Thus, spam was born!
There are, unfortunately, still many SMTP servers out there that allow anyone and everyone to connect and send mail, trusting blindly that the sender is who they claim to be. These servers are called "open relays" and are routinely black-listed by smarter administrators of other mail servers, because of the spam they churn out.
Responsible SMTP server admins set up their server to accept mail for delivery only in special cases 1) the mail is coming from "its own" network, or 2) the mail is being sent to "its own" network, or 3) the user presents credentials that identifies him as a trusted sender. Case #1 is probably what happens when you send mail from work; your machine is on the trusted network, so you can send mail to anyone. A lot of corporate mail servers still don't require authentication, so you can impersonate anyone in your office. Fun! Case #2 is when someone sends you mail. And case #3 is probably what happens with your GMail example. You're not coming from a trusted network, you’re just out on the Internet with the spammers. But by using a password, you can prove to GMail that you are who you say you are.
The historical aspect is that in the old days, the link between gmail and hotmail was likely to be intermittent. By queuing your mail up at a local server, you could wash your hands of it, knowing that when a link was established, the local server could transfer your messages to the remote server, which would hold the message until the recipient's agent picked it up.
The first server will look at DNS for a MX record of Hotmail server. MX is a special record that defines a mail server for a certain domain. Knowing IP address of Hotmail server, GMail server will sent the message using SMTP protocol and will wait for an answer. If Hotmail server goes down, GMail server wiil try to resend the message (it will depend on server software configuration). If the process terminates ok, then ok, if not, GMail server will notify you that he wasn´t able to deliver the message.
If you really want to know how email works you could read the SMTP RFC or the POP3 RFC.
All emails are transferred using SMTP (or ESMTP).
The important thing to understand is that the when you send message to someguy#hotmail.com this message's destination is not his PC. The destination is someguy's inbox folder at hotmail.com server.
After the message arrives at it's destination. The user can check if he has any new messages on his account at hotmail server and retrieve them using POP3
Also it would be possible to send the message without using gmail server, by sending it directly from your PC to hotmail using SMTP.
Step 2 to 3 (i.e. Gmail to Hotmail) would normally happen through SMTP (or ESMTP - extended SMTP).
Hotmail doesn't send anything to a client via POP3. It's important to understand some of the nuances here. The client contacts Hotmail via POP3 and requests its mail. (i.e. the client initiates the discussion).