I am trying to build a mail server using Ubuntu to send mail
I have done some research on that and find it is nearly impossible for a individual to send
the mail e.g. hotmail , gmail.
The question i am asking is not how to build a own server, it is why i can not build my own server.
To be precise:
Questions:
1) what are the requirements to send to those e.g. hotmail ,gmail server ? e.g. mx record , clear dns record . (only from server aspect , not concerning other factors such as headers or mail content), It would be easier to understand if they are listed out.
2) I read some document and it said the problem can be overcome by relayhost, what is it about and is it feasible?
3) For those ISP , what are their procedure in building the mail server? How is it different from my own small Ubuntu one?
Sorry for asking a lot of question, any help would be nice and well appreciated .
Most people use an out-of-the-box package as a mail server, rather than trying to write one that follows all of the relevant RFC specifications for SMTP, Internet Message Format, IMAP4, POP3, etc.
I'm not saying "don't write your own", just that if you do, be prepared for months and months of hard work, lots of bugs and even more frustration. It's a big project.
In terms of sending messages, you will need to follow the Simple Mail Transfer Protocol (SMTP) to send messages; and they should be sent to the correct server, as per the recipient's DNS records - see RFC 1034 and RFC 1035.
If you are correctly using SMTP to send valid messages to the right server, there's not a lot else you can do.
Your next problem is going to be reputation. This would be the same, whichever software you use to send your messages.
It's easy for a spammer to set up a new mail server and start sending messages, so it will take a while for some mail servers to trust you (particularly those that are regularly targeted, such as Hotmail, Gmail, etc).
Instead of sending messages directly to the recipient's server, you can use SMTP to send the messages to a relay server. This would usually be your own ISP's server, but it can be any willing partner. You would normally need to make advance arrangements, so that they will permit you to relay messages.
The relay server would then attempt to send the message to the recipient server. If it cannot do so, it must report the failure to the sender.
I have an e-mail account with hostgator. I would like to build a console application (later a service application) that will check a mysql database for e-mails that need to be sent out and will process them by sending a certain number of e-mail every so many seconds. These e-mails will each have a 2-3mb pdf price sheet attached to be sent to our regular customer base. About 500 are set up to receive e-mails.
My question is: Should I connect to hostgator directly and send the
e-mails or should I use the IIS Virtual Server Relay and have it
connect to hostgator?
The only thing I can think of is if Hostgator were down the virtual relay might be a good option since it has retries. There again, I guess I could put some try/catch in the code where if it doesn't connect to hostgator or if their is a problem it will go to the timer and wait x minutes before retrying to process e-mails.
We have found that the easiest approach is to generate the outgoing emails and drop them in the inetpub\mailroot\pickup folder and let the SMTP service handle them (you can set it up to forward the emails to the appropriate host).
This falls firmly under the category of not re-inventing the wheel: the SMTP service easily handles the situation when the connection to the mail server is down, retries, and all of the other "fun" stuff that you would otherwise have to implement in your code if you handle the SMTP sends directly.
Am developing and newsletter email software in vb.net for a friend, and he has asked me to add option to send email using dns servers,
How do i send an emai using dns server without smtp or imap?
Send email "using DNS servers" still uses SMTP. And it's generally a bad idea. Here's what it's about:
Usually, an email app (your own, Outlook, etc.) send mail using SMTP or similar protocol to a configured mail server. It could be a company's own server, or an ISP's server, or something like that. The mail may pass through various outgoing mail servers on its way out of an organisation, but at some point it hits a "boundary server" (There are pretty names for all these different types of server, but they're not important right now.)
This boundary server uses DNS to find the recipient's mail server and then sends the message (using SMTP) to that server. (I have omitted the details on purpose)
Sending email "using DNS" means that your app acts as a boundary server. It uses DNS to find the recipient's mail server and then sends the mail directly - rather than using any outgoing servers.
It's a bad idea for two reasons.
More and more ISPs block SMTP traffic that doesn't pass through their outgoing servers
More and more incoming servers validate the boundary server using various techniques. Your app is not likely to be accepted by these servers
Both of these issues are anti-spam countermeasures, so using this technique makes you look like a spammer, leaves many of your messages blocked, and is likely to get your IP address blacklisted.
Hopefully you can explain this to your friend so that they will understand that the "normal" way of sending mail is the only way to ensure that they don't get listed as a spammer.
I'm sending an email to the following recipients:
test#example.com, test1#test.com, test2#test.com
The message is sent to my local smtp server that has to relay it to #example.com and #test.com.
My question is: how the server should do it ?
Leave the message as is and relay it to a more sophisticated smtp server that will do one of the other options
Break it into two messages and relay one message to #example and two others to #test.com in one message
Break it into three messages and relay once to #example.com and twice to #test.com
And how the receiving server, for example #example.com know it should not send the #test.com message by himself too? (any mail header?)
What if #example.com and #test.com are served by the same server, will it receive the message more than once?
I'm probably missing something in the SMTP protocol.
The envelope (RFC822 and further) are just the outside bounds. The fun happens within the ESMTP protocol.
In terms of plain SMTP, each recipient (RCPT TO) fires up a new message in the outbox queue - One for each recipient. Then, it is delivered.
Now lets talk about the delivery: Supposed I asked to deliver to user#host.com. Here is how it works:
The host.com address is queried on DNS, in particular, for a record of MX (Mail Exchanger) type. (nslookup -q=MX should show you how it is done)
They are sorted (lowest number first), and delivery is tried on a round-robin basis using this sort.
When it is delivered to the any MX record host, its done. However, the reason there are multiple MX hosts is due to the need to queue up mail while on outages, for instance. So, a higher MX host is likely to just queue and not let the originating host server bounce and expire the message (and that is what happens when the message gets stuck on a given host - You can trace it via its Headers)
Each server has its own rules for delivery, however, when you forward to a host which tried to deliver itself on the MX target, we call that a Smart Host. A host knows whether or not it should queue for another host (relaying) or deliver locally via the SMTP greeting (HELO ).
Also: A single SMTP connection will let you send multiple messages, so even if you have to recipients, there is going to happen only one connection (although two messages in the mail queue)
For the past few weeks, I've been trying to learn about just how email works. I understand the process of a client receiving mail from a server using POP pretty well. I also understand how a client computer can use SMTP to ask an SMTP server to send a message. However, I'm still missing something...
The way I understand it, outgoing mail has to make three trips:
Client (gmail user using Thunderbird) to a server (Gmail)
First server (Gmail) to second server (Hotmail)
Second server (Hotmail) to second client (hotmail user using OS X Mail)
As I understand it, step one uses SMTP for the client to communicate. The client authenticates itself somehow (say, with USER and PASS), and then sends a message to the gmail server.
However, I don't understand how gmail server transfers the message to the hotmail server.
For step three, I'm pretty sure, the hotmail server uses POP to send the message to the hotmail client (using authentication, again).
So, the big question is: when I click send Mail sends my message to my gmail server, how does my gmail server forward the message to, say, a hotmail server so my friend can recieve it?
Thank you so much!
~Jason
Thanks, that's been helpful so far.
As I understand it, the first client sends the message to the first server using SMTP, often to an address such as smtp.mail.SOMESERVER.com on port 25 (usually).
Then, SOMESERVER uses SMTP again to send the message to RECEIVESERVER.com on port 25 (not smtp.mail.RECEIVESERVER.com or anything fancy).
Then, when the recipient asks RECEIVESERVER for its mail, using POP, s/he recieves the message... right?
Thanks again (especially to dr-jan),
Jason
The SMTP server at Gmail (which accepted the message from Thunderbird) will route the message to the final recipient.
It does this by using DNS to find the MX (mail exchanger) record for the domain name part of the destination email address (hotmail.com in this example). The DNS server will return an IP address which the message should be sent to. The server at the destination IP address will hopefully be running SMTP (on the standard port 25) so it can receive the incoming messages.
Once the message has been received by the hotmail server, it is stored until the appropriate user logs in and retrieves their messages using POP (or IMAP).
Jason - to answer your follow up...
Then, SOMESERVER uses SMTP again to send the message to RECEIVESERVER.com on port 25 (not smtp.mail.RECEIVESERVER.com or anything fancy).
That's correct - the domain name to send to is taken as everything after the '#' in the email address of the recipient. Often, RECEIVESERVER.com is an alias for something more specific, say something like incoming.RECEIVESERVER.com, (or, indeed, smtp.mail.RECEIVESERVER.com).
You can use nslookup to query your local DNS servers (this works in Linux and in a Windows cmd window):
nslookup
> set type=mx
> stackoverflow.com
Server: 158.155.25.16
Address: 158.155.25.16#53
Non-authoritative answer:
stackoverflow.com mail exchanger = 10 aspmx.l.google.com.
stackoverflow.com mail exchanger = 20 alt1.aspmx.l.google.com.
stackoverflow.com mail exchanger = 30 alt2.aspmx.l.google.com.
stackoverflow.com mail exchanger = 40 aspmx2.googlemail.com.
stackoverflow.com mail exchanger = 50 aspmx3.googlemail.com.
Authoritative answers can be found from:
aspmx.l.google.com internet address = 64.233.183.114
aspmx.l.google.com internet address = 64.233.183.27
>
This shows us that email to anyone at stackoverflow.com should be sent to one of the gmail servers shown above.
The Wikipedia article mentioned (http://en.wikipedia.org/wiki/Mx_record) discusses the priority numbers shown above (10, 20, ..., 50).
You're looking for the Mail Transfer Agent, Wikipedia has a nice article on the topic.
Within Internet message handling services (MHS), a message transfer agent or mail transfer agent (MTA) or mail relay is software that transfers electronic mail messages from one computer to another using a client–server application architecture. An MTA implements both the client (sending) and server (receiving) portions of the Simple Mail Transfer Protocol.
The terms mail server, mail exchanger, and MX host may also refer to a computer performing the MTA function. The Domain Name System (DNS) associates a mail server to a domain with mail exchanger (MX) resource records containing the domain name of a host providing MTA services.
You might also be interested to know why the GMail to HotMail link uses SMTP, just like your Thunderbird client. In other words, since your client can send email via SMTP, and it can use DNS to get the MX record for hotmail.com, why doesn't it just send it there directly, skipping gmail.com altogether?
There are a couple of reasons, some historical and some for security. In the original question, it was assumed that your Thunderbird client logs in with a user name and password. This is often not the case. SMTP doesn't actually require a login to send a mail. And SMTP has no way to tell who's really sending the mail. Thus, spam was born!
There are, unfortunately, still many SMTP servers out there that allow anyone and everyone to connect and send mail, trusting blindly that the sender is who they claim to be. These servers are called "open relays" and are routinely black-listed by smarter administrators of other mail servers, because of the spam they churn out.
Responsible SMTP server admins set up their server to accept mail for delivery only in special cases 1) the mail is coming from "its own" network, or 2) the mail is being sent to "its own" network, or 3) the user presents credentials that identifies him as a trusted sender. Case #1 is probably what happens when you send mail from work; your machine is on the trusted network, so you can send mail to anyone. A lot of corporate mail servers still don't require authentication, so you can impersonate anyone in your office. Fun! Case #2 is when someone sends you mail. And case #3 is probably what happens with your GMail example. You're not coming from a trusted network, you’re just out on the Internet with the spammers. But by using a password, you can prove to GMail that you are who you say you are.
The historical aspect is that in the old days, the link between gmail and hotmail was likely to be intermittent. By queuing your mail up at a local server, you could wash your hands of it, knowing that when a link was established, the local server could transfer your messages to the remote server, which would hold the message until the recipient's agent picked it up.
The first server will look at DNS for a MX record of Hotmail server. MX is a special record that defines a mail server for a certain domain. Knowing IP address of Hotmail server, GMail server will sent the message using SMTP protocol and will wait for an answer. If Hotmail server goes down, GMail server wiil try to resend the message (it will depend on server software configuration). If the process terminates ok, then ok, if not, GMail server will notify you that he wasn´t able to deliver the message.
If you really want to know how email works you could read the SMTP RFC or the POP3 RFC.
All emails are transferred using SMTP (or ESMTP).
The important thing to understand is that the when you send message to someguy#hotmail.com this message's destination is not his PC. The destination is someguy's inbox folder at hotmail.com server.
After the message arrives at it's destination. The user can check if he has any new messages on his account at hotmail server and retrieve them using POP3
Also it would be possible to send the message without using gmail server, by sending it directly from your PC to hotmail using SMTP.
Step 2 to 3 (i.e. Gmail to Hotmail) would normally happen through SMTP (or ESMTP - extended SMTP).
Hotmail doesn't send anything to a client via POP3. It's important to understand some of the nuances here. The client contacts Hotmail via POP3 and requests its mail. (i.e. the client initiates the discussion).