I am relatively new to arena of emails. Just drilling out tutorial by tutorial about email internals. My requirement is I need to send emails and receive emails from my webhost. For this to happen, I need an smtp server(daemon) running right? I recently studied about MTAs, which are responsible for transferring emails from one host to the other. So this smtp server(daemon) acts as an MTA. And I also studied sendmail is an MTA, which boils down to sendmail runs a smtp daemon in the background. Right?
It's probably fairer to say that sendmail is an SMTP daemon than to say it runs one, since sendmail is pretty monolithic. But basically you have it right - in order to receive email, you need an MTA listening on port 25 when an incoming connection comes. There are many choices for an MTA. I prefer postfix because it's not monolithic, it has a very easy to read config file, and it has a good security model. Other good choices are exim and qmail.
For outgoing mail, you need a program that can figure out where the mail is supposed to go, and make a connection to that receiver's port 25. Once again, just about any MTA will do that for you, but some programs that want to send mail will attempt to do it directly instead of invoking the local MTA. The problem with that is that they then have to duplicate all the things MTAs give you, such as knowing how to fall back and retry when the mail receiver can't be reached.
You need to run daemon to receive emails and to resend emails if they failed to be sent for some reason.
To send them, you just invoke sendmail which will connect to the destination's sendmail (which, of course, runs as a daemon), send you mail and exit.
If you sending this from a website, you may use mail functions of your scripting language, because spawning a process is quite costly thing under a heavy load.
Related
I have a basic question about mail servers. Usually any general server (not necessarily mail servers) handle all kinds of related requests i.e. both direction interaction with client (like sending and receiving msg to and from client) but in case of mail servers, there are 2 different servers -- one for outgoing messages called outgoing server following SMTP protocol and another for incoming messages called incoming server following POP3/IMAP protocol. Why so? . For that matter couldn't the 2 protocols be accommodated in one single protocol to handle both direction message flow. Also, typically are these 2 servers hosted on same machine in a general business?
Because the protocols are old. In the old Unix tradition, and ignoring UUCP for now:
SMTP came first, to transfer mail between sites, and mail was read locally on the server/network using a text mode client when logged in. There was no need to fetch mail, you used the 'mail' command, and it accessed your local mail spool (a file containing your mail, sitting on the file system, that the SMTP server appended to).
Later on, came the development that people wanted to read mail on their own hosts so a protocol was invented to serve that. The POP server would read that same spool file, and allow you to download all your messages to your intermittently connected client computer. SMTP was reused for sending mail because it already existed and was easily adapted for that purpose.
These days, there are usually three servers of note: SMTP submission server, SMTP transmission server, and IMAP. The submission server is where end users of the service submit their e-mail to be forwarded onto the final host, for example the Google server a Gmail user submits their email to (on port 465 or port 587, usually, with authentication). The transmission server is responsible for delivering and receiving email between sites (eg. when Yahoo and Gmail exchange mail for their customers with each other, on port 25). And IMAP is used by an end user to fetch their email.
These three services, on large sites, are generally served by separate servers on separate hosts. On very large services, like Gmail, they are separate pools of servers.
On a small business host, they were often just one machine.
There are newer and more integrated protocols. For example, both EAS and EWS have mail fetch and submission contained in the same protocol.
Personally, I regret that almost nobody uses UUCP anymore :)
The separation of shipping, management and collection of mail has a historical, philosophical and design basis.
This is in line with the spirit of the Unix world: "do one thing but do it right" and "make everything as simple as possible".
How complex the issue of e-mail is and how much technology it covers, read here:
https://en.wikipedia.org/wiki/Email
and then see the relevant RFCs in the footnotes for details.
It is difficult to implement a monolithic program that takes into account all the nuances of e-mail and at the same time
keeping it simple (Simple in SMTP). Such a monolith would be a nightmare to develop, maintain and administer,
and the post office has changed many times since the 1970s.
As for the second question, there is no reason to use the same physical machine, but there are also no contraindications other than the amount of resources available.
I have an ec2 micro instance running Centos on which I deploy my projects for testing purpose. Lately I developed a system which sends users email upon some events using sendmail. As I am the only one who use the system, the amount of emails sent per day is less than 50 and they are typically very simple emails.
After a few days I notice that the emails get delayed a lot after I issue send command, sometimes up to 15 minutes, and when they come, they come in batch (that is, several emails are delayed and then arrive at my inbox at one time). I understand there are reasons why we pay for email services like SES, but does sending email really take that much resources and why so?
Does your server have a fully-qualified domain name? Sendmail injects a delay when the sending host doesn't have a FQDN
To get one
first, in /etc/hostname, change your hostname from whatever to a FQDN like whatever.one-of-your-hosted-domain-names.com
then, in /etc/hosts, add a host entry like:
99.123.45.67 whatever.mysite.com
Reboot and try to send a mail to yourself again. Sendmail wil not delay at all.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
This question got me thinking, and I now realize that I don't know anything about the internals of MTAs.
What exactly does an MTA do? Everything after the SMTP protocol seems like dark magic to me. Let's say that I wanted to code a minimalistic MTA (or MDA) just for sending emails, what would I need to learn/do?
Edit: I don't actually plan on writing an MTA, I just want to understand how it works internally.
--- edit after somehow noticing you talked about possibly writing a MTA ---
To write a MTA, you need to open a server socket. When someone connects, you need to send and receive text (ascii) data on that socket in compliance with the SMTP protocol. SMTP is very chatty, so you can expect a few rounds of communication.
The initial round of communication typically tells you whether SMTP is supported or ESMTP is supported. The second (optional) round of communication is to determine security / encryption / feature support. Eventually the "client" side will ask to send a message to a particular address / set of addresses. When done, the server will indicate that it's ready to get the body of the email message. When the body of the message (and it's optinal attachments) have all been transmitted, the MTA will tell you it received the message fine. At that point in time, the MTA will act as a client to other MTAs discovered via DNS MX records to get your email closer to it's destination MTA which will copy it into someone's inbox.
So an MTA is needed because mail delivery on the client side is the equivalent to handing a physical letter to a post office. Post offices are responsible for inter-postoffice routing (which parallels to MTA-to-MTA transmission). The destination Post office is then responsible for delivery of the letter to the post office box or home address (which parallels one's computer inbox).
They don't call it e-mail for nothing.
--- original post follows ---
A MTA will accept a mail message, see if it can forward or deliver it, respond if it can be forwarded or delivered, and then forward or deliver it if it indicated it could.
How the message gets closer to it's final destination usually has a bit to do with DNS. MX (mail exchange) records in DNS indicate servers which are responsible (or at least closer to the responsible server) for particular email domain names. It is not possible to fully understand how a mail message gets closer to it's destination without understanding how DNS works.
A MTA typically looks at the delivery address, and either is configured to be the "end point" of the email address's mail domain, or knows that server XYZ is one hop closer to the email address's mail domain. If it's an endpoint, it will copy the message from the wire into someone's inbox. If it's relaying it will "forward" the message to the next MTA.
Here ya go: http://en.wikipedia.org/wiki/Message_transfer_agent
Quickly, the MTA receives the raw message, decides where it's ultimate destination is, and then forwards the message on to that destination.
A very simple MTA can be written the delivers only to local inboxes. The MTA is an "easier" part of the system to write because you can behave badly but still be functional, so your interoperability with other systems is less of an issue (that's where much of the complexities of email lie nowadays, that and spam/virus checking).
The real contract of an MTA is simply that if you accept the message from the system sending it to you, you accept responsibility to deliver that message. Thus, when that socket closes with an acknowledgement of acceptance, the delivering systems job is done and it's all in your hands.
If you happen to do a crummy job, mail is lost, and it's your problem. But it's still fun to play around with.
Edit: The original tutorial I linked to has gone 404. Here's another that's ok: https://troubleshootguru.wordpress.com/2014/07/06/mail-server-components-mta-mda-mua/
In short, a MUA is a user client that uses SMTP to send an email to an MTA. The MTA is a server that is responsible for routing the MTA to its destination. If that destination is another server, the MTA hands the email to an MDA. The MDA is a client on the server that uses SMTP to forward the email to the other server, which is also an MTA.
So what do you need to learn? If you want to write an MUA or MDA, you need to learn how to open a socket to another computer, send SMTP commands, and receive SMTP responses. If you want to write an MTA, you need to learn how to listen for socket connections on a port, receive SMTP commands, and send SMTP responses.
If you like Java, try the code on this page as a starting point for a client.
I am looking for a SMTP server that we can use for outgoing mail that permits a high volume (1,000+ per hour) without getting blocked. It could be downloaded on our machine or run online, but cannot cost over $25. Isn't there a good open source solution? I haven't been able to find one yet. Thanks.
Well, there are even free SMTP servers that can send a high volume of e-mail.
The problem is not in sending them, the problem is in delivering. You see, SMTP is a best-effort store-and-forward protocol - the e-mails are passed from one SMTP server to another in hope that it gets closer to the adressee's mail server (and usually it works). However, this also means that your message can be rejected or dropped anywhere in this chain of servers - or by the recipient. Once the e-mail leaves your network, there's not much you can do about it. In other words, it's not your server blocking the e-mails, but the servers it will be passing the mails to.
This question may be a useful reference for your problem; see also this article for some tips that will help your e-mail get delivered.
For that ammount of email, probably any server will do just fine. The problem is not to be blacklisted as a spammer. Jeff Atwood has a good blog post on the subject.
I personnaly would recommend Postfix or qmail, but that just me ...
I want to send out one off emails from a linux server. The server does not need to receive emails back.
Is there a simpler solution than sendmail for sending outgoing emails only?
I would prefer to use Perl to send the email.
Calling out to /usr/lib/sendmail is usually the preferred method because it handles delivery problems correctly. This does not mean using sendmail. Most (all?) mail transport system provide a command that's just named sendmail and provides an interface compatible to the original.
Alternatively you can send directly through a SMTP relay using a package like Net::SMTP but be extra sure to correctly cater for delivery problems.
Sendmail is not the only choice. you can use Postfix, Qmail, and many others
my Perl scripts call the mailx command. to know how to use it, type 'man mailx'
However this require to have a MTA correclty configured.
Or you can just use the Net::SMTP perl library and use your smtp server of choice
I have used msmtp successfully, ie it supports great authentication
TJ Luoma did a nice writeup of it on TUAW
I find nullmailer a very useful solution for the described scenario. Nullmailer is a sendmail/qmail/etc replacement MTA which relays to a fixed set of mail servers. It is very simple to configure and consumes little resources.
One important advantage of nullmailer over other mentioned solutions like mstmp and ssmtp is that it maintains a queue of emails to be sent. The application sending the mail is blocked only a very short time while the mail is queued (milliseconds). The sending of the mail happens in the context of a another process.
Solutions like mstmp and ssmtp don't maintain a queue of email. The sending happens in the context of the application, thereby blocking the application. Sending an email this way can easily take 1 to 2 seconds or longer. This may not be a problem in many cases but can become a problem if the email is sent by a web application.
http://untroubled.org/nullmailer/
You do not need a mail transport agent (MTA) instance on the machine doing the sending if you have another mailserver already running in your organization: you can make Perl deliver the email through SMTP to that server, so there's no need for having (another) MTA like sendmail on "your" machine.
I'm fond of creating a gmail or other free account and then using the java mail api in J2EE to send messages (from your new gmail account) to whoever... Typically i'll create a Mailer class which can be constructed with a default constructor and then give it a send(String dest, String subj, String body[, Obj attach...if you want]) and then in your case you might wrap the thing in a main method so that you can call it from else with some command line args, or call from within some java program. If you interested i'll shoot you the code.
I know you said perl, but the simplest cross-platform email sending library I have used is python's smtplib. Certainly worth a look.