Honestly, I think I have a fundamental gap in understanding how SMTP works. I can't seem to find a good explanation of what is happening behind the scenes and I think this is preventing me from being able to do what I am attempting to do.
To explain, I'm trying to setup an application which sends notifications to users by connecting to an SMTP server. Fair enough. I figure, since I'm using my own domain, I have SPF/DKIM/DMARC configured, I can add an MX record for the host I set the application up on (my SPF record has the mx keyword to authorize any hosts in my MX records to send/receive mails). Then, I can have that same host run a super lightweight SMTP server that can accept mails from the application, and send them on to recipients.
Almost crucially, I want this server to basically just run on localhost so that only this application can connect and send mails through it, but so that it can't really "receive" mails sent to my domain (I have set the MX priority very low (well, a high number) for this app server). I figure since I'm running my own SMTP server, that I don't really need to authenticate against it (it's running on localhost), just take in any mail and send it on to recipient domains.
When sending on to recipient domains... does the SMTP server need to authenticate to say, the gmail SMTP server as a user in order to send mails over there? That seems weird, since it's not a user logging into gmail to send mails, it's an SMTP server that is authorized within SPF sending mail from my domain (From address from my domain as well) to where ever the app server user's email is based (in this example, the user would be e.g., some_user#gmail.com).
I tried using python's aiosmtpd command-line and telnet to send a mail from test#MY_DOMAIN.TLD to test#MY_DOMAIN.TLD and it didn't seem to deliver the message; I figured aiosmtpd would connect to the preferred MX servers for my domain (my "real" MX's) to transfer the message, which would then put it in my inbox. That didn't seem to be the case, and I'm not sure why.
Exact repro steps, where example.com is my domain, and terminals are running on a box with a hostname listed in my MX records.
Terminal A:
$ aiosmtpd -n
Terminal B:
$ telnet localhost 8025
EHLO <example.com>
MAIL FROM: test#example.com
RCPT TO: test#example.com
DATA
FROM: Application Notifications <test#example.com>
TO: User Name <test#example.com>
SUBJECT: App Notify Test
This is a test!
.
QUIT
How do SMTP servers normally send mail between each other? Do they each get some login to each other's SMTP servers to authenticate with, and since I'm not doing that, this is a problem? Can I run a SMTP server on localhost and have it send mail out of the network without receiving mails (a no-reply service)? Is there something obvious that I'm just missing here that solves all my problems?
Thanks
It sounds like you want to run a mail transfer agent (MTA) that relays email to remote SMTP servers. An MTA will typically act as an SMTP server to receive messages, and then it will act as an SMTP client when it relays the messages to remote hosts.
MTAs generally operate in two different modes: (1) They will relay messages from authenticated users to remote hosts, and (2) they will receive messages from remote hosts to its users and store them somehow. The combination of those two modes - where the MTA will accept messages from remote hosts and relay them to different remote hosts - is called an open relay and is sure to attract spammers and place your server on spam blacklists.
aiosmtpd is not an MTA or an email relay out of the box - it is merely an SMTP server that will receive messages and do whatever with the messages you program it to do. By default it will do nothing - that is, it will receive the messages and throw them away. If you want to implement an email relay in aiosmtpd, then you need to implement the SMTP client portion of the MTA, e.g. by implementing an aiosmtpd handler that instantiates smtplib.SMTP to connect to remote hosts.
However, if all you want is an email relay, then you most likely don't need aiosmtpd at all - postfix is probably a better choice.
aiosmtpd can be a good choice if you need to implement mailing list software or perform some automation tasks based on incoming emails from e.g. cameras or scanners.
If you want to implement an email relay in aiosmtpd, then you need to ensure that both the software and your server are configured in a way that you don't relay unauthenticated messages from the outside internet.
See also: Python aiosmtpd - what is missing for an Mail-Transfer-Agent (MTA)?
So, I actually figured out what was missing here.
I need to run an SMTP server, yes, but I also needed to write code to parse the "to" domain (the recipient domain), perform a DNS request for the MX server(s) of the recipient domain, and then use the smtplib client to then send mail over to the recipient domain. Authentication is not needed to relay that message to the recipient server, authentication is only required for reading from a given inbox or authenticating a sender to send on behalf of a domain (I trust myself and myself only to send mail). I can do all this while also only listening for mail on localhost so that only my local server can use the local SMTP server for relaying messages/emails off to recipient domains.
Additionally, I don't need to have my external IP listed as an MX server since it's not accepting mail for the domain, only sending. I do need an SPF record for it though so that it is an authorized relay/sender for email from my domain.
Related
I'm trying to implement a SMTP server at the moment and I've question regarding the routing...
Imagine, a client (foo#bar.com) sends a mail to foo1#bar1.com, foo2#bar2.com and foo3#bar3.com.
What happens when the SMTP server sees the message? Now, I'm not sure if the following sequence is correct:
The SMTP server will put the mail into the outbox of foo#bar.com
For each recipient (except foo#bar.com) in the envelope, the SMTP server will do:
Retrieve the MX record
Connect to the SMTP server determined by the MX record
Send HELO/EHLO command
Send MAIL FROM command (foo#bar.com)
Send RCPT TO command (e.g. foo1#bar1.com)
Send DATA command
Send QUIT command
Are these steps correct? For me it makes sense that the SMTP of the sender processes all the communication, but I've found some pages (e.g. https://www.msxfaq.de/internet/envelope.htm) where a mail server only forwards the message to the next SMTP server and each SMTP server removes one recipient from envelop's rcpt-to list, e.g.
So, what's correct...?
Sender's SMTP handles all the routing
Sender's SMTP removes one recipient (in this case foo#bar.com) and then forwards the message to the next SMTP server
The steps you listed are basically correct.
The complexity arises when you compose an email and try to send it to a lot of different domains, e.g. steve#apple.com, woz#apple.com, and bill#microsoft.com.
In this case, the server must do MX lookups on each one of these domains and split the email. Steve and Woz are both at the same domain, so the server can send an email with those two recipients to the MX record for apple.com. The server must open a separate connection to the MX record for microsoft.com and attempt to send the email to Bill over that connection.
I have a Coldfusion page (CF10) that creates a report and sends it via cfmail to a list of recipients. The ones in our internal domain are receiving, but the external addresses are not.
I found a couple threads where people were having the exact same problem, and the solution was to add the CF server's IP address to the Exchange server. So our IT guys did just that - they added the IP addresses of all of our production servers to the receive connector on the Exchange server. However, the mails are still not being relayed, and I'm continuing to get the same message in the mail log:
javax.mail.SendFailedException: Invalid Addresses; nested exception is: com.sun.mail.smtp.SMTPAddressFailedException: 550 5.7.1 Unable to relay
I'm not familiar with everything that needs to happen to make the Exchange Server relay email to external addresses. Adding the IP addresses to the receive connector obviously isn't enough. Are there other steps that need to be performed?
Coordinate with your IT guys. Write a ColdFusion page that they can run that sends mail to an external address they can access. Then tell them to do what they have to do to receive mail at that address.
Solution was simple - we originally specified the mail server in the cfadmin mail settings, but included no username/password. That worked fine, until we wanted to start sending cfmails to addresses outside of our own domain. So IT created a dedicated "Noreply" account for this purpose, and I entered the username/password in the cfadmin mail settings.
Following Problem: I have written a mailing-list via PHP, SwiftMailer and Postfix. On my Old server it worked without any problems, but Since I moved to another Server (Exactly same configuration, but other IP) the list-mails stuck in the Spamfilter of the Receivers. (Espacially in Google Mail)
Is there anything I have to do escept rerouting the URL DNS-Records, that Google sees, that "I am that Server" and the mail is no spam?
First, check that your outgoing mail server is identifying itself correctly in the HELO command (or EHLO command) when it connects to a receiving mail server. There should also be an A record for this name that should point to the IP address of the mail server. Also, this IP address should reverse to some name (possibly, but not necessarily, the same name as above), and this name should point to the mail server's IP. If any of this is not right, then most spam filters will not consider your server to be a 'real' mail server, and will most likely flag any message sent from your server as spam. This is how many spam filters block spam that originates from computers that have been taken over as 'zombies'.
Another thing to do is to check that you mail server IP is not on any blacklists. You can use MXToolbox for this: http://mxtoolbox.com/blacklists.aspx
Another thing you can do is use port25's verifier tool. This tool will spot any red flags that might be causing your messages to be flagged as spam. See http://www.port25.com/support/authentication-center/email-verification/ for more info.
Last but not least, you might want to setup an SPF record for the domain that you are sending these messages from, to indicate that the IP of your mail server is authorized to send mail from this domain. This will help a lot. For more info, see: www.openspf.org.
How do i send mail on my ubuntu system. I need to send mail in my ruby on rails application but can't test, because mail isn't being sent. What do i need to do on my system. (now testing this app is making me realize why mail wasn't being sent through that Evolution thing i have installed)
The Evolution thing is an MUA (Mail User Agent) like Outlook, Thunderbird, or any number of programs intended for user to read and create email messages. Your program is also a limited MUA (likely creation only).
You need to send your e-mail to an MTA (Mail Transfer Agent). If you configure the return address correctly, you may be able to use the same MTA that you connect to with Evolution. In this case the SMTP server used to send email. If you are using one of the e-mail libraries you should be able to specify the SMTP server name as well as a user-id and password if required.
Many packages require an MTA so you may have Postfix, Exim4, or Sendmail already installed. Check for programs listening on port 25. If so you can use localhost as your SMTP server. If you need to send email other systems you may need to configure your MTA as a satellite to your ISPs server which will relay your message out to the Internet.
As I know the email message sending process looks like this:
The message is sent from sender's client to his server (say mail.com).
The mail.com server redirects the message to recipients server (say gmail.com).
Recipient's client connects to gmail.com and recieves new messages.
I'm interested in stage 2.
I thought that the servers use SMTP protocol to make this negotiation. And gmail.com server should listen to 25 port to recieve a connection from mail.com and other servers.
But then I try to connect gmail.com (exactly, Google service) on port 25, the server is not responding...
Why? What's wrong?
You need to delve into MX records a bit: Oversimplified DNS and MX record. The SMTP gateways use the MX records to route emails between each other. So, for a domain X the SMTP gateway might be mail.X : in other words, for gmail.com the mail gateway might be mail.gmail.com (example).
Mail servers do not redirect.
You have to use MX records to find a mail server. You can't just assume that there will be one on the bare domain name.
But then the mail server will either accept or reject your message. In some cases, usually only between very close and trusted friends, the mail server might RELAY your message onward to another mail server.
At one time, in the mid 90's, just about any mail server would accept a message and relay it. Many people relied on this behavior because IP packets would only travel 30 hops, and the edges of the Internet were more than 30 hops apart. For instance, Peru and Ukraine could not contact each other with IP packets, but they could both send mail to a server at aol.com and that mail server would relay it on to its destination.
It is best not to play around with email unless you read some modern up-to-date descriptions of how it works, and perhaps check the white papers at maawg.org. Also, many domains now expect strangers to submit their email via port 587 and authentication is required.