Max e-mail sizes vary wildly from server to server. Gmail has a cap at 25MB, some ISPs still cap at 10MB or even 5MB, and Exchange servers can be configured to however the individual company wants.
With all that in mind, is there a good recommendation for what the maximum size of a mail should be?
I would like to show my users a "people may not get this e-mail" warning, but I am not sure as to the threshold. Is 10MB still the magic number?
Related
I don't have much experience with servers and costs, so I was looking for suggestions about minimum RAM required on a VPS for an email server that receives emails 24/7, and only occasionally sends out email.
Around 5000 emails per day to many addresses/aliases, with each email being less than 200KB, and mostly without attachments.
The amount of mails grow over time, but right now I'm trying to keep costs as low as possible, and the numbers will stay under these for a while. Thanks!
Edit: I'm planning to use Postal, which requires Ubuntu 17.04 or above.
For a newsletter mailing, about 50,000 users, using pear, is it convenient to order the list by mail provider or leave it all randomly?
From my experiences using Exim to send large amounts of emails, performance will suffer heavily if your email queue grows too large. Depending on your hardware, once you have around 10,000 emails in the queue, you will start to see significant effects of bogosorting, where the server uses more CPU just juggling the queue than actually getting any useful work done.
One way of avoiding large queues, is of course to get the emails delivered as fast and efficient as possible. One of the many ways of achieving this, is to get Exim to deliver multiple emails over the same TCP connection. This in turn can be achieved by sorting the recipients by domain, but that is not enough! By default, Exim will try to deliver each mail it receives immediately and then each delivery will open its own connection (this gives fast deliveries for very small volumes but will drive server load through the roof for larger volumes). You need to first spool the mails to Exim, and then let a queue runner handle the actual delivery which will then automatically see all other emails in the queue that should go to the same host and will deliver them over the same connection.
Optimizing Exim for sending large amounts of emails is a very complex subject that cannot be solved with just a few magical tricks. Crucial configuration options are (but not limited to): queue_only, queue_run_max, deliver_queue_load_max, remote_max_parallel, split_spool_directory, but also fast spool disk, enough RAM, and making sure Exim starts new queue runners often enough (command-line option when starting the Exim daemon).
How this relates to PEAR escapes me, but perhaps this gives you some ideas of how to approach your problem.
I wonder how those email management apps (such as Mailbox) keep checking the inboxes of a large number of email accounts constantly. Do they keep an IMAP connection with the server for each account by sending the IDLE command? It seems to me that this will need a lot of resources by keeping a large number of connections. Is there any more efficient way?
See my answer at https://stackoverflow.com/questions/16065164/automatic-imap-connection-kill-script/16066325#16066325 -- now removed, but available through the wayback machine.
It is, unfortunately, a pretty wide-spread misconception that "a lot of connections" implies "using a lot of resources", but unless both you and your target servers support the NOTIFY extension, there isn't anything better than IDLE that you could use.
I am currently developing an email server in C, and the end goal is to be able to send millions of emails to millions of people every day. Many organizations have email lists with large numbers of users that they email every week/month/etc.
The big question: how can I prevent the server and the emails from being marked as a spam? All of the SPAM-prevention stuff I've seen so far deals mostly with poor configurations, or at least does not require large numbers of emails to be send every hour. I have yet to see anything that addresses the scope of millions-of-emails-per-hour.
Here are some assumptions you can make:
EVERY single email sent is legitimate
all SPF records and MX records are accurate, up-to-date, and valid
all other common SPAM-prevention tactics are being used (reverse DNS is good, DKIM is used, return-addresses are valid, etc etc etc)
emails are one-to-one (ie, I'm not CC'ing 1000 gmail addresses; I'm sending one email to each address)
Here are some questions to get us moving in the right direction:
should I limit the number of emails sent to X emails per minute per domain? If so, how do sites like GMail and MailChimp get around this? note: there are no ISP restrictions; this is only an issue for the receiving mail server...
should I limit the number of connections to a domain at a given time? (eg, will Google think I'm a spam agent if I open 10/100/1000 simultaneous connections to gmail servers?)
how many bounce-backs (5xx errors on an address) should I accept for automatically removing that email from a subscription list? does this affect a server's spam rating?
is there anything else I should or should not do?
Final note: please remember this is a programming question, NOT a library question - I don't want to use someone else's service; we are writing our own for a reason. I'm looking for practical programming advice.
This is not a programming question, but here goes:
I strongly recommend you join your local mail operators mailing list, as well as "Spam-L" mailing list. Read the archives, and see what issues others are having.
The short answer is that destination servers can, and do, use all sorts of methods to try to prevent spam. THere are many things you will need to be aware of in order to have good deliverability, and those things change all the time.
First and most important, remember:
Free speech also includes free listening.
Nobody has to accept or transmit your mail.
Independent operators, businesses and individuals have a perfect right to refuse your mail for any reason or no reason. ISPs are limited only by their contracts with the customer and common-carrier laws, which generally give them broad discretion in what is considered spam and how they block it.
Their system, their rules. If you want your messages delivered, you must cooperate with receiving ISPs. This may mean jumping through hoops, or complying with requirements you think are stupid, or pointless.
Ensure you are not listed by SpamHaus. Most ISPs small and large use SpamHaus DNSBL service. Presence on one of SpamHaus' lists asserts their opinion that your mail meets their listing criteria. Because of SpamHaus' high reputation, most ISPs will simply block all mail you send based on their opinion.
Make sure you process unsubscribes.
Make sure you process non-delivery reports. You may not want to kill a subscription on the first NDR, as there can be intermittent network or server problems which can result in non-delivery, or even erroneous reports that an address is incorrect. But if you get several over the course of a month or two with no successful deliveries, you should kill the subscription.
Join a pay-for reputation service. These may require posting a bond which you may lose if you send Spam. SpamHaus offer one. There are others.
Get professional advice from someone like Return-Path. You will have to pay for this also.
Monitor. The hoops you have to jump through change all the time. Ensure you are aware of emerging deliverability problems.
Join feedback loops. most large ISPs offer feedback programmes where you can get feedback on how users are perceiving your mail, whether they are reporting it as spam, etc.
Ben had some good practical advice, but for others with this problem, here is what I have discovered in the past month:
Email is all about REPUTATION. You will never be able to throw together a server, ip, and/or domain name and expect to be able to send out millions upon millions of emails.
On Stack Overflow, we have a rating system (up and downvotes) to estimate the value/trust that person has with the SO community. But it takes time and effort to get points. It's the same with email - you have to start sending out small amounts of email that people actually open up and read (and would never mark as spam), and then slowly send out more and more every month until you reach the goal of millions and millions of emails.
Everytime someone "downvotes" - marks the email as spam, flags the domain, flags the ip address, deletes the email without reading it, etc - you get a hit against your reputation. You need to be continually monitoring and putting effort and best-practices into your reputation if you want to gain good standing with people.
So start small, expand in a stable and steady manner, and always keep a watchful eye out for abuse, misuses, good and bad feedback, or anything else that might affect your reputation.
It's not only possible, but very practical; you just need to give it time and effort.
I am working on a module that enables our system users to send bulk emails to all the registered active applicants from the applicant pool. Currently, there are more than 10 million active applicants in the pool to which emails can be sent. I am thinking to create blocks of emails and wait for a few minutes before sending individual blocks. What I am more concerned about is the attachment.
Since every email can contain an attachment(max. 2MB), There is a possibility that a huge amount of bandwidth will be consumed, even if the email is sent to only 10,000 applicants (2MB X 10,000 applicants = 20GB bandwidth approximately). My questions are:
Since every attachment is a MIME type, will the size of the email be calculated the the way I have calculated above? Or there is a different mechanism specially in context of bandwidth usage?
In your opinion, what options do I have If I have to send a document to thousands of people and want to save the bandwidth as well? I can put the document on the server and let everybody download, but will it not consume the some amount of bandwidth? (I don't want to go down the FTP route)
Somebody was saying moving these kinds of documents to the cloud?? Does cloud technology offer solutions that cater for this kind of need?
Many thanks,
The attachment creates a problem of being flagged as spam. Best avoid it if you can.
The attachment is MIME encoded rather than gzip compressed. This takes up 1.5 times the bandwidth.
It is not easy to see if the attachment has been opened unless it has some payload that does that for you - again this could be flagged as spam.
Putting these documents on a regular web server will make sense. You can use normal Google Analytics to see what is going on. You can also use public caching to make sure that the document is cached by ISPs etc, thereby reducing your download. The document can also be compressed with gzip to be opened with a browser, unobtrusively doing the un-compressing for your recipients.