Send a limited number of emails every hour with Perl Mime::Lite - perl

I know that Perl Mime::Lite is deprecated, but I have to work on a hosted server where only Mime::Lite is installed. This server also limits to 500 the number of emails that can be sent every hour.
I have a large list of participants that need to be emailed instructions to complete questionnaires and reminders if they haven't completed their questionnaire weeks later. I have a script that check if they have completed their questionnaire and if a reminder should and has been sent or not. Otherwise a reminder is sent. However, I have to limit the number of emails sent to 500 per hour.
Is there a way to tell Mime::Lite to send 500 emails, wait 1 hour and then send 500 other emails or do I need to program it myself in perl using external files: Sending 500 emails, marking that those emails have been sent and at what time. Every time the script is run, it checks again if an email has to be sent and at what time the last email has been sent. If it is more than one hour from current time, it sends 500 new emails.
Or any other more convenient ways?
Just to be sure, my emails are legitimate and expected by the users (and wanted).

Mime::Lite itself doesn't implement this, but it's easy to implement yourself. Assuming you have a sub send_to($recipient, $msg) that actually uses Mime::Lite to send the message, you can wrap it with something like:
my $msg = ...;
my #recipients = ...;
while (#recipients) {
for (1 .. 500) {
last unless #recipients; # batch is implicitly over if we're out of people to send to
send_to(shift #recipients, $msg);
}
sleep 3600 if #recipients; # wait an hour before the next batch
}
Note that this is contingent upon your host server allowing you to keep a process running for enough hours to work through the entire list. If they don't, then you'll need to work up something with a database to track all recipients and which have already been mailed to.
Honestly, though, it would probably be better (and likely easier) to use real mailing list software to handle this instead of writing your own semi-functional list server. Perhaps your hosting service offers mailing lists as well?

Related

Service invoked too many times for one day: email with only 19 emails sent on a Time Trigger

I have looked over all the MailApp limitations over at:
https://developers.google.com/apps-script/guides/services/quotas
The company I work for has their own Google domain and should qualify for over 1,500 emails a day. I am getting this error no where near that number.
Upon writing this code and debugging at the start of the day:
var y = MailApp.getRemainingDailyQuota();
It appears to say I have a maximum of 100 emails a day from the MailApp command. If my company is using Google's services, why do I qualify for only 100 emails instead of 1,500?
Furthermore, why is the service being blocked if I send significantly under 100 emails?
You question can be answered with the following information
Description of the getRemainingDailyQuota():
Returns the number of remaining emails a user can send for the rest of the day. Quotas are based on the number of email recipients.
Email recipients per day:
This means that if your company's Google Domain is new, you might be
temporary limited to 100 email recipients per day, which can result in less than 100 emails per day.

The fast way to execute rest requests that require incremented value (nonce)

I'm working with Rest Api that requires an incremented parameter to be sent with each request. I use unix miliseconds as nonce and originally naively sent requests one after another but even if I send one message before another, they can arrive in a reversed order which results in an error.
One solution could be sending the next request only after the previous one got back. But it would be too slow. I'm thinking about less strict solution like measuring latency over the last 10 requests and waiting for x% of latency before sending the next message. I feel like this problem should've been already solved but can't find any good reference. Would appreciate any advice.

rate limiting outgoing e-mails with postfix

I am renting a VPS that blocks outgoing port 25 so that I have to use their relayhost. Which works fine accept for one thing. The relayhost has these restrictions for outgoing mails:
1000 mails / hour
3000 mails / day
20000 mails / month
Exceeding these limits either cost more money or result in being banned.
I therefore would like to set the same restrictions in my own postfix server in such a way that mails stay in the defer queue if the outgoing limit will be reached. I don't mind that mail is delayed a few hours in order to stay within the limit of the relayhost.
There does not seem to be a postfix setting that will do this out of the box. However for incoming mails there are settings like smtpd_client_recipient_rate_limit and anvil_rate_time_unit that can throttle incoming mails.
I was therefore thinking of putting 3 additional smtpd processes in master.cf each of which sets smtpd_client_recipient_rate_limit and anvil_rate_time_unit according to the 3 rate restrictions.
Is this the most practical approach or is there a simpler solution?
I ran into a similar issue with my VPS host saying I'm sending too fast. I used the below config to slow the email rate.
default_destination_rate_delay = 5s
This puts a 5 second delay between each outbound smtp connnection to the same destination. There are other default_destination parameters that maybe of use to you on the man page
http://www.postfix.org/postconf.5.html

How should I send scheduled e-mails?

My question is theoretical.
I have a database with e-mails. For each email I store the desired sending time (as an UNIX timestamp) and the contents of the e-mail (sender, receiver, subject, body, etc.). There's a large number of e-mails scheduled.
This's how I wanted to send the e-mails so far:
I would have a worker process or server which periodically queries the database for "overdue" e-mails based on the timestamps. Then it sends those e-mails, and in the end it deletes them from the DB.
I started to think about two things:
What if the worker dies when it has sent the e-mail but hasn't
deleted it from the database? If I restart the worker, the e-mail
will be sent again.
How do I do it if I have a really large number of
e-mails and therefore I run multiple workers? I can mark an e-mail in
the database as "being sent", but how do I re-initiate sending if the
responsible worker dies? I mean I won't know if a worker has died or
it's just so slow that it's still sending the messages. I'm assuming I cannot get notified about a worker has died, so I can't re-send the e-mails that it failed to send.
I know that e-mail sending is not a so serious thing like bank transactions, but I think there must be a good solution for this.
How is this used to be done?
I would actually use a flag on each email record in the database:
Your worker (or multiples) update the oldest record with their unique worker ID (e.g. a PID or IP/PID combination).
Example for Oracle SQL:
update email set workerid = 'my-unqiue-worker-id' where emailid in (
select emailid from email where
rownum <= 1 and
duetime < sysdate and
workerid = null
order by duetime
)
This would just take 1 not yet processed record (ordered by duetime, which has to be in the past) and set the worker ID. This procedure would be synchronized by the normal database locking mechanism (so only one thread writes at the same time).
Then you select all records with:
select * from email where workerid = 'my-unique-worker-id'
which will be either 0 or 1 record. If it is 0, there is no due mail.
If you have finished sending the email you set the workerid = 'some-invalid-value' (or you use another flag-column to mark the progress. That way it doesn't get picked up by the next worker.
You probably won't be able to find out if the email really has been sent. If the worker dies after sending and before updating the record, there's not much you can do. To be a bit more self-sufficient the worker could create a process file locally (e.g. an empty file with the emailid as the file name. This could at least detect if the crash was just a database connection issue..
If the worker is started and before updating any record already finds a message, which has its ID as the workerid then I would raise an alert / error which should be handled manually (by checking the SMTP server log and manually updating the record).

Throttling in VBA

The Back Story
A little while back, I was asked if we could implement a mass email solution in house so that we would have better control over sensitive information. I proposed a two step plan: Develop a prototype in Excel/VBA/CDO for user familiarity, then phase a .Net/SQL server solution for speed and robustness.
What's Changed
3 Months into the 2nd phase, management decides to go ahead and outsource email marketing to another company, which is fine. The 1st problem is that management has not made a move on a company to go through, so I am still implicity obligated to make the current prototype work.
Still, the prototype works, or at least it did. The 2nd problem came when our Exchange 2003 Relay Server got switched with Exchange 2010. Turns out that more "safety" features are turned by default like Throttling Policies, which I have been helping the sysadmin iron out a server config that works. What's happening is that the after +100 emails get sent, the server starts rejecting the send requests with the following error:
The message could not be sent to the SMTP server. The transport error code is 0x800ccc67.
The server response was 421 4.3.2 The maximum number of concurrent connections has exceeded a limit, closing transmission channel
Unfortunately, we only get to test the server configuration when Marketing has something to send out, which is about once per month.
What's Next?
I am looking at Excel's VBA Timer Function to help throttle my main loop pump to help throttle the send requests. The 3rd problem here is, from what I understand from reading, is that the best precision I can get is 1 second on the timer. 1 email per second would be considerably longer ( about 4x-5x longer) as oppossed to the 5 email/sec we have been sending at. This turns a 3 hour process into a an all day process past the hours of staff availability. I suppose I can invert the rate by sending 5 emails for every second that passes, but the creates more of a burst affect as opposed a steady rate if had more precision on the timer. In my opinion, this creates a less controlled process and I am not sure how the server will handle bursts as opposed a steady rate. What are my options?
You can use the windows sleep API if you need finer timer control. It has it's units in milliseconds:
Private Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Public Sub Testing()
'do something
Sleep(1000) 'sleep for 1 second
'continue doing something
End Sub
I'm not very familiar with Exchange, so I can't comment on the throttling policies in place.