Say I have an app that send an email to a user when something interesting in the system happens. The system would add a sendNotification job to a message queue, which when processed sends out the email.
I want to be able to add these jobs to the queue whenever something interesting happens, but have the system wait for 10 minutes to gather and group all these notifications into one email, rather than send out multiple email within that short timeframe.
How would this be implemented in a system such as RabbitMQ? As far as I can tell delay functionality isn't built in for a start, and you can't update messages once they are sent. So are there any patterns recommended for implementing this kind of thing?
Related
I scheduled some emails through SendGrid without using the "batch" functionality and I would like to cancel them. It doesn't seem like there is a cancel function through the api if not done using the batch functionality.
Is there a way to ensure that the emails fail to send at their scheduled time by doing some hackery, like: temporarily blocking/blacklisting email addresses, closing the account entirely, revoking keys, changing dns entries, etc?
You can stop the scheduled emails by canceling the schedule. You can find more information here. If the schedule has started and you want to stop the emails from being sent then you may need to clear the Sendgrid Queue. More information - Link.
I've just read this excellent question about the advantages of sending an email asynchronously vs synchronously.
I fully agree with the approach that the single answer concludes, which is: store the email to send in a queue, and return the page to the user as soon as possible.
My current implementation uses a database table to store the email queue. This approach has several advantages:
It's very fast: saving a record to a table takes no time, as opposed to placing an API call (emails are sent through Amazon SES).
It's transactional: if anything goes wrong, the transaction is rolled back, and my email will never be sent for an account that wouldn't have been committed to the database.
Then I've implemented a job that runs permanently, and will check on a regular basis (currently 30s) if there are some mails to send, and process them.
I'm happy enough with this solution, apart from the fact that it's not exactly instant. People often expect to receive a mail (even more if it's a SMS) in the next few seconds.
I could obviously make the job pause no more than one second between two runs, but I don't want to waste server resources needlessly with frequent transactions. Also, because my emails are sent through an API call, I'd benefit from concurrent processing, which my current implementation does not do (it processes them sequentially).
Are there some common solutions to still process the email asynchronously, but start processing as soon as the email has been sent to the queue?
When you put an e-mail into the database, you also send a notification to your e-mail sending job to wake it up and send the new e-mail. Rather than waiting for the sending job to check, it is actively notified and deals with the new e-mail immediately.
I'm quite new to Google App Script and playing around with GmailApp.sendEmail.
When I try to send several Email with the same subjects, how can I make them grouped in one single Conversation (given that the Gmail of receiver has Conversation option turn on).
Many thanks,
Minh
Since you're using GmailApp, take a look at the Gmail Thread Class. If you use thread.replyAll(body, options) to respond to the first message you sent, the second message will appear in the same thread in the receiver's mailbox.
When sending the first message, you would need to cc yourself for it to end up in your own inbox, then you could use [GmailApp.search()][2] to find the threads with messages matching the one you sent. From there, drill down to the thread, and replyAll.
With regards an OMS, what is the best method to send a confirmation email? The 2 options I have so far are;
A script on the order page sends an email once the record is written to the database.
A scheduled task on the server, send the email, polling the database every-so-often to find new entries.
Which method do systems currently use?
For e-commerce websites, it might be better to think about the best user experience.
Given that, you would want to send the email as soon as the order is received so the user knows that they have purchased the item. The sooner it gets into their inbox, the sooner they will be happy that they have made their purchase.
I agree with Digbyswift that sending the confirmation email once the record is written to the database is the least scalable. But I would argue that if your system has gotten to the point that you are taking so many orders that your system cannot keep up, you have a wonderful problem on your hands that you now probably have resources to handle.
At PostageApp, we handle the emails of a few e-commerce websites, so perhaps you would benefit from an arrangement with an email service provider to off-load this task so that all of your resources can be spent on keeping your site up and your databases running.
Here are some great alternatives if PostageApp is not your style:
Sendgrid
Postmark
Mailjet
This is a question of scalability. Sending a confirmation email once the record is written to the database is the least scalable. The more orders that are taken , the more emails are sent potentially tying up resources.
A scheduled task is certainly better as emails can be queued up and can be sent in a separate process.
A further option which you could consider is using neither and delegating the responsibility of sending emails to a 3rd party dedicated emailing service, i.e. via an API. This is much better since your hosting does not have to consider the load and you can utilise any reporting offered by the 3rd party. Plus many services offer a free quota up to a certain threshold. This will allow you OMS and business to scale appropriately.
If you apply a message based architecture; you could just publish an order created message and have any number of subscribers respond to that event. You could create a listener that sends the email in house (bespoke option) or another listener that called the API of a 3rd party emailer to send the email on your behalf (as per #Digbyswift)
What I've always liked about this approach is
You can have any number of listeners live at any one time.
You can create a new listener and change how you send the email without needing to change/redeploy the OMS application itself.
You can take the listener(s) off line and stop / delay the sending of the email without losing any notifications or affecting the OMS itself.
I'm looking for some opinions here, I'm building a web application which has the fairly standard functionality of:
Register for an account by filling out a form and submitting it.
Receive an email with a confirmation code link
Click the link to confirm the new account and log in
When you send emails from your web application, it's often (usually) the case that there will be some change to the persistence layer. For example:
A new user registers for an account on your site - the new user is created in the database and an email is sent to them with a confirmation link
A user assigns a bug or issue to someone else - the issue is updated and email notifications are sent.
How you send these emails can be critical to the success of your application. How you send them depends on how important it is that the intended recipient receives the email.
We'll look at the following four strategies in relation to the case where the mail server is down, using example 1.
TRANSACTIONAL & SYNCHRONOUS
The sending of the email fails and the user is shown an error message saying that their account could not be created. The application will appear to be slow and unresponsive as the application waits for the connection timeout. The account is not created in the database because the transaction is rolled back.
TRANSACTIONAL & ASYNCHRONOUS
The transactional definition here refers to sending the email to a JMS queue or saving it in a database table for another background process to pick up and send.
The user account is created in the database, the email is sent to a JMS queue for processing later. The transaction is successful and committed. The user is shown a message saying that their account was created and to check their email for a confirmation link. It's possible in this case that the email is never sent due to some other error, however the user is told that the email has been sent to them. There may be some delay in getting the email sent to the user if application support has to be called in to diagnose the email problem.
NON-TRANSACTIONAL & SYNCHRONOUS
The user is created in the database, but the application gets a timeout error when it tries to send the email with the confirmation link. The user is shown an error message saying that there was an error. The application is slow and unresponsive as it waits for the connection timeout
When the mail server comes back to life and the user tries to register again, they are told their account already exists but has not been confirmed and are given the option of having the email re-sent to them.
NON-TRANSACTIONAL & ASYNCHRONOUS
The only difference between this and transactional & asynchronous is that if there is an error sending the email to the JMS queue or saving it in the database, the user account is still created but the email is never sent until the user attempts to register again.
What I'd like to know is what have other people done here? Can you recommend any other solutions other than the 4 I've mentioned above? What's a reasonable way of approaching this problem? I don't want to over-engineer a system that's dealing with the (hopefully) rare situation where my mail server goes down!
The simplest thing to do is to code it synchronously, but are there any other pitfalls to this approach? I guess I'm wondering if there's a best practice, I couldn't find much out there by googling.
My 2 cents:
Once you have a user sign up, never roll back the registration if sending the E-Mail fails. For simple business reasons: They may not come back or re-register if it doesn't work out at the first try. Rather tolerate an incomplete registration and nag the user to confirm their E-Mail address as soon as possible.
In most cases when sending an E-Mail goes wrong, your app will not get immediate feedback anyway - non-existent E-Mail addresses on valid servers will send back a "undeliverable" message with some delay; if the mail gets eaten by a spam filter, you'll get no feedback at all; in other scenarios, it may take several minutes (greylisting) to several days (mail server temporarily down) for an E-Mail to get delivered. A synchronous approach waiting for the delivery of the mail is therefore doomed IMO. Even an immediate failure (because the user entered a obviously fake address) should never result in the registration getting rolled back.
What I would do is, make account creation as easy as possible, allow the user access to the account before it is confirmed, and then nag the hell out of them to confirm their E-Mail (if necessary, limit access to certain areas until confirmation). I would prevent the creation of a second account with the same E-Mail, though, to prevent clutter.
Make sure you allow changing the E-Mail address even if the previous address hasn't been confirmed yet, and enable the user to re-request the confirmation message to a different address.