We are running few jobs every day using Spring-XD and it might happen the jobs might fail. Spring-XD has dashboard which shows the details of the jobs and even there are REST api's which could be used to fetch the details of the jobs.
We are planning to automate this process by sending out mail notification when there is job failure or sending out of one mail that contains the status of all the jobs.
There are couple of approaches which we can use
Use the REST api's (/jobs/executions?jobname=xx), transform json to html and use the mail sink to send out the mails to the required recipients
Get the status from the Spring-Metadata repository database and use mail-api to send out the mails. This would be like having separate module
We can use the Management over JMX but not sure how to send out mails.
Need your suggestions in arriving on best approach.
You can deploy an XD stream that listens for job events, filters success/failure and send failures to the mail sink
Related
We have written a pyspark code to create a report which we needed in a specific format. The requirement is to send email once the job gets executed successfully with the link of report. I understand how to send a notification on failures, but is it possible to send email on success. Thanks in advance!
I have an [Azure Storage] queue where I put e-mail messages to be sent. Then, there is a separate service which monitors that queue and send e-mails using some service. In this particular case I'm using SendGrid.
So, theoretically, if the sender crashes right after a successful call to SendGrid Mail Send API (https://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/index.html), the message will be returned to the queue and retried later. This may result in the same e-mail being delivered more than once, which could be really annoying for some type of e-mail.
The normal way to avoid this situation would be to provide some sort of idempotency key to Send API. Then the side being called can make sure the operation is performed at most once.
After careful reading of SendGrid documentation and googling, I could not find any way to achieve what I'm looking for here (at most once semantic). Any ideas?
Without support for an idempotency key in the API itself your options are limited I think.
You could modify your email sending service to dequeue and commit before calling the Send API. That way if the service fails to send the message will not be retried as it has already been removed from the queue, it will be sent at most once.
Additionally, you could implement some limited retries on particular http responses (e.g. 429 & 5xx) from SendGrid where you are certain the message was not sent and retrying might be useful - this would maintain "at most once" whilst lowering the failure rate. Probably this should include some backoff time between each attempt.
I want to develop a mail notification service to send order approval to customer. The order data is in the denormalized view (query side) and it should be filled to the mail template. Then, we send the email in html string format via mail notification service. But, the order status should be changed to "order approval email sent".
I also try to implement the CQRS, ES, and DDD concept in microservices architecture.
Is this procedure correct and still align with the concept?
Develop HTTP POST API in order command to send approval mail so the order status could be changed in command-side.
The command side generate the event "order approval mail processed"
The event processor process the event. It should get the order data from query-side / denormalized view.
The event processor generates the approval mail from the data and fill the data to the template.
The event processor call HTTP POST to the mail notification service with mail body (html format) in the payload.
The event processor call HTTP PUT to the order service (command-side) to change the order status to "order approval mail sent".
But, if this procedure is applied, the user can't get the response "mail sent" in real-time. How to trigger the client / front-end side that the mail is successfully sent? So, the client side don't have a need to refresh or retry many calls to the API.
Thanks.
I've written a post on this subject a while ago, you can find it here: How to Send Emails the Right Way in a CQRS System
The short version is that I would use a process manager. A process manager listens to events and can issue commands as a result of these events. Just make sure you have a mechanism to not re-send emails if you ever re-run your events.
Regarding the UI. I have another post dealing with this question. You can find it here: 4 Ways to Handle Eventual Consistency on the UI
Here is a short answer. How often do you think once the code is run to send the email, that the email fails to send? Assuming you have a reasonably robust system, I would hope the vast majority of the time it would work. So fake it. And only if there is a problem find some way to notify the user and or admin users. If you want to get fancy you can use things like Signalr or some pub-sub framework for sending messages to the UI.
Anyway - hope that helps.
Currently our server connects to the Amazon SES SMTP interface to send out email reminders to clients, emails that are undelivered and bounce back are forwarded onwards using the SNS service.
All this works perfectly.
There has never been the capability to store the email reminders that get sent out as the server simply fires and forgets.
I would like to add this capability using the AWS architecture to capture and store the emails.
I think this possible using some combination of SES and SQS and S3 but has anyone got any suggestions on how to architect this requirement within AWS to achieve the goal? I am unsure of the best approach and I'm concerned that I might be missing some component of AWS that could easily achieve this.
I found the solution hidden in the identity management section of AWS, it is here that you set up the process of forwarding email notifications.
Location of notifications
Step 1 Click on the domain to edit the notifications and create a new SNS (Push Notification Service) topic for each type of notification (Bounce, Complaint or Delivery)
Step 2 Create a new Queue (SQS)
Step 3 Go to SNS and create a new subscription to link the queue and the notification together
Step 4 Use Lamdba to process your queue and direct the output to wherever you wish
Note If you are sending large quantities of email then this solution may cost considerably more and you should research different solutions including using Cloudwatch and Kinesis
I am trying to have a kind of observer pattern in ColdFusion
We want to listen to the incoming Email messages and act on them. Scenario is something like this :
Application sends email to the helpdesk system
Helpdesk system automatically generates a ticket and responds with an email to the email address of the application
The application's email is configured in the Lotus notes
Now the application should listen to this incoming email message, decode that and update the coressponding ticketid
I see there is a possibility with Event Gateways, but I am unable to realize the whole picture.
Thoughts or suggestions?
One way is to setup an email server with IMAP support, and use some sort of polling (every minute, good enough?) in CF using <cfimap> to get the emails.
http://help.adobe.com/en_US/ColdFusion/9.0/CFMLRef/WS371453EC-36D5-44ce-BF1E-750E3016BBD6.html
We have a system like this.
We have a postfix server configured to handle mail for a domain. A small script (Perl) on the postfix server places each email on an ActiveMQ queue.
We have a cluster of CF boxes with the ActiveMQ event gateway listener that takes the messages off the queue and processes them using Java Mail.
The delay between postfix receiving the email and a CF server processing it is generally under 1s.
We needed to do it this way for a number of reasons, processing delay being one of them, dealing with a large cluster of CF which made the POP/IMAP solution complicated, and CF's mail handling not being quite what we wanted were others.
It works great.
I've created similar applications in the past using cfpop to interogate a mailbox on a scheduled basis.
It was pretty easy to write, but usually gets thrown for a loop when "users" start being "helpful" with the email content.
The other thing is that this isn't instantaneous, but is the process really time critical to the second?