Send out 1 email after all Oozie jobs finish - email

So far, I've been using oozie email actions to send out email enriched with some information, when submitting job in our hadoop cluster :
<workflow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:workflow:0.1">
...
<action name="[NODE-NAME]">
<email xmlns="uri:oozie:email-action:0.2">
<to>[COMMA-SEPARATED-TO-ADDRESSES]</to>
<cc>[COMMA-SEPARATED-CC-ADDRESSES]</cc> <!-- cc is optional -->
<subject>[SUBJECT]</subject>
<body>[BODY]</body>
<content_type>[CONTENT-TYPE]</content_type> <!-- content_type is optional -->
<attachment>[COMMA-SEPARATED-HDFS-FILE-PATHS]</attachment> <!-- attachment is optional -->
</email>
<ok to="[NODE-NAME]"/>
<error to="[NODE-NAME]"/>
</action>
...
</workflow-app>
This is for sending out emails after each submitted workflows.
Let's say i have 560 jobs running and instead of receiving 560 emails, i would like to receive only 1 single email that will contain a sum up of what happened in that specific business day ( successes, failures etc..)
I am not finding any solution on how to implement this.
Has anyone already tried something like that, and if yes, could you please point me to some possible approach to this idea?
Please comment out if what i am trying to achieve is not clear enough,
Thanks

Related

How to get MAM message ID of sent messages?

I am using eJabberd server (MAM enabled) with client library Strophe.js. Client app is storing fixed amount of the messages in local storage.
All the messages, that I receive from the server include elements <archived/> and <stanza-id/>, which provide server-side generated IDs:
<message
xmlns="jabber:client" to="aaa#example.net/8667085700924567016834" from="aaa#example.net">
<result
xmlns="urn:xmpp:mam:2" id="1520510373346685">
<forwarded
xmlns="urn:xmpp:forward:0">
<message
xmlns="jabber:client" xml:lang="en" to="bbb#example.net" from="aaa#example.net/60965696931000870402419" type="chat">
<archived
xmlns="urn:xmpp:mam:tmp" by="aaa#example.net" id="1520510373346685"/>
<stanza-id
xmlns="urn:xmpp:sid:0" by="aaa#example.net" id="1520510373346685"/>
<body>asdf</body>
</message>
<delay
xmlns="urn:xmpp:delay" from="example.net" stamp="2018-03-08T11:59:33.346685Z"/>
</forwarded>
</result>
</message>
I use these IDs to get fixed amount of messages on each MAM call (using RSM paging - before/after/max attributes). For example:
params = {
"before": "1520510373346685",
"max": 10,
onMessage: <some handler>,
onComplete: <some handler>,
}
this.connection.mam.query(Strophe.getBareJidFromJid(this.myJid), params);
This gets me 10 messages before the message with stanza-id 1520510373346685.
However, the messages I send inherently do not have this ID, until I send them and retrieve from MAM.
I did not find any examples on getting the currently sent message ID from the server, and the Strophe does not provide any callbacks after the message was sent from the client and received by the server.
So, the question - is there any way to get the server-generated message ID (stanza-id/archived) for the currently sent message?
Maybe my own approach is wrong - should I generate my own IDs and add it to each message from the client?
There is currently no mechanism specified besides querying the archive. One prominent idea within the XMPP community to solve this in the future is to reflect the send carbon back to the original sender, which would include a stanza-id element with the ID assigned by the archive.

Simultanous sending and receiving

I am trying to send and receive message to and from JMS topics.
<testcase name="DeliveryToPT3PLIT">
...
<actions>
...
<send endpoint="fromEndpoint">
<message>
<resource file="com/roche/icc/citrus/messages/input/PT-3PLWoBatchSplit.xml"/>
</message>
<header>
...
</header>
</send>
<receive endpoint="toEndpoint">
<description>Receive asynchronous message from topic</description>
<message>
<resource file="com/roche/icc/citrus/messages/output/PT-3PLWoBatchSplit.xml"/>
</message>
<header>
...
</header>
</receive>
</actions>
</testcase>
It seems that these operations go one after another. The problem is that my application works really fast and when I send a message to first topic it appears on "toEndpoint" almost immediately. So the receive operation does not manage to catch appropriate message since it has been already processed.
Is there any way to make this operations simultaneously?
Regards
There is one thing to note about using JMS topics, because they work in a publish/subscribe manner, you have to subscribe first in order to receive the message.
That means you need to subscribe to the topic before a message is going to be published. Also, if you say that your application is fast, you may need to wait a few millis before sending the message. Here is a Java DSL example of what I think may work for you. (Note: I have tested this example with a JMS topic)
parallel().actions(
sequential().actions( // Thread #1
...
receive(action -> action.endpoint(toEndpoint).payload("message to be received"))),
sequential().actions( // Thread #2
sleep(500),
send(action -> action.endpoint(fromEndpoint).payload("message to be sent"))));
Notice the parallel() action container. It will execute every action inside it in a separate thread. Each sequential() represents a series of actions that allows you to group them together. Because both sequential() containers are inside the parallel(), the actions inside each of them will be executed on different threads.
So what happens here: you split your actions on two threads. On the first thread you wait for a message. On the second thread, you make it sleep for 500 millis first, after which you send the message.
I'm sorry that I don't have an XML example, but I hope this helps anyway.
Also don't forget to set the pub-sub-domain property to true on your endpoints. See the Citrus documentation on JMS topics: here.

How to get the latest email using CFIMAP

I am using the below code block in CF9 to get all the mail data,
<cfimap
action="GETHEADERONLY"
name="LOCAL.checkEmail"
secure="true"
folder="Inbox"
server="#ImportAquireMailServer#"
username="#ImportAquireUsername#"
password="#ImportAquirePassword#" />
This query object provides the information about all the mails present in inbox. While doing so it takes a lot of time and also times out frequently.
Is there any way to get the latest mail only, so that it will not provide the information about all the mails and will also not run into time out situation?
Please help.
you may need to update the read messages to "read" like:
<cfimap action="MarkRead" connection = "Conn" messagenumber="#getHeaders.messagenumber#">
after the message id done with work.
so you can query the latest messages as:
<cfquery dbtype="query" name="getMails">
select * from getHeaders
where seen=<cfqueryparam value="no" cfsqltype="cf_sql_varchar">
</cfquery>also you can filter with received date
you can refer http://shemy-coldfusion.blogspot.in/2013/03/coldfusion-code-to-get-multiple-mail.html

Debugging 500 Response for Create Sales Receipt Through API Explorer

I've been developing an IPP application using the PHP devkit. I've been using the XML returned from the $object->asIDSXML() method to debug my objects through the API Explorer. Within the API Explorer I have been recieving a very ambiguous error when trying to create a Sales Receipt and I'm hoping I can learn more about the scheme requirements. Is there a validation xsd file somewhere? I'm currently unable to understand why I receive the following 500 code.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <FaultInfo
xmlns="http://www.intuit.com/sb/cdm/baseexceptionmodel/xsd">
<Message>Internal Server Error</Message>
<ErrorCode>500</ErrorCode>
<Cause>SERVER</Cause> </FaultInfo>
The documentation regarding a 500 states something along the lines of :
Possible Causes:
Illegal permissions.
Illegal values that are not verified in the validation process.
Invalid data that the business logic verified.
I know that I have proper permissions, I have authenticated with an OAuth Token that has all privileges. The later two possibilities I believe are the issue, but I don't know how to get any additional information regarding the potentially invalid or missing data. I have literally tried creating a duplicate by using the API explorer to retrieve the SalesReceipt XML and then removing the DocumentID and sending to a create POST through the API explorer at which point I receive the same error shown above.
The following is the XML post data provided to the API. As far as I can tell the xml contains all the required fields fore creating a Sales Receipt. Thanks in advance for any help.
<SalesReceipt>
<Header>
<TxnDate>2013-04-16-07:00</TxnDate>
<CustomerId idDomain="QBO">78</CustomerId>
<TotalAmt>4.00</TotalAmt>
<ShipAddr>
<Line1>123 Fake Ave</Line1>
<City>Fake City</City>
<CountrySubDivisionCode>ID</CountrySubDivisionCode>
<PostalCode>83854</PostalCode>
</ShipAddr>
</Header>
<Line>
<Id>1</Id>
<Desc>Face to face IT Consulting Services, providing solutions to client's IT issues.</Desc>
<Amount>4.00</Amount>
<Taxable>false</Taxable>
<ItemId>17</ItemId>
<UnitPrice>4.00</UnitPrice>
<Qty>1</Qty>
</Line>
</SalesReceipt>
Can you please try the following
<SalesReceipt xmlns='http://www.intuit.com/sb/cdm/v2'>
<Header xmlns:ns3='http://www.intuit.com/sb/cdm/v2'>
<ns3:TxnDate>2013-04-16-07:00</ns3:TxnDate>
<ns3:CustomerId>78</ns3:CustomerId>
<ns3:TotalAmt>4.00</ns3:TotalAmt>
<ShipAddr>
<Line1>123 Fake Ave</Line1>
<City>Fake City</City>
<CountrySubDivisionCode>ID</CountrySubDivisionCode>
<PostalCode>83854</PostalCode>
</ShipAddr>
</Header>
<Line xmlns:ns11='http://www.intuit.com/sb/cdm/v2'>
<ns11:Id>101</ns11:Id>
<ns11:Desc>Face to face IT Consulting Services, providing solutions to client's IT issues.</ns11:Desc>
<ns11:Amount>4.00</ns11:Amount>
<ns11:Taxable>false</ns11:Taxable>
<ns11:ItemId>17</ns11:ItemId>
<ns11:UnitPrice>4.00</ns11:UnitPrice>
<ns11:Qty>1</ns11:Qty>
</Line>
</SalesReceipt>

How to send a response to the invoker of a service without reaching the end of the pipeline

I have 2 custom actions exectuting one after the other in the same service like this
<actions mep="RequestResponse">
<action name="ActionA" class="ClassA"/>
<action name="ActionB" class="ClassB"/>
</actions>
Suppose that ActionA does some validations over the received msg. If the validations found that the msg is invalid, how does you send a response to the invoker about that failure?
Right now in my actions I set the response in the message at ActionA and put a mark in it indicating that I found an error in ActionA and ActionB checks for that mark before executing its code. I found this method useful but a burden because all of my actions have to start with:
if (!markIspressent) {
//Code goes here
}
return message.
I have tried setting the response msg at ActionA and returning null to stop the pipeline but that isn't working. I also tried another method that I found of throwing an ActionProcessingFaultException(message,"SomeTextGoesInHere") but that also isn't working.
My main problem with this second one is that the ESB tries to reprocess the msg that thrown that exception and I don't see the response that I set into message until the ESB gives up and sends it back. But that giving up takes up to 60s.
So my question is how can you send a response msg to the invoker before reaching up the end of the service pipeline.
Thanks
The options in finishing earlier the pipeline are:
- return null;
- split the service into three (note service 1 is OneWay and based on your action's results routes to the appropriate service)
Service 1
<actions mep="OneWay">
<action name="ActionA" class="ClassA"/>
<action name="ActionA1" class="RouteBasedOnMsgContentTo(Service 2 or Service 3)"/>
</actions>
Service 2
<actions mep="RequestResponse">
<action name="ActionB" class="ClassB"/>
</actions>
Service 3
<actions mep="RequestResponse">
<action name="ActionC" class="ClassC-SimplyRespondsTheProblem"/>
</actions>