How to implement tLoop in Talend? - talend

I'm new to Talend and need an example job to implement tLoop. I want to run a job 10 times if it fails. I've looked at the documents, but I can't seem to figure this out.

This answer has 2 sections
Creating a loop with tJava
Retying a failed connection to a data source 5 times (with adding tJavaFlex)
___________________________________
SECTION 1 : Creating a loop with tJava
-----------------------------------------------------------
I just write a tJava component and then iterate to false.
Like this
Step 1: create a context variable
Step 2: write some java code in tJava (tJava1)
// setting loop flag
context.continueLooping = true;
//log.info("Starting job...");
then connect On Component Ok
Step 3: Create the tLoop
in the loop condition put your context context.continueLooping
which should be true by the first iteration.
then iterate
to the next tJava (tJava2)
if ( ((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) == 1)
{
// code
}
else if(((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) == 2)
{
// code
}
else if (((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) == 3)
{
// code
}
else if (((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) == 4)
{
// code
}
else if (((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) == 5)
{
// code
context.continueLooping = false;
// log.info("DONE");
}
else
{
context.continueLooping = false;
// log.error("out of bounds...");
}
this tJava runs different code for each iteration till it reaches 5
I use this area to count stuff and load value to other contexts and more.
Then it runs the nest part n times till the context value is set to false.
___________________________
SECTION 2 : TO Retry Failed Connections
___________________________
if you need to retry a DB connection.
add a tJavaFlex between tLoop1 and tJava2 like so
and add the following code in the 3 sections
Start:
// start part of your Java code
try{
Main:
// here is the main part of the component,
// a piece of code executed in the row
// loop
if ( ((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) > 1)
{
Thread.sleep(10000);
}
End:
// end of the component, outside/closing the loop
}catch (Exception e) {
if ( ((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) > 5)
{
context.continueLooping = false;
}
else
{
System.out.println("Connection failed. Retrying...next");
}
}
and add On Component Ok tJava with the code to stop looping on the success (tJava3)
context.continueLooping = false;

Related

How to use simulation time to triger action in modules?

I want to create simulation with server and 2 nodes. Nodes have defined vector that contain times for turn on/off.(example timersOnOff = 5,10,13,25 … nod will turn on in 5th second of beginning simulation, and then will be shutdown in 10th seconds etc). How to trigger action at certain time to send msg to node to "turn on" or "turn off".?
Let's assume that these times are written in timersOnOff declared as:
std::vector<simtime_t> timersOnOff;
In initialize() add the following code:
for (int i = 0; i < timersOnOff.size(); i = i + 2) {
simtime_t timeOn = timersOnOff[i];
simtime_t timeOff = timersOnOff[i+1];
cMessage * msgOn = new cMessage("nodeOn"); // (1)
cMessage * msgOff = new cMessage("nodeOff"); // (2)
scheduleAt (timeOn, msgOn);
scheduleAt (timeOff, msgOff);
}
The above code schedules all ON and OFF events.
Then, in handleMessage() add:
if (msg->isSelfMessage()) {
if (msg->isName("nodeOn")) { // the same name as in (1)
delete msg;
// turning on the node
} else if (msg->isName("nodeOff")) { // the same name as in (2)
delete msg;
// turning off the node
}
} else {
// ...
}

Using a Beakerx Custom Magic

I've created a custom Magic command with the intention of generating a spark query programatically. Here's the relevant part of my class that implements the MagicCommandFunctionality:
MagicCommandOutcomeItem execute(MagicCommandExecutionParam magicCommandExecutionParam) {
// get the string that was entered:
String input = magicCommandExecutionParam.command.substring(MAGIC.length())
// use the input to generate a query
String generatedQuery = Interpreter.interpret(input)
MIMEContainer result = Text(generatedQuery);
return new MagicCommandOutput(MagicCommandOutcomeItem.Status.OK, result.getData().toString());
}
This works splendidly. It returns the command that I generated. (As text)
My question is -- how do I coerce the notebook into evaluating that value in the cell? My guess is that a SimpleEvaluationObject and TryResult are involved, but I can't find any examples of their use
Rather than creating the MagicCommandOutput I probably want the Kernel to create one for me. I see that the KernelMagicCommand has an execute method that would do that. Anyone have any ideas?
Okay, I found one way to do it. Here's my solution:
You can ask the current kernelManager for the kernel you're interested in,
then call PythonEntryPoint.evaluate. It seems to do the job!
#Override
MagicCommandOutcomeItem execute(MagicCommandExecutionParam magicCommandExecutionParam) {
String input = magicCommandExecutionParam.command.substring(MAGIC.length() + 1)
// this is the Scala code I want to evaluate:
String codeToExecute = <your code here>
KernelFunctionality kernel = KernelManager.get()
PythonEntryPoint pep = kernel.getPythonEntryPoint(SCALA_KERNEL)
pep.evaluate(codeToExecute)
pep.getShellMsg()
List<Message> messages = new ArrayList<>()
//until there are messages on iopub channel available collect them into response
while (true) {
String iopubMsg = pep.getIopubMsg()
if (iopubMsg == "null") break
try {
Message msg = parseMessage(iopubMsg) //(I didn't show this part)
messages.add(msg)
String commId = (String) msg.getContent().get("comm_id")
if (commId != null) {
kernel.addCommIdManagerMapping(commId, SCALA_KERNEL)
}
} catch (IOException e) {
log.error("There was an error: ${e.getMessage()}")
return new MagicKernelResponse(MagicCommandOutcomeItem.Status.ERROR, messages)
}
}
return new MagicKernelResponse(MagicCommandOutcomeItem.Status.OK, messages)
}

Trying to work with down() method from ExtJS 4.2.1

I am trying to find a specific element from my page using ExtJS 4 so I can do modifications on it.
I know its id so it should not be a problem BUT
-I tried Ext.getCmp('theId') and it just return me undefined
-I tried to use down('theId') method by passing through the view and I still get a nullresponse.
As I know the id of the element I tried again the two methods by setting manually the id and it didn't work neither.
Do these two methods not function?
How should I do?
Here is the concerned part of the code :
listeners: {
load: function(node, records, successful, eOpts) {
var ownertree = records.store.ownerTree;
var boundView = ownertree.dockedItems.items[1].view.id;
var generalId = boundView+'-record-';
// Add row stripping on leaf nodes when a node is expanded
//
// Adding the same feature to the whole tree instead of leaf nodes
// would not be much more complicated but it would require iterating
// the whole tree starting with the root node to build a list of
// all visible nodes. The same function would need to be called
// on expand, collapse, append, insert, remove and load events.
if (!node.tree.root.data.leaf) {
// Process each child node
node.tree.root.cascadeBy(function(currentChild) {
// Process only leaf
if (currentChild.data.leaf) {
var nodeId = ""+generalId+currentChild.internalId;
var index = currentChild.data.index;
if ((index % 2) == 0) {
// even node
currentChild.data.cls.replace('tree-odd-node', '')
currentChild.data.cls = 'tree-even-node';
} else {
// odd node
currentChild.data.cls.replace('tree-even-node', '')
currentChild.data.cls = 'tree-odd-node';
}
// Update CSS classes
currentChild.triggerUIUpdate();
console.log(nodeId);
console.log(ownertree.view.body);
console.log(Ext.getCmp(nodeId));
console.log(Ext.getCmp('treeview-1016-record-02001001'));
console.log(ownertree.view.body.down(nodeId));
console.log(ownertree.view.body.down('treeview-1016-record-02001001'));
}
});
}
}
You can see my console.log at the end.
Here is what they give me on the javascript console (in the right order):
treeview-1016-record-02001001
The precise id I am looking for. And I also try manually in case...
h {dom: table#treeview-1016-table.x-treeview-1016-table x-grid-table, el: h, id: "treeview-1016gridBody", $cache: Object, lastBox: Object…}
I checked every configs of this item and its dom and it is exactly the part of the dom I am looking for, which is the view containing my tree. The BIG parent
And then:
undefined
undefined
null
null
Here is the item I want to access:
<tr role="row" id="treeview-1016-record-02001001" ...>
And I checked there is no id duplication anywhere...
I asked someone else who told me these methods do not work. The problem is I need to access this item to modify its cls.
I would appreciate any idea.
You are looking for Ext.get(id). Ext.getCmp(id) is used for Ext.Components, and Ext.get(id) is used for Ext.dom.Elements. See the docs here: http://docs.sencha.com/extjs/4.2.1/#!/api/Ext-method-get
Ok so finally I used the afteritemexpand listener. With the ids I get the elements I am looking for with your Ext.get(id) method kevhender :).
The reason is that the dom elements where not completely loaded when I used my load listener (it was just the store) so the Ext.get(id) method couldn't get the the element correctly. I first used afterlayout listener, that was correct but too often called and the access to the id was not so easy.
So, here is how I did finally :
listeners: {
load: function(node, records, successful, eOpts) {
var ownertree = records.store.ownerTree;
var boundView = ownertree.dockedItems.items[1].view.id;
var generalId = boundView+'-record-';
if (!node.tree.root.data.leaf) {
// Process each child node
node.tree.root.cascadeBy(function(currentChild) {
// Process only leaf
if (currentChild.data.leaf) {
var nodeId = ""+generalId+currentChild.internalId;
var index = currentChild.data.index;
if ( (index % 2) == 0 && ids.indexOf(nodeId) == -1 ) {
ids[indiceIds] = nodeId;
indiceIds++;
}
console.log(ids);
}
});
}
},
afteritemexpand: function( node, index, item, eOpts ){
/* This commented section below could replace the load but the load is done during store loading while afteritemexpand is done after expanding an item.
So, load listener makes saving time AND makes loading time constant. That is not the case if we just consider the commented section below because
the more you expand nodes, the more items it will have to get and so loading time is more and more important
*/
// var domLeaf = Ext.get(item.id).next();
// for ( var int = 0; int < node.childNodes.length; int++) {
// if (node.childNodes[int].data.leaf && (int % 2) == 0) {
// if (ids.indexOf(domLeaf.id) == -1) {
// ids[indiceIds] = domLeaf.id;
// indiceIds++;
// }
// }
// domLeaf = domLeaf.next();
// }
for ( var int = 0; int < ids.length; int++) {
domLeaf = Ext.get(ids[int]);
if (domLeaf != null) {
for ( var int2 = 0; int2 < domLeaf.dom.children.length; int2++) {
if (domLeaf.dom.children[int2].className.search('tree-even-node') == -1){
domLeaf.dom.children[int2].className += ' tree-even-node';
}
}
}
}
},
With ids an Array of the ids I need to set the class.
Thank you for the method.

How can I add a way to track a collection of entities and flag them on/off (to signal if a process has been done to them)?

My first idea is extend the partial class 'Step' which is an entity. I could add a bool property and then set that true/false as I complete processing on them.. But then every time I regenerate the DBContext after making changes to the DB won't I have to manually keep doing this? OR is some other way.. The code below explains what I am trying to do
if ((si.StepID == 20) || (si.StepID == 23) || (si.StepID == 24))
{
//Step is a process. Each Step must be run in order
//get list of steps in tasks in RUN (1,2,3,4...) order
int stepToRun = 0;
var stepsInTask = InstanceExpert.GetAllStepsInTaskOrderedByRunOrder(ti.TaskInstanceID);
//Get which step is next to be processed
foreach (Step s in stepsInTask)
{
//get the stepToRun
//Need to extend STEP Entity property with a bool??
}
//Take that step and run the specific function
if (stepToRun == 20)
{
}
if (stepToRun == 23)
{
}
if (stepToRun == 24)
{
}

Callback functions that reference each other? Nested if statements? Need guidance

first time long time here.
I just started programming in javascript recently, I'm running into a question of design.
I have some working code that:
1. Waits for specific input from the serial port,
2. When input is found it moves to the next function.
3. The next function sends a command(s) over the serial port and then waits for input again.
Now I have 9 functions defined as stepone() steptwo() etc.... There has to be a better way to do this. Each function is the same except with different variables for input and output desired.
However, I do not want the program to skip steps. It needs to wait for the correct serial input before sending the next command.
I've tried using callback functions referencing each other, it just seems...wrong?
Also, it doesn't work. It doesn't wait for the right input before sending commands.
var waitforinput = function(input, regex, callback)
{
if (regex.search != -1)
callback();
};
var sendcommand = function(command,callback)
{
port.writeline(command);
if (callback)
callback();
};
var connect = function()
{
var int = setInterval(function()
{
waitforinput(input, "Please choose:", function()
{
sendcommand("1", function()
{
waitforinput(input, "You choosed", function()
{
sendcommand("saveenv 1");
});
});
});
},50);
};
I ended up using switch() with cases and keeping track of a variable called step:
step = 1;
switch(step)
{
case 1:
if (inputbuffer.search('Please choose') !== -1)
{
if (!waitdisplaystarted)
{
waitdisplaystarted = true;
waitint = setInterval(showwait,1000);
}
window.$("#instructions").hide();
window.$("#status").html("Step 1: Choosing boot option.");
SELF.sendserialcommand("1");
step = 2;
}
break;
case 2:
if (inputbuffer.search('You choosed 1') !== -1)
{
SELF.sendserialcommand('setenv bootargs "board=ALFA console=ttyATH0,115200 rootfstype=squashfs,jffs2 noinitrd"\r');
setTimeout(function(){SELF.sendserialcommand('saveenv\r');}, 50);
window.$("#status").html("Step 2: Transferring new kernel.");
setTimeout(function(){SELF.sendserialcommand('tftp 0x80600000 kernel.bin\r');}, 2000);
step = 3;
}
break;
case 3:
if (inputbuffer.search('Bytes transferred = ' + 878938) !== -1)
{
window.$("#status").html("Step 3: Erasing old kernel.");
SELF.sendserialcommand('erase 0x9f650000 +0x190000\r');
step = 'finished';
}
}