I've filtred some data with a tFilterRow, it is working fine, but at O rows the flow with 0 rows continue executing components :
I have 2 queries at the end of filter and reject flows and both are executed !
I don't want my query to be executed at 0 rows
any help ?
Thanks in advance.
Just for reference. tJava is a special component that always gets executed. It shouldn't be used with an incoming flow.
Here's a demonstration :
tJava and tJavaFlex have the same code. It gets printed in tJava but not in tJavaFlex.
You could use the Run If trigger for your components. And put the trigger condition as per the criteria when you connect your one component to another stating that -
execute the component only if the number of rows is greater than 0
Sample job layout -
You can also use global variable available in tFilterow like 1>((Integer)globalMap.get("tFilterRow_1_NB_LINE_OK")) and 2> ((Integer)globalMap.get("tFilterRow_1_NB_LINE_REJECT")). Create two link one with filter name and other with reject
In if link just use this variables with condition like
for Filer link: - ((Integer)globalMap.get("tFilterRow_1_NB_LINE_OK")) > 0
for Reject link : - ((Integer)globalMap.get("tFilterRow_1_NB_LINE_REJECT")) > 0
Related
I have several run-if conditions.I could have a catch all that ANDs and NOTs the several run-if conditions to run when none of them trigger.
Is there an alternative method to have a path chosen when none of the run-if conditions returned true?
There is no "else" link in Talend to do that automatically.
One solution would be to rewrite a condition which is the negative of all the others one combined.
Otherwise, if you can check the condition on your input flow (e.g : "XX"!=row1.myData), you can direct your flow on a tMap , and have multiple outputs on your tMap : each output could be filtered with one condition, and you can have a resulting output with "catch output reject" option. Only doable if the condition is on a data on the flow.
I currently have a file with a sql query on each row.
I'd like to read each row of this file with a tHDFSInput, and execute the query with a tHiveInput.
How can I do that ?
I have something like this :
But it just goes in the thiveinput only once.
You should consider using the component tFlowToIterate between the tHDFSInput and the subjob with your tHiveInput.
In the below example, I generate a flow which contains a sequence of number then for each number, I do a request on my database(I confess it is not a tHiveInput but I guess it is the same logic).
Here is the configuration of the request I use in my tDBRow_1:
Hope it will help you.
I am designing a ADF pipeline that copies rows from a SQL table to a folder in Azure Data Lake. After that the rows in SQL should be deleted. But for this delete action takes place I want to know if the number rows that are copied are the same as the number of rows that were I selected in the beginning of the pipeline.
Is there a way to get the rowcount fo the copy action and use this in another action (like a lookup)
Edit follow up question:
Bo Xiao's answer is OK. BUt then I have a follow up question. After the copy-activity I put an If Condition with the following expression:
#activity('LookUpActivity').output.firstRow.RecordsRead == #{activity('copyActivity').output.rowsCopied
But then I get the error: #activity('LookUpActivity').output.firstRow.RecordsRead == #{activity('copyActivity').output.rowsCopied
Isn't it possible to compare output parameters of two activities to see if this is True?
extra edit: I just found an error in this piece of code. I forgot a "{" at the begin of the code. But then the code is still wrong. To compare two outputs from earlier activities the code must be:
#equals(activity('LookUpActivity').output.firstRow.RecordsRead,activity('copyActivity').output.rowsCopied)
You can find copied rows in activity output as pictured below.
And you can use the output value like this:
#activity('copyActivity').output.rowsCopied
I have a talend job that have the following components:
- 1 database input component that fetched data from the database
- 1 output component that writes the fetched data into flat file.
Now, I have a scenario wherein, from two fields on the fetched data, only one should be put on in the output based on some if else logic.
Can anyone assist me in this matter? Is it using tMap?
Your if/else logic could be put in the tMap; as a ternary operator in the output of your tMap.
condition?resultifOK:resultifKO
For example, you want to insert in your file the value of ColumnA if it is "MY CONDITION", otherwise you insert value of ColumnB :
You'll have, directly in the output row of tMap:
"MY CONDITION".equals(row1.ColumnA)?row1.ColumnA:row1.ColumnB
You can't directly use a if/else in tMap. If you have several conditions , consider using a Routine instead of multiple ternary operators.
I want to execute some subjob if the previously processed number of rows are greater than N. To do this, i'm using the following configuration:
tFixedFlowInput have some rows.
tAggregateRow uses the count function and outputs one row with the number.
tSetGlobalVar then stores this value into a global var that I can check in the Run If connector (In this case, (Integer)globalMap.get("tSetGlobalVar_1") > 3 ).
tMsgBox then shows if the condition is true.
What I would like is to do the same, but in a more elegant way, using the minimum components required. I would like to connect tAggregateRow with the Run If connector directly (or even tFixedFlow) with tMsgBox, but I haven't found a way to refer to the number of rows previously processed without using the output row2.count variable.
How could I do something like this?
What should I put in the If condition to refer to the tAggregateRow operation result without connecting it to another meaningless component like exposed at the beginning?
for any talend component look under outline tab under the left side workspace pane at the bottom. this lists down the properties available via global variables for that component. Some properties like count of records inserted by output components are only available once the component is executed completely (After).
for your case you can try directly using ((Integer)globalMap.get("tFixedFlowInput_1_NB_LINE")) which gives number of lines (after) given by tFixedFlowInput.