Find RowNumber in Talend - talend

How to find row number in Talend Open Studio 6.3?I want to insert rows in a order like
Row 1 to Row 10 in File1
Row 11 to Row 20 in File2
Row 21 to Row30 to File3
And then again from Row 31 to file1
How to achive that?I have generated a sequence column.now how to proceed?is it can be done using tsamplerow?
Suppose the Sourcefile is like this:-
EMPNO,EMPNAME,DEPTNO
10,A,1
11,B,2
12,C,3
13,D,4
14,E,1
15,F,1
16,G,2
17,H,3
18,I,4
19,J,2
20,K,3
21,L,1
22,M,2

You can get the current row number by defining an increment value in talend job using Numeric.sequence("s1",1,1).
Note: I have used OP's sample data and divided rows per file as 3. But in OP's scenario it is 10 rows
Below is the sample job which I tried out,
I am generating a sequence number to know the current row number like below
After getting the current row value, I have an another variable which will be incremented for every 3 rows (in my example) like below (in OP's example, it is for every 10 rows.)
This is the expression I am doing on SequenceRow context variable.
context.SequenceRow = (input_row.SequenceNumber > context.RowRangePerFile && input_row.SequenceNumber % context.RowRangePerFile == 1) ? context.SequenceRow+1 : context.SequenceRow;
Finally I am filtering the rows in tMap_2 based on the SequenceRow value, like below,
For out1, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 1) || out2.SequenceRow == 1
For out3, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 2) || out2.SequenceRow == 2
For out4, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 0) || out2.SequenceRow == 3
I have taken your sample data that you have provided in your question,
EMPNO,EMPNAME,DEPTNO
10,A,1
11,B,2
12,C,3
13,D,4
14,E,1
15,F,1
16,G,2
17,H,3
18,I,4
19,J,2
20,K,3
21,L,1
22,M,2
and I am writing every 3 rows in each file and the output I got is
Hope this may help you.

Related

Extracting kdb list values based on some condition

Say we have a kdb list
L1:(1 2 3 4 5)
Apply condition
L1 < 3
And how can I retrieve result in another list (1 2)
You can use the where keyword for this:
q)l1 where l1<3
1 2
Applying l1<3 will return a list of booleans 11000b. Using where on this list will return the index of every 1b
q)where 11000b
0 1
Then indexing back into the original list will return the result in another list.

How to add values to last column of a table based on certain conditions in MATLAB?

I have a 29736 x 6 table, which is referred to as table_fault_test_data. It has 6 columns, with names wind_direction, wind_speed, air_temperature, air_pressure, density_hubheight and Fault_Condition respectively. What I want to do is to label the data in the Fault_Condition (last table column with either a 1 or a 0 value, depending on the values in the other columns.
I would like to do the following checks (For eg.)
If wind_direction value(column_1) is below 0.0040 and above 359.9940, label 6 th column entry corresponding to the respective row of the table as a 1, else label as 0.
Do this for the entire table. Similarly, do this check for others
like air_temperature, air_pressure and so on. I know that if-else
will be used for these checks. But, I am really confused as to how I
can do this for the whole table and add the corresponding value to
the 6 th column (Maybe using a loop or something).
Any help in this
regard would be highly appreciated. Many Thanks!
EDIT:
Further clarification: I have a 29736 x 6 table named table_fault_test_data . I want to add values to the 6 th column of table based on conditions as below:-
for i = 1:29736 % Iterating over the whole table row by row
if(1st column value <x | 1st column value > y)
% Add 0 to the Corresponding element of 6 th column i.e. table_fault_test_data(i,6)
elseif (2nd column value <x | 2nd column value > y)
% Add 0 to the Corresponding element of 6 th column i.e. table_fault_test_data(i,6)
elseif ... do this for other cases as well
else
% Add 1 to the Corresponding element of 6 th column i.e. table_fault_test_data(i,6)
This is the essence of my requirements. I hope this helps in understanding the question better.
You can use logical indexing, which is supported also for tables (for loops should be avoided, if possible). For example, suppose you want to implement the first condition, and also suppose your x and y are known; also, let us assume your table is called t
logicalIndecesFirstCondition = t{:,1} < x | t{:,2} >y
and then you could refer to the rows which verify this condition using logical indexing (please refer to logical indexing
E.g.:
t{logicalIndecesFirstCondition , 6} = t{logicalIndecesFirstCondition , 6} + 1.0;
This would add 1.0 to the 6th column, for the rows for which the logical condition is true

Tableau mixing aggregate and non-aggregate results error

I have a problem creating a calculated field in Tableau. I have data like so:
ID ... Status Step1 Step2 Step3
1 ... Accepted 1 1 1
2 ... Waiting 1 0 0
3 ... Discard 0 0 0
4 ... Waiting 1 1 0
...
I would like to create a calculated column that will give me the name of the last Step, but only when status is 'Accepted'. Otherwise I want the status. The syntax is quite easy, it looks like this:
IF [Status] = 'Accepted' THEN (
IF [Step3] = 1 THEN 'Step3' ELSEIF [STEP2] = 1 THEN 'Step2' ELSEIF [STEP1] = '1' THEN 'Step1' ELSE 'Step0')
ELSE [Status]
The problem is that the column 'Status' is a Dimension and the 'Step' statuses come from Measure. So they are AGG(Step1), AGG(Step2),...
I guess that is the reason I get this error:
Cannot mix aggregate and non-aggregate comparisons or results in 'IF' expressions.
I am not very familiar with Tableau. Any idea how I can solve this?
Solution:
Just use function ATTR that will make the non-aggregate function (Status) into an aggregate one. Then it is possible to combine them and the calculation is working.
IF ATTR([Status]) = 'Accepted' THEN (
IF [Step3] = 1 THEN 'Step3' ELSEIF [STEP2] = 1 THEN 'Step2' ELSEIF [STEP1] = '1' THEN 'Step1' ELSE 'Step0')
ELSE ATTR([Status])
Tableau automatically interprets numeric values as measures. It appears though that in your case they are a boolean (0 for false, 1 for true) and really ought to be dimensions.
Convert Step 1, Step 2, and Step 3 to dimensions. Highlight the fields, right click, and choose Convert to Dimension.

Reference to non-existent field 'd'

My mat file contains 40,000 rows and two columns. I have to read it line by line
and then get values of last column in a single row.
Following is my code:
for v = 1:40000
firstRowB = data.d(v,:)
if(firstRowB(1,2)==1)
count1=count1+1;
end
if(firstRowB(1,2)==2)
count2=count2+1;
end
end
FirstRowB gets the row checks whether last column equals 1 or 2 and then increases the value of respective count by 1.
But I keep getting this error:
Reference to non-existent field 'd'.
You could use vectorization (it is always convenient especially in Matlab). Taking advantage of the fact that true is one and false is zero, if you just want to count you can do :
count1 = sum ( data.d(:, 2) == 1 ) ;
count2 = sum (data.d(:,2) == 2 ) ;
in fact in general you could define :
getNumberOfElementsInLastColEqualTo = #(numb) sum (data.d(:,end) == numb ) ;
counts =arrayfun( getNumberOfElementsInLastColEqualTo , [1 2 ] );
Hope this helps.

Filter multiple columns using sqlite.swift?

let part = self.parts.filter(self.deleteStatus == 0).order(self.id.asc)
Above is the query I run to get all the records from local database. Now I want to add more filters like id==5 or name='product' in the filter. But the filter in sqlite.swift doesn't allow more than one expression in filter. How can I add more or filters in the filter method?
Filters can be chained with && (SQLite AND) and || (SQLite OR).
self.filter(deleteStatus == 0 && id == 5 && name == "product")
https://github.com/stephencelis/SQLite.swift/blob/master/Documentation/Index.md#filter-operators-and-functions
The filter in Sqlite.swift does allow more than one expression; you just chain them together like this:
users.filter(id == 1 %% location == "Office" %% supervisor == "John")