I'm struggling in finding the right way to achive this result: I would like to have the first value of a column related to a "group by" column.
Suppose that original table has 3 columns: a, b, c. I want to aggregate by the column b and have the first value of c.
I have tried to use the function below, but it doesn't work as I expect:
Aggr(nodistinct[c],[b])
Thanks for any help.
To achieve that you need to use FirstSortedValue in table:
FirstSortedValue(c,c)
Here is a link to example qliksense app (I put your data and saved it on my OneDrive):
https://1drv.ms/u/s!Am4ZTQ8qcLwRtACt1wAoUTdFZ6lt
If you need to implement it in your loading script you can use the same function.
Related
I would like to filter out all values that does not end with ":Q_TT".
I have tried with the following activities
My bronze data has a column named "pnt_name". One of the rows in the table ends with ":Q_TT" so I would expect that the Exists activity would pass that row through.
Custom expression in Exists1
endsWith(':Q_TT', pnt_name)
In the future I would like the SourceData dataset to hold the filter values.
Thanks very much
You should be using filter activity instead of exists activity for your case.
The pipeline(where you can see the 2 test cases of A and B:Q_TT:
Here is the preview of the filter activity, you should use an expression of endsWith(pnt_name, ":Q_TT") too. You can see A is removed and B:Q_TT is kept.
I thought this would be fairly straight forward but i can't really find a simple way of doing it. I want to add a unique rownumber to a source dataset in a ADF Mapping Dataflow. In SSIS i would have done this with a script component but there's no option for that as far as i can see in ADF. I've looked for suitable functions in the derived columns expressions editor and also the aggregate component but there doesn't appear to be one.
Any ideas how this could be achieved?
Thanks
Many options:
Add a surrogate key transform
Hash row columns in Derived Column using SHA2
Use the rowNumber() function in a Window transformation
Give those a shot and let us know what you think
I did like this:
Add a Column with the same value to all the rows (I've used an integer with value = 1);
Added a window, using the column create previously on step 1 (Over);
Add a column on step 4 to window (window columns) with any name and rowNumber() as expression;
I have a job where I am getting a flow into tOracleOutput where I am updating the table. Now, I have to update that table using an SQL statement, which I guess we have option in Advanced settings of tOracleOuptut, but I don't know how to use it or you can say that I am not getting the settings properly. I referred to official documentation but could not understand. Can any one explain the fields like Name, SQL expression, Position, Reference Column in a better way?
the SQL query which I am using is:
update set COL1=SOMETHING1
where COL2=SOMETHING2
Now, value for COL1 is coming from the flow but COL2 is some column in the table which is not coming from the flow.
Have a look to tOracleRow for such a case.
Hope this helps.
TRF
Using tOracleOutput is helpful when a ready data source (table or file (...) with same columns as destination) the more elaborate your query is, the more you should do as TRF said (and use tOracleRow), but here's an example to your question:
file contain 3 column,
DB table of destination contains 4 column, where the 4th is the date of update, (the first 3 are identical to the input)
so you add the destination's column's name in Name and put the SQL function for the date (eg: SYSDATE) and where to put it (Position) in reference to a column of your choice (Reference Column)
In my view it helps avoid using tMap for a miserable additional column when you want to Insert, but you want to Update, in which case the component doesn't offer the additional column section, plus I don't think you can add the WHERE clause here
Hope it helps
I have some data which I need to pivot in Talend. This is a sample:
brandname,metric,value
A,xyz,2
B,xyz,2
A,abc,3
C,def,1
C,ghi,6
A,ghi,1
Now I need this data to be pivoted on the metric column like this:
brandname,abc,def,ghi,xyz
A,3,null,1,2
B,null,null,null,2
C,null,1,6,null
Currently I am using tPivotToColumnsDelimited to pivot the data to a file and reading back from that file. However having to store data on an external file and reading back is messy and unnecessary overhead.
Is there a way to do this with Talend without writing to an external file? I tried to use tDenormalize but as far as I understand, it will return the rows as 1 column which is not what I need. I also looked for some 3rd party component in TalendExchange but couldn't find anything useful.
Thank you for your help.
Assuming that your metrics are fixed, you can use their names as columns of the output. The solution to do the pivot has two parts: first, a tMap that transposes the value of each input-row in into the corresponding column in the output-row out and second, a tAggregate that groups the map's output-rows according to the brandname.
For the tMap you'd have to fill the columns conditionally like this, example for output colum named "abc":
out.abc = "abc".equals(in.metric)?in.value:null
In the tAggregate you'd have to group by out.brandname and aggregate each column as sum ignoring nulls.
In my frontend application I have a function that is called pick(VALUE,'col1','col2','col3'). If the VALUE is 2 the value in col2 should be picked.
This is very handsome for replacing long code using "case when", "switch case" or "if else" calculations.
I have tried to find a similar function in Postgres, but no luck so far. Seen some function array() and values() mentioned, but cannot find the correct syntax.
The goal is to set an return on of three column values depending on first column value.
Pseudo code (not working):
Select status values(column1,column2,column3)from code
I know I can do this by using "case-when-then-else-end", but I am looking for a shorter way to achieve the same thing.
Jsfiddle showing the principe. But I only want to pick ONE value depending on type:
http://sqlfiddle.com/#!15/e0b41/10
You can create an array of values from pr_* columns, then pick one of them in this way:
(array[prl_1,prl_2,prl_3])[code_type]
Here is a simple demo: http://sqlfiddle.com/#!15/e0b41/23
select *,
(array[prl_1,prl_2,prl_3])[code_type]
from code
left join prl on prl_id =1