My task is , I have 7 logs generated ,
A_TempLoad_CSP_11.log: 0 Rows not loaded due to data errors.
B_TempLoad_CSP_11.log: 0 Rows not loaded due to data errors.
C_TempLoad_CSP_11.log: 0 Rows not loaded due to data errors.
D_TempLoad_CSP_11.log: 0 Rows not loaded due to data errors.
E_TempLoad_CSP_11.log: 0 Rows not loaded due to data errors.
F_TempLoad_CSP_11.log: 2 Rows not loaded due to data errors.
G_TempLoad_CSP_11.log: 4 Rows not loaded due to data error
Here for F and G the count of rows not loaded is greater than 0 , I want the job to abort if for any of these the number of rows not loaded is greater than 0 , and in the director , I want this to be mentioned that Job aborted F and G .
Eg: Job failed for F and G
You can Keep a reject link for the same where data would be collected. Now link that dataset to a transformer which will have stage variables doing a count. The moment it gets increased to 1 from 0 map it to a job abort stage. This might work.
Related
I have Student Attendance information Date-wise as if Attended on a particular day then the value is 0, if not then 1. I want to have a running sum that shows consecutively how many days a student has missed. So the running sum has to reset when it encounters 0. How I can achieve this in Tableau.
After so many trial and errors I finally got to this expression and it seems to be working fine after testing.
IF ATTR([Attended]) != 0
THEN IF FIRST() = 0 # This IF is basically for the first value to be displayed as 1 instead of 2 (1 + 1)
THEN 1
ELSE SUM([Attended])+PREVIOUS_VALUE(SUM([Attended]))
END
ELSE SUM([Attended])-SUM([Attended]) # Reset on 0
END
There might be a simpler logic, but this is working for me.
Hope it works for others too(make sure to delete the Comments in the expression if used in Tableau!!!)
I have a partitioned table and can read it using a get command as such:
get `:hdb/2018.01.01/trade
and will give me:
sym size exchange
-----------------
0 100 2
1 200 2
1 300 2
I like to modify the cell value like size from 200 and 300 to a 1000 given an index or list of rows. So I am using
.Q.ind[`:hdb/2018.01.01/trade; 1 2j]
to get the rows and then change the cell. But I am getting a `rank error when running .Q.ind[].
The error you're getting is that the first input param to .Q.ind is the mapped table name, not a symbol representing the table name/location
I'm not sure if .Q.ind is going to help you here though, it's more useful for data retrieval than data (re)write.
A couple of approaches you could take:
Pull in the whole date slice select from table where date=X, modify it in memory and then write it back down using `:hdb/2018.01.01/trade/ set delete date from modifiedTable. This is assuming you're not modifying any enumerated/symbol columns. You'd have to be careful to maintain same schema, maintain same compression etc
Use the dbmaint package to handle the changes: https://github.com/KxSystems/kdb/blob/master/utils/dbmaint.md
If you're careful enough you could pull in only the column itself, modify it and write it back down. p set #[get p:`:hdb/2018.01.01/trade/col1;1 2;:;1000]
You could also use an amend operation to update the values.
#[`:hdb/2018.01.01/trade;`size;#[;1 2;:;1000]
This will edit your table on disk.
q)get`:hdb/2018.01.01/trade
sym size exchange
-----------------
0 100 2
1 200 2
1 300 2
q)#[`:hdb/2018.01.01/trade;`size;#[;1 2;:;1000]]
`:hdb/2018.01.01/trade
q)get `:hdb/2018.01.01/trade/
sym size exchange
-----------------
0 100 2
1 1000 2
2 1000 2
My root job which has two steps,
Transformation Executor(to copy rows to results) & a Job Executor(Executing for each input row)
what I want is, that my sub-job should execute completely for first incoming row before it start execution for second row.
Click on the Job executor step and check the box Execute for every input row.
Tell me if it is not what you need.
Unless you specify a different value than 1 on Change Number Of Copies To Start (Right click on any Transformation Entry to see that option), that will always be the expected behavior.
If the number is greater than 1 then the Job Executor will have that number of copies running in parallel, distributing the input rows (for example, 100 input rows, with 10 copies, each copy will execute 10 rows no matter what).
I'm having some problems with grouping and running totals in crystal reports, I've found loads of info whilst googling but I'm looking a bit more info.
Currently I have 3 tables, jobs, times and costs, I have a running total to calculate the value of the time.hours and another for the costs.amount.
The issue I'm having is that the costs are being multiplied by the amount of time.hours records that exist.
so if a job has 3 time.hours records and there is only one record of costs.amount that
cost is being multiplied by that about of records
So what i end up with is this
Job Hours Costs
123 5 0
123 4 100
123 3 0
=========================
12 100
which is exactly right, now if i add another running total without resetting then my total ends up as
========================
12 300
I think this is happening because of the way i have them grouped but im not sure how to make them group to only job.ID
This was solved using ( ) to break the formula down
({timesheet.Time_Date} <={?timDateUJC} and {jobsheet1.Job_JobCompleted} = 0)
or
({materialcost.Inv_Date} <= {?supDateUJC} and {jobsheet1.Job_JobCompleted} = 0)
I have a control table in my PostgreSQL Database. Some sample data is shown below:
id section_code ddu_area ddu_action status_flag
1 4636 1 1 0
2 4636 1 2 0
3 4636 2 1 0
4 4636 2 2 0
I have few staging tables of each of the ddu_area's. There are a total 5 distinct ddu_areas. I want to pass the one id at a time and execute my ETL and id to be passed should have the status_flag as 0. When the status_flag is 1, it should take that particular id and pass it to the ETL job and take the data from staging to target.
I don't know whether we have this functionality in Talend or not. I am new to Talend. In SSIS, we can do this using For Each Loop container. Can anyone help me around with this?
You have to use tFlowToIterate component to iterate over each record after that use tRunJob component to pass the parameters using various way you can see some of them here
your job flow should be like this.
tFlowToIterate--if----tRunJob