I start 7 Talend parallel Jobs. Sometimes ALL Jobs started without mistake! Sometimes 4 or 5 Jobs started successfully and for the rest became the mistakes:
"Error by calling Job"
or
"Unable to create new native thread"
Can anyone give me some tips? Thanks
Maybe this can help you. There is more than one possibilities.
Talend Expert or Talend by example
Related
I had developed a Job in Talend and built the job and automated to run the Windows Batch file from the below build
On the Execution of the Job Start Windows Batch file it will invoke the dimtableinsert job and then after it finishes it will invoke fact_dim_combine it is taking just minutes to run in the Talend Open Studio but when I invoke the batch file via the Task Scheduler it is taking hours for the process to finish
Time Taken
Manual -- 5 Minutes
Automation -- 4 hours (on invoking Windows batch file)
Can someone please tell me what is wrong with this Automation Process
The reason of the delay in the execution would be a latency issue. Talend might be installed in the same server where database instance is installed. And so whenever you execute the job in Talend, it will complete as expected. But the scheduler might be installed in the other server, when you call the job through scheduler, it would take some time to insert the data.
Make sure you scheduler and database instance is on the same server
Execute the job directly in the windows terminal and check if you have same issue
The easiest way to know what is taking so much time is to add some logs to your job.
First, add some tWarn at the start and finish of each of the subjobs (dimtableinsert and fact_dim_combine) to know which one is the longest.
Then add more logs before/after the components inside the jobs.
This way you should have a better idea of what is responsible for the slowdown (DB access, writing of some files, etc ...)
I am new to Spark.
In our project,
we have converted seven PLSql scripts into Scala-Spark.
The existing PLSql scripts are scheduled as jobs on Talend. Each
script is a scheduled on a separate job and these seven jobs run on a sequence as only after the first job completes successfully, the second job starts and same continues until the last job(seventh).
My team is exploring the possibilities to schedule the Scala-Spark programs as jobs in other ways. One of the suggestion was to convert/write the same job that is running on Talend into Scala. I have no idea if it is possible.
So, Could anyone let me know whether it is possible to do the same on Scala.
You can submit your spark job in Talend using tSystem or tSSH component. and get the response code (exit code) from the mentioned component. If the exit code=0 (Success) then you can submit next spark job. We did the same in our project.
I created a Datastage Parallel job with Parallel Shared Container. The job was working fine. I did not make any change to the job. Suddenly the job has started failing with below error since 1 day:
main_program: Failed to create collation sequence from IBM InfoSphere
DataStage Enterprise Edition 9.1.0.6791 . Failed to create collation
sequence from IBM InfoSphere DataStage Enterprise Edition 9.1.0.6791.
Has anyone has faced similar issue?
Please help and let me know for further clarification.
The above mentioned error came due to incorrect NLS in the present transformers in the Container. I changed the NLS to ProjectDefault(OFF) and the issue got solved.
Hi i am beginer in Talend Open Studio 5.3.1 version.
currently i am facing issue in project i.e. schedule a job to run every 10 seconds and it monitor the other job and display output as status of another job which means the job is running or idle state.
Currently i am using Talend Open Studio 5.3.1 version by using this version it is possible or not .
explain me how to schelude a job for 10 seconds and display output as status of another job.
can anyone suggest and help me to solve my problem.
We should think a bit out of the box here. I'd solve this by using Project level logging: https://help.talend.com/display/TalendOpenStudioforBigDataUserGuide520EN/2.6+Customizing+project+settings
You'll have the jobs status stored in a database table, you just have to check whether the last execution of the job is still running or not. (Self join the stats table)
Monitoring jobs is not supported in Talend Open Studio, but there is some workaround:
Use a master job that launch the job to be monitored using tRunJob component, and your master job will have an idea whats going on.
Use empty files to synchronize your jobs, an empty file with a tricky name created by monitored jobs and the master job check them to get other jobs states.
Much easier is to use Quartz.
I am being driven mad with some PeopleSoft jobs that I'm scheduling. Job Sets and PSJobs with Crystal will not initiate the Crystal processes. The process (or any other Cyrstal process I try) will run if scheduled independently but in any form of PSJob or Job Set they remain with teh first Crystal job status=queued.
I've spent a day googling, reading Metalink, etc. Anyone got any ideas?
Cheers
Karl
I have seen this happen when the job is run on a unix process scheduler and, as you probably know, crystal will only run on a windows process scheduler. The solution was to force the entire job to run on a windows process scheduler.