I have a Spring Batch job setup using JavaConfig (entirely through java code) which I deploy on a module in Spring XD. Typically, when you launch a job, you should see it in Spring XD's admin-ui under the Executions tab. Not mine, however, and I have no clue why. I've spent hours scouring through the documentation and looking for an answer, but I can't find anything.
Am I missing something? Is there something I need to put in my Job to make this work? What could cause Spring XD to not display a job's execution under Executions?
If you need me to provide logs or something, let me know, although I am not seeing any error in the Spring XD console output.
EDIT: This is how the job is defined int the code:
return jobBuilderFactory.get("Job")
.incrementer(new RunIdIncrementer())
.start(setupStep)
.next(verifyStep)
...
.next(zipFilesStep)
.next(teardownStep)
.build();
Does this happen only at the Admin UI? Can you see the job executions when you run the XD shell command job execution list?
If you are sure the batch job gets executed but not getting listed then it appears a bug. Do you see any stacktrace at the admin log when the execution tab is clicked?
Related
We have some spring-batch jobs are triggered by autosys with shell scripts as short lived processes.
Right now there's no way to view what is going on in the spring-batch process so I was exploring ways to view the status & manage(stop) the jobs.
Spring Cloud Data Flow is one of the options that I was exploring - but it seems that may not work when jobs are scheduled with Autosys.
What are the other options that I can explore in this regard and what is the recommended approach to manage spring-batch jobs now?
To stop a job, you first need to get the ID of the job execution to stop. This can be done using the JobExplorer API that allows you to explore meta-data that Spring Batch is aware of in the job repository. Once you get the job execution ID, you can stop it by calling the JobOperator#stop method, please refer to the Stopping a job section of the reference documentation.
This is independent of any method you used to launch the job (either manually, or via a scheduler or a graphical tool) and allows you to gracefully stop a job and leave the repository in a consistent state (ready for a restart if needed).
I am new to spring-batch, got few questions:-
I have got a question about the restart. As per documentation, the restart feature is enabled by default. What I am not clear is do I need to do any extra code for a restart? If so, I am thinking of adding a scheduled job that looks at failed processes and restarts them?
I understand spring-batch-admin is deprecated. However, we cannot use spring-cloud-data-flow right now. Is there any other alternative to monitor and restart jobs on demand?
The restart that you mention only means if a job is restartable or not .It doesn't mean Spring Batch will help you to restart the failed job automatically.
Instead, it provides the following building blocks for developers for achieving this task on their own :
JobExplorer to find out the id of the job execution that you want to restart
JobOperator to restart a job execution given a job execution id
Also , a restartable job can only be restarted if its status is FAILED. So if you want to restart a running job that was stop running because of the server breakdown , you have to first find out this running job and update its job execution status and all of its task execution status to FAILED first in order to restart it. (See this for more information). One of the solution is to implement a SmartLifecycle which use the above building blocks to achieve this goal.
I had developed a Job in Talend and built the job and automated to run the Windows Batch file from the below build
On the Execution of the Job Start Windows Batch file it will invoke the dimtableinsert job and then after it finishes it will invoke fact_dim_combine it is taking just minutes to run in the Talend Open Studio but when I invoke the batch file via the Task Scheduler it is taking hours for the process to finish
Time Taken
Manual -- 5 Minutes
Automation -- 4 hours (on invoking Windows batch file)
Can someone please tell me what is wrong with this Automation Process
The reason of the delay in the execution would be a latency issue. Talend might be installed in the same server where database instance is installed. And so whenever you execute the job in Talend, it will complete as expected. But the scheduler might be installed in the other server, when you call the job through scheduler, it would take some time to insert the data.
Make sure you scheduler and database instance is on the same server
Execute the job directly in the windows terminal and check if you have same issue
The easiest way to know what is taking so much time is to add some logs to your job.
First, add some tWarn at the start and finish of each of the subjobs (dimtableinsert and fact_dim_combine) to know which one is the longest.
Then add more logs before/after the components inside the jobs.
This way you should have a better idea of what is responsible for the slowdown (DB access, writing of some files, etc ...)
We have a Spring Batch Application that is triggered by a Task Command Line Runner that is periodically triggered. We are looking for a UI to view the Job Execution status, can we use the Spring Cloud Data Flow UI dependency and get the UI view capability of these Job Executions?
You cannot just use the SCDF GUI outside on your own without SCDF — they are tightly coupled.
When Task/batch-job are launched from SCDF, the task/job executions are automatically tracked in the common datasource; likewise, the SCDF GUI will show task and batch-job details automatically, as well [see task executions / job executions].
Whether using a scheduler or manually launching the jobs, as far as the launch from both approaches goes through SCDF, everything should just work.
I have a Spring Batch job developed with Spring Boot (1.4.1.RELEASE).
It successfully runs from command line and writes job execution data to MySQL. It shows up as non-launchable job in Spring Batch Admin (2.0.0.M1, pointing to MySQL) and I can see job execution metrics.
Now I'd like to turn it into a launchable job so I can run it within Spring Batch Admin.
I wonder if anyone has done that before. The documentation has a section Add your Own Jobs For Launching. But it does not specify where to add the implementation jar(s) for the job?
Is it spring-batch-admin/WEB-INF/lib?
With Spring Boot, the non-launchable job is one big, all-in-one executable jar. Its dependencies overlap with Spring Batch Admin. For example, they both have spring-batch*.jar, spring*.jar but different versions.
Is there a way, like the job definition xml file, to keep them in separate contexts? Thank you.
Spring Batch Admin looks for your job definitions in src/main/resources/META-INF/spring/batch/jobs folder. You could add your job-definition.xml file in that folder and define your batch jobs in that xml.