I integrate spring batch into a restful controller of a spring boot, which means now we operate spring batch program by send a restful call. In this case, we cannot make a jar and register the jar on spring data flow server. So my question is that how to register a task if we don't have jar
You've asked a few similar questions today.
My recommendation is that you could consider referring to the ref. guide of Spring Cloud Task and Spring Cloud Data Flow. Specifically, pay attention to the Spring Batch section.
Once you have the understanding as to what to do, you can build a batch-job as a Spring Cloud Task application, and run it standalone successfully.
If it runs locally as expected, you can switch to SCDF and register the JAR using the REST-API, Shell, or in the GUI. You'd need a physical uber-jar of the application for it. With that registered, you can then build a Task definition with it, and launch it from SCDF.
If you want to do all of the above programmatically, please have a look at the acceptance-test suite for examples.
Related
I'm working on creating a kogito bpm spring boot project. I'm very happy to see reduced level of complexity on integration on jbpm in spring boot with the help of KOGITO. I'm struggling to find answers for my question, so posting them here,
Kogito is a open source cloud offering for jbpm. I'm I correct?
I see only mongodb or infinispan can only be used or supported with Kogito. I can't integrate psql with kogito. I'm I correct?
I successfully created the spring boot kogito mongodb project and when I placed a .bpmn file in the resource folder, automatically endpoints got created. I was able to access, run the process and get response. But I don't see any entries created in MONGODB :| I don't even see the table being created. The .bpmn contains a simple hello world flow with start+scripttask+end nodes. Please explain help me understand this. Is the RuntimeMangar configured for per request stratergy? How can I change it?
Answers inline.
Kogito is a open source cloud offering for jbpm. I'm I correct?
Kogito is open-source and has jBPM integrated into its codebase to run on a cloud-native environment. In addition, a lot has been made to make it also run with native compilation if used with Quarkus.
I see only mongodb or infinispan can only be used or supported with Kogito. I can't integrate psql with kogito. I'm I correct?
To this date, Kogito has the following add-ons to support persistence:
Infinispan
Postgres
MongoDB
JDBC (so you can extend to support any database you wish)
See more about it here https://docs.jboss.org/kogito/release/latest/html_single/#con-persistence_kogito-developing-process-services.
But I don't see any entries created in MONGODB
Do you mind sharing a reproducer? Have you taken a look at the examples in https://github.com/kiegroup/kogito-examples/tree/stable/process-mongodb-persistence-springboot? This example shows a call to a sub-process that relies on a user task. Hence the process must be persisted to fire up again on a new request to solve the task. However, since your process starts and ends in one request, there's nothing to be persisted in the DB:
Runtime persistence is intended primarily for storing data that is required to resume workflow execution for a particular process instance. Persistence applies to both public and private processes that are not yet complete. Once a process completes, persistence is no longer applied. This persistence behavior means that only the information that is required to resume execution is persisted.
I want to execute my apps using spring-complex-task and I have already build complex spring-batch Flow Jobs which executes perfectly fine.
could you please explain what is difference between spring Batch flow job Vs spring composed task? and which is best among them?
A composed task within Spring Cloud Data Flow is actually built on Spring Batch in that the transition from task to task is managed by a dynamically generated Spring Batch job. This model allows the decomposition of a batch job into reusable parts that can be independently tested, deployed, and orchestrated at a level higher than a job. This allows for things like writing a single step job that is reusable across multiple workflows.
They are really complimentary. You can use a composed task within Spring Cloud Data Flow to orchestrate both Spring Cloud Tasks as well as Spring Batch jobs (run as tasks). It really depends on how you want to slice up your process. If you have processes that are tightly coupled, package them as a single job. From there, you can orchestrate them with Spring Cloud Data Flow's composed task functionality.
In general, there's not one that's "better". It's going to be dependent on your use case and requirements.
Spring Batch is a nice framework to run batch processing applications.
Spring Cloud Task is a wrapper that allows you to run short lived microservices using Spring Cloud along with Spring Boot. Once you setup a test with #EnableTask it will then launch your *Runner. The framework also comes with Spring Batch integration points and ComposedTaskRunner helps facilitate that integration.
I'd start with the Spring Cloud Task batch documentation and then come back to ask more specific questions.
Spring Batch Admin Manager implementation is not updated since January 2015.
Is there something else that have to be used instead?
Or should I still use Spring Batch Admin Manager to provide some generic UI for Spring Batch application?
Assuming you mean the Spring Batch Admin v1.3.1 release then:
Is there something else? Yes, if your spring batch application is a spring boot variant have a look at https://github.com/codecentric/spring-boot-admin and how to configure it checkout http://codecentric.github.io/spring-boot-admin/1.4.1/#getting-started and https://blog.codecentric.de/en/2014/09/spring-boot-admin-first-official-release/
The Spring Batch Admin Git Repository claim
NOTE: This project is being moved to the Spring Attic and is not recommended for new projects. Spring Cloud Data Flow is the recommended replacement for managing and monitoring Spring Batch jobs going forward. You can read more about migrating to Spring Cloud Data Flow here.
Spring XD provides platform for launching for the batch jobs. Does that cover comprehensive workflows for all batch job use-cases? Or it is meant to be used within the context of Spring XD use-cases.
For example someone who wants to use just spring-batch not necessarily all the features of data ingestion/real-time analytics, will they still be benefited by setting up the Spring XD DIRT just for executing batch workflows? In that case, are there any limitations not being able to use all batch workflows supported by spring-batch?
In short, yes it can be used as a comprehensive batch platform. Spring XD provides a number of compelling features currently with more coming in the future. Features Spring XD provide for batch solutions:
Job orchestration - Spring Batch explicitly avoids the problem of job orchestration so that the developer can use whatever tool they want. Spring XD brings orchestration in a distributed environment via scheduling of jobs, executing ad hoc jobs, and executing jobs on the result of some form of logic (polling a directory for a file for example).
Abstraction of Spring Batch and Spring Integration - Spring Batch and Spring Integration are commonly used in solutions to address more complex scenarios. For example, if you need to FTP a file to a server, then kick off a batch job once it's there, you'd use Spring Integration for the FTP piece and to kick off the job with Spring Batch handling the processing of the job. Spring XD provides an elegent abstraction of those components to allow for easy assembly of these into more robust solutions.
Simplification of remote partitioning - Spring XD provides facilities to simplify the wiring of the communication aspects of remote partitioning within Spring Batch.
Interaction of jobs via UI, shell, or REST - Spring XD exposes a number of metrics and functionality to be consumable via their web based UI, the interactive shell, or REST based end points.
The main limit as of Spring XD 1.0 for batch processing is the inability to execute nested jobs (using a JobStep). I believe this will be part of Spring XD 1.1 (https://jira.spring.io/browse/XD-1972).
Looking forward, other features that I would expect in future versions of Spring XD are around high availability for jobs. Currently if a job is deployed on a node and the node goes down, it will be redeployed automatically. In future releases, the ability to restart the job automatically upon redeployment would be possible.
I developed one spring batch application which is deployed as executable jar using batch/shell script. It works fine.
Now recently I read about spring batch admin application release. As per their doc, they say you have to point to job-context.xml and that will allow to manage spring batch app to be started,restarted and stopped from admin app. Now my question is do I have to keep my job-context.xml outside the jar or what are the exact steps, i am confused about this configuration.
Any insight on this is very useful and by the way I am using spring batch 2.1.
Thanks
The Spring Batch admin application is a good reference implementation and is highly customizable. All interface implementations may be replaced via Spring DI using your own classes. UI is also template driven(FreeMarker I think) and therefore may be customized to display relevant information, change skin etc.
I had a similar need like yours - need admin functionality included in an app built as jar. I did not quite like the fact that I had to package my jobs as a .war file. Instead I extracted relevant configurations from Spring Batch Admin source and created a deployment that works off file system and runs on embedded Jetty server.
See screen shots here : https://github.com/regunathb/Trooper/wiki/Trooper-Batch-Web-Console
Source, configurations etc are available here : https://github.com/regunathb/Trooper/tree/master/batch-core . This project actually creates a .jar and not .war
If your application has custom classes and is deployed as a runnable jar and not contained within the spring batch admin, you cannot start jobs. You can only view the status of jobs and "kill" their status in the database.
If you look at http://static.springsource.org/spring-batch-admin/reference/reference.xhtml at the end of the Configuration Upload section it states
You can see a new entry in the job registry ("test-job") which is
launchable in-process because the application has a reference to the
Job. (Jobs which are not launchable were executed out of process, but
used the same database for its JobRepository, so they show up with
their executions in the UI.)
If your jobs are strictly configurable jobs, as-in you use only XML to define them and do not need to do any customized item readers/processors/writers or other custom classes, then you can upload the job XML and it will be runnable from within the admin site. If you have custom classes then, from my experience, you will have to have the spring batch application deployed within your web application and then upload an XML that contains the jobs you want to run separately.
I personally just used the Admin tool to view job status and provide me with statistics through some custom pages. I left the scheduler to run the jobs and I didn't want those with access to the admin site to kick off a job when they knew nothing about it. Basically, used it to give the users a warm fuzzy without allowing them to muck it up. (leave it to a user to find an edge case you didn't account for)