I need to get files with data to run my own program for each file or bunch of files.
What is the best way to receive those files?
Do I need to set up my own server that expose API for file upload?
Should I use S3 and set up a server that fetch files from it automatically? or Maybe DropBox?
Or should I use SFTP server?
The amount of files are not big (max 1Giga per day)
I'm using Spring boot for my own server
By using SFTP, you can listen to changes (e.g. file upload) in a remote directory by connecting a listener. SFTP, in my opinion, is a great choice for file integration.
You said you will be using Spring Boot. Spring has a project called Spring Integration, which has SFTP support.
Spring Boot has a starter called spring-boot-starter-integration useful for bringing the best suited Spring Integration versions to your Spring Boot project. Spring Integration SFTP is really easy to use, since a lot of abstraction is provided by the framework.
Related
I integrate spring batch into a restful controller of a spring boot, which means now we operate spring batch program by send a restful call. In this case, we cannot make a jar and register the jar on spring data flow server. So my question is that how to register a task if we don't have jar
You've asked a few similar questions today.
My recommendation is that you could consider referring to the ref. guide of Spring Cloud Task and Spring Cloud Data Flow. Specifically, pay attention to the Spring Batch section.
Once you have the understanding as to what to do, you can build a batch-job as a Spring Cloud Task application, and run it standalone successfully.
If it runs locally as expected, you can switch to SCDF and register the JAR using the REST-API, Shell, or in the GUI. You'd need a physical uber-jar of the application for it. With that registered, you can then build a Task definition with it, and launch it from SCDF.
If you want to do all of the above programmatically, please have a look at the acceptance-test suite for examples.
Based on what parameters can we decide on a config server? aws cloud config vs spring cloud config server?
if the application is developed based on Spring framework, which one will be a better option? aws-cloud-config / spring-config?
does both support XML/properties configuration other than YAML?
Can both be configured for High Availability?
Is it possible to define a workflow (for approval of changes that need to be moved to PRODUCTION, using jenkins for example?)
If your application is based on Spring, it makes sense to use Spring Cloud Config as you would be using a specific framework instead of a Generic framework like AWS config
yes, Spring Cloud Config supports both YAML and property files
Ultimately the Spring Cloud Config is a Spring Boot App, hence you can make it high available using the same strategy that we use for any Spring Application
The simplest Approval work flow would be
Configure your Jenkins Job to send mail for approval
Set-up Promotion using https://wiki.jenkins.io/display/JENKINS/Promoted+Builds+Plugin
Deploy the Artificats to the specific environment
I am trying to keep External configuration in Spring Boot Project.
So I have achieved this using
a) Placing config at Git
b) Creating one config-server.
c) APplication get the config from config server.
But can I achieve this WITHOUT using External Config server?
I mean can't my application directly access configuration from GIT?
Assuming you're talking about Spring Cloud Config, it supports having the server and client in the same app, which is what you're trying to achieve. See this thread:
https://github.com/spring-cloud/spring-cloud-config/issues/100
Spring Batch Admin Manager implementation is not updated since January 2015.
Is there something else that have to be used instead?
Or should I still use Spring Batch Admin Manager to provide some generic UI for Spring Batch application?
Assuming you mean the Spring Batch Admin v1.3.1 release then:
Is there something else? Yes, if your spring batch application is a spring boot variant have a look at https://github.com/codecentric/spring-boot-admin and how to configure it checkout http://codecentric.github.io/spring-boot-admin/1.4.1/#getting-started and https://blog.codecentric.de/en/2014/09/spring-boot-admin-first-official-release/
The Spring Batch Admin Git Repository claim
NOTE: This project is being moved to the Spring Attic and is not recommended for new projects. Spring Cloud Data Flow is the recommended replacement for managing and monitoring Spring Batch jobs going forward. You can read more about migrating to Spring Cloud Data Flow here.
I am planning to use Bluemix for a ReST API development using Java. I wanted to use Application Security on Cloud for scanning the application to eliminate security concern.
Can I use it? Is there something more appropriate?
You can use the Static analysis feature of Application Security on Cloud to scan Java applications for security vulnerabilities. To accomplish this, a small utility needs to be downloaded to convert the application byte code files into an Intermediate Representation (IRX) of the code. This IRX file is uploaded to the server and scanned using trace analysis to find security vulnerabilities (the IRX file is encrypted to keep your data safe). IRX files can be generated using a small client command-line interface (CLI) that you need only download and extract to your local disk. In addition, you can run a small installer that adds static analysis plug-ins to Eclipse or Maven. Note that the Client Utility and cloud service versions must be compatible.
Take a look at Getting started with Application Security on Cloud for more information.