Spring Batch Job Parameters in Item Reader - spring-batch

My parameter (workingDirectory) is always null and I do not understand why?
I am using the job launcher to kick off the job.
I see the job parameter in the database but it is not getting injected into my reader.
This is my reader.
#Component("customerReader")
#StepScope
public class CustomReader implements ItemReader<Customer> {
#Value("#{jobParameters['workingDirectory']}")
protected String workingDirectory;
private BufferedReader reader;
protected ObjectMapper objectMapper = new ObjectMapper();
private int fileIdx = 1;
#Override
public Customer read() throws IOException {
if(reader == null) {
reader = new BufferedReader(new FileReader(new File(workingDirectory + "output1.json")));
}
String line = reader.readLine();
if (line == null) {
try {
fileIdx++;
reader = new BufferedReader(
new FileReader(new File(workingDirectory + "output" + fileIdx + ".json")));
}
catch (FileNotFoundException ex) {
return null;
}
line = reader.readLine();
}
return objectMapper.readValue(line, Customer.class);
}
}
I am launching by calling:
JobParameters p = new JobParametersBuilder().addString("workingDirectory", workingDirectory).toJobParameters();
JobExecution e = jobLauncher.run(
recordGenerator.getJob(file, "/Users/abc/data/"), p);

Related

Spring Batch ExecutionContext deserialization for PostGreSQL JOB_EXECUTION_CONTEXT failing

I am trying to create a Spring Batch POC with Java Configuration and PostGreSQL.
I have successfully created beans that would have otherwise been provided via the in memory DB using #EnableBatchProcessing and #EnableAutoConfiguration.
I am not able to get the beans (JobExplorer) to return a JobExecution list given a JobInstance bean created from the same JobExplorer bean.
The error I am getting is "Unable to deserialize the execution context" which seems to be coming from the method trying to deserialize the "SHORT_CONTEXT" field of the JOB_EXECUTION_CONTEXT table.
I have passed the created JobExplorer bean DefaultExecutionContextSerializer. Later passed a DefaultLobHandler with "wrapAsLob" set to True when I was still getting the error.
#Bean
public JobRegistry jobRegistry() {
JobRegistry jr = new MapJobRegistry();
return jr;
}
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor() {
JobRegistryBeanPostProcessor jrbpp = new JobRegistryBeanPostProcessor();
jrbpp.setJobRegistry(jobRegistry());
return jrbpp;
}
#Bean
public JobOperator jobOperator() {
SimpleJobOperator sjo = new SimpleJobOperator();
sjo.setJobExplorer(jobExplorer());
sjo.setJobLauncher(jobLauncher());
sjo.setJobRegistry(jobRegistry());
sjo.setJobRepository(jobRepository());
return sjo;
}
#Bean
public JobExplorer jobExplorer() {
JobExplorerFactoryBean jefb = new JobExplorerFactoryBean();
jefb.setDataSource(dataSource());
jefb.setJdbcOperations(jdbcTemplate);
jefb.setTablePrefix("batch_");
jefb.setSerializer(new DefaultExecutionContextSerializer());
DefaultLobHandler lh = new DefaultLobHandler();
lh.setWrapAsLob(true);
jefb.setLobHandler(lh);
JobExplorer je = null;
try {
je = jefb.getObject();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return je;
}
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean
public JobRepository jobRepository() {
JobRepositoryFactoryBean jrfb = new JobRepositoryFactoryBean();
jrfb.setDataSource(dataSource());
jrfb.setDatabaseType("POSTGRES");
jrfb.setTransactionManager(new ResourcelessTransactionManager());
jrfb.setSerializer(new DefaultExecutionContextSerializer());
jrfb.setTablePrefix("batch_");
JobRepository jr = null;
try {
jr = (JobRepository)jrfb.getObject();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return jr;
}
Below is the get method in my rest controller where I am trying handle generate a list of failed Job executions
#Autowired
JobLauncher jobLauncher;
#Autowired
JobRegistry jobRegistry;
#Autowired
JobOperator jobOperator;
#Autowired
JobExplorer jobExplorer;
#SuppressWarnings("unchecked")
#GetMapping("batch/failedJobs")
public Map<String, List<JobExecution>> getFailedJobs() {
try {
if (jobRegistry == null || jobOperator == null || jobExplorer == null) {
System.out.println("job registry, operator or explorer is null");
} else {
Map<String, List<JobExecution>> allJobInstances = new HashMap<String, List<JobExecution>>();
// Get all jobs
jobRegistry.getJobNames().stream().forEach(jobName -> {
jobExplorer.getJobInstances(jobName, 1, 1000).forEach(l -> {
System.out.println("jobName: " + jobName + " instance: " + l);
});
jobExplorer.getJobInstances(jobName, 1, 1000).stream().forEach(jobInstance -> {
List<JobExecution> execultionList = jobExplorer.getJobExecutions(jobInstance); //Failing here
if (execultionList != null) {
System.out.println("" + execultionList);
execultionList.stream().forEach(l2 -> {
System.out.println("jobName: " + jobName + " instance: " + jobInstance
+ " jobExecution: " + l2);
});
if(allJobInstances.get(jobName) == null) {
allJobInstances.put(jobName, new ArrayList<JobExecution>());
}
allJobInstances.get(jobName).addAll((Collection<? extends JobExecution>) jobExplorer.getJobExecutions(jobInstance).stream().filter(e -> e.getStatus().equals(BatchStatus.FAILED)));
}else {
System.out.println("Could not get jobExecution for jobName " + jobName + " jobInstance: " + jobInstance);
}
});
});
return allJobInstances;
}
}catch (Exception e) {
System.out.println(e.getMessage());
logger.info(e.getMessage());
}
return null;
}
I fixed a similar issue by changing to the Jackson2 serializer:
jefb.setSerializer(new Jackson2ExecutionContextStringSerializer());
You may try it.

Curator ServiceCacheListener is triggered three times when a service is added

I am learning zookeeper and trying out the Curator framework for service discoveries. However, I am facing a weird issue that I have difficulties to figure out. The problem is when I tried to register an instance via serviceDiscovery, the cacheChanged event of the serviceCache gets triggered three times. When I removed an instance, it is only triggered once, which is the expected behavior. Please see the code below:
public class DiscoveryExample {
private static String PATH = "/base";
static ServiceDiscovery<InstanceDetails> serviceDiscovery = null;
public static void main(String[] args) throws Exception {
CuratorFramework client = null;
try {
// this is the ip address of my VM
client = CuratorFrameworkFactory.newClient("192.168.149.129:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
JsonInstanceSerializer<InstanceDetails> serializer = new JsonInstanceSerializer<InstanceDetails>(
InstanceDetails.class);
serviceDiscovery = ServiceDiscoveryBuilder.builder(InstanceDetails.class)
.client(client)
.basePath(PATH)
.serializer(serializer)
.build();
serviceDiscovery.start();
ServiceCache<InstanceDetails> serviceCache = serviceDiscovery.serviceCacheBuilder()
.name("product")
.build();
serviceCache.addListener(new ServiceCacheListener() {
#Override
public void stateChanged(CuratorFramework curator, ConnectionState state) {
// TODO Auto-generated method stub
System.out.println("State Changed to " + state.name());
}
// THIS IS THE PART GETS TRIGGERED MULTIPLE TIMES
#Override
public void cacheChanged() {
System.out.println("Cached Changed ");
List<ServiceInstance<InstanceDetails>> list = serviceCache.getInstances();
Iterator<ServiceInstance<InstanceDetails>> it = list.iterator();
while(it.hasNext()) {
System.out.println(it.next().getAddress());
}
}
});
serviceCache.start();
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.print("> ");
String line = in.readLine();
} finally {
CloseableUtils.closeQuietly(serviceDiscovery);
CloseableUtils.closeQuietly(client);
}
}
}
AND
public class RegisterApplicationServer {
final static String PATH = "/base";
static ServiceDiscovery<InstanceDetails> serviceDiscovery = null;
public static void main(String[] args) throws Exception {
CuratorFramework client = null;
try {
client = CuratorFrameworkFactory.newClient("192.168.149.129:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
JsonInstanceSerializer<InstanceDetails> serializer = new JsonInstanceSerializer<InstanceDetails>(
InstanceDetails.class);
serviceDiscovery = ServiceDiscoveryBuilder.builder(InstanceDetails.class).client(client).basePath(PATH)
.serializer(serializer).build();
serviceDiscovery.start();
// SOME OTHER CODE THAT TAKES CARES OF USER INPUT...
} finally {
CloseableUtils.closeQuietly(serviceDiscovery);
CloseableUtils.closeQuietly(client);
}
}
private static void addInstance(String[] args, CuratorFramework client, String command,
ServiceDiscovery<InstanceDetails> serviceDiscovery) throws Exception {
// simulate a new instance coming up
// in a real application, this would be a separate process
if (args.length < 2) {
System.err.println("syntax error (expected add <name> <description>): " + command);
return;
}
StringBuilder description = new StringBuilder();
for (int i = 1; i < args.length; ++i) {
if (i > 1) {
description.append(' ');
}
description.append(args[i]);
}
String serviceName = args[0];
ApplicationServer server = new ApplicationServer(client, PATH, serviceName, description.toString());
server.start();
serviceDiscovery.registerService(server.getThisInstance());
System.out.println(serviceName + " added");
}
private static void deleteInstance(String[] args, String command, ServiceDiscovery<InstanceDetails> serviceDiscovery) throws Exception {
// in a real application, this would occur due to normal operation, a
// crash, maintenance, etc.
if (args.length != 2) {
System.err.println("syntax error (expected delete <name>): " + command);
return;
}
final String serviceName = args[0];
Collection<ServiceInstance<InstanceDetails>> set = serviceDiscovery.queryForInstances(serviceName);
Iterator<ServiceInstance<InstanceDetails>> it = set.iterator();
while (it.hasNext()) {
ServiceInstance<InstanceDetails> si = it.next();
if (si.getPayload().getDescription().indexOf(args[1]) != -1) {
serviceDiscovery.unregisterService(si);
}
}
System.out.println("Removed an instance of: " + serviceName);
}
}
I appriciate if anyone can please point out where I am doing wrong and maybe can share some good materials/examples so I can refer to. The official website and the examples on github does not help a lot.

Javafx Task for Bluetooth data reciever

I am creating javafx application where I have this case that I need to listen for data sent over Bluetooth.
I have one fxml window on which I need to initialize Bluetooth and start listening from data.
Following is my Code for fxml controller:
//all imports
public class NewBarcodeInvoicePaneController implements Initializable{
private BluetoothController bc;
public BluetoothController getBc() {
return bc;
}
#Override
public void initialize(URL location, ResourceBundle resources) {
try {
bc = new BluetoothController();
new Thread(bc).start();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
And BluetoothController is task where I initialize bluettoth and listen to the data
public class BluetoothController extends Task<Void> {
#Override
protected Void call() throws Exception {
LocalDevice local = null;
StreamConnectionNotifier notifier;
StreamConnection connection = null;
// setup the server to listen for connection
try {
local = LocalDevice.getLocalDevice();
try {
local.setDiscoverable(DiscoveryAgent.GIAC);
} catch (BluetoothStateException e) {
}
UUID uuid = new UUID(80087355); // "04c6093b-0000-1000-8000-00805f9b34fb"
String url = "btspp://localhost:" + uuid.toString() + ";name=RemoteBluetooth";
notifier = (StreamConnectionNotifier) Connector.open(url);
} catch (Exception e) {
e.printStackTrace();
return null;
}
try {
System.err.println("THIS IS HAPENING");
connection = notifier.acceptAndOpen();
System.err.println("HAPENING???????????????????????????");
InputStream inputStream = connection.openInputStream();
BufferedReader bReader = new BufferedReader(new InputStreamReader(inputStream));
String lineRead = bReader.readLine();
connection.close();
inputStream.close();
notifier.close();
local.setDiscoverable(DiscoveryAgent.NOT_DISCOVERABLE);
JSONParser parser = new JSONParser();
Object obj = parser.parse(lineRead);
JSONArray array = (JSONArray) obj;
array.stream().map((o) -> (String) o).forEach((stringObj) -> {
System.out.println(stringObj);
});
System.out.println("AFTER DATA RECIEVED");
} catch (Exception e) {
e.printStackTrace();
return null;
}
return null;
}
}
It Works fine if I send data over bluetooth and blocking call to notifier.acceptAndOpen() is unblocked.
My problem is when we do not pass any data and I just want to close the window I opened..
It still have blocking call open with extra thread by the task.
I tried to cancel BluetoothController task in Main controller where I open this window like following
private void openNewBarcodeInvoicePane(ActionEvent ae) {
//following are custom classes to open windows from fxml and getting controller back for further manipulation
PostoryModalWindow modalWindow = new PostoryModalWindow();
modalWindow.openNewModalPaneWithParent("New Invoice", "fxml/newbarcodeinvoicepane.fxml", ae);
//getting controller object
NewBarcodeInvoicePaneController controller = (NewBarcodeInvoicePaneController) modalWindow.getDswFromController();
controller.getWindowStage().showAndWait();
BluetoothController bc = controller.getBc();
if(bc != null){
System.err.println("CANCELLING");
bc.cancel(true);
}
}
But it doesn't throw InterrupttedExeption (In which I might have Choice to close Bluetooth thread) and after research I found that waiting on Socket doesn't work on interrupt.
Any help on this?
Thanks
Got Solution After Some Research.
I just added new task to call notifier.acceptAndOpen();
And added method to close Bluetooth notifier.
public class BluetoothController extends Task<Void> {
private final ObservableList<Item> items = FXCollections.observableArrayList();
public ObservableList<Item> getItems() {
return items;
}
StreamConnectionNotifier notifier;
#Override
protected Void call() throws Exception {
try {
BluetoothConnectionTask bct = new BluetoothConnectionTask(items);
new Thread(bct).start();
Thread.sleep(2000);
notifier = bct.getNotifier();
} catch (Exception e) {
e.printStackTrace();
return null;
}
return null;
}
public void cancelandExit() {
try {
if (notifier != null) {
notifier.close();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Here is new task for blocking call
public class BluetoothConnectionTask extends Task<Void>{
private StreamConnectionNotifier notifier;
private StreamConnection connection;
private ObservableList<Item> items = FXCollections.observableArrayList();
public StreamConnection getConnection() {
return connection;
}
public StreamConnectionNotifier getNotifier() {
return notifier;
}
public BluetoothConnectionTask(ObservableList<Item> is){
items = is;
}
#Override
protected Void call() throws Exception {
try {
LocalDevice local = LocalDevice.getLocalDevice();
try {
local.setDiscoverable(DiscoveryAgent.GIAC);
} catch (BluetoothStateException e) {
}
UUID uuid = new UUID(80087355); // "04c6093b-0000-1000-8000-00805f9b34fb"
String url = "btspp://localhost:" + uuid.toString() + ";name=RemoteBluetooth";
notifier = (StreamConnectionNotifier) Connector.open(url);
} catch (Exception e) {
e.printStackTrace();
return null;
}
connection = notifier.acceptAndOpen();
InputStream inputStream = connection.openInputStream();
BufferedReader bReader = new BufferedReader(new InputStreamReader(inputStream));
String lineRead = bReader.readLine();
connection.close();
inputStream.close();
notifier.close();
LocalDevice local = LocalDevice.getLocalDevice();
local.setDiscoverable(DiscoveryAgent.NOT_DISCOVERABLE);
JSONParser parser = new JSONParser();
Object obj = parser.parse(lineRead);
JSONArray array = (JSONArray) obj;
ItemDAO idao = new ItemDAO();
array.stream().map((o) -> (String) o).forEach((stringObj) -> {
String barcode = (String) stringObj;
Item i = idao.getItemByBarCode(barcode);
System.err.println("Adding Item "+i.getName());
items.add(i);
});
System.out.println("AFTER DATA RECIEVED");
return null;
}
}
Now for cancelling closing my bluetooth thread I am calling cancelandExit() after window is closed.

CAS consumer not working as expected

I have a CAS consumer AE which is expected to iterates over CAS objects in a pipeline, serialize them and add the serialized CASs to an xml file.
public class DataWriter extends JCasConsumer_ImplBase {
private File outputDirectory;
public static final String PARAM_OUTPUT_DIRECTORY = "outputDir";
#ConfigurationParameter(name=PARAM_OUTPUT_DIRECTORY, defaultValue=".")
private String outputDir;
CasToInlineXml cas2xml;
public void initialize(UimaContext context) throws ResourceInitializationException {
super.initialize(context);
ConfigurationParameterInitializer.initialize(this, context);
outputDirectory = new File(outputDir);
if (!outputDirectory.exists()) {
outputDirectory.mkdirs();
}
}
#Override
public void process(JCas jCas) throws AnalysisEngineProcessException {
String file = fileCollectionReader.fileName;
File outFile = new File(outputDirectory, file + ".xmi");
FileOutputStream out = null;
try {
out = new FileOutputStream(outFile);
String xmlAnnotations = cas2xml.generateXML(jCas.getCas());
out.write(xmlAnnotations.getBytes("UTF-8"));
/* XmiCasSerializer ser = new XmiCasSerializer(jCas.getCas().getTypeSystem());
XMLSerializer xmlSer = new XMLSerializer(out, false);
ser.serialize(jCas.getCas(), xmlSer.getContentHandler());*/
if (out != null) {
out.close();
}
}
catch (IOException e) {
throw new AnalysisEngineProcessException(e);
}
catch (CASException e) {
throw new AnalysisEngineProcessException(e);
}
}
I am using it inside a pipeline after all my annotators, but it couldn't read CAS objects (I am getting NullPointerException at jCas.getCas()). It looks like I don't seem to understand the proper usage of CAS consumer. I appreciate any suggestions.

wicket download output stream

I want to download csv file , i take the response content and write to it , apprently wicket write after me and the content iam getting is the page html where it should be my csv
I have seen in the example the usage of throw new AbortException();
I am using version 6.7 , do you know if my version wicket has somthing instead of it ?
or rather I am doing somthing wrong ....
can you please help me ...
add(new Link<Void>("export") {
#Override
public void onClick() {
WebResponse response = (WebResponse) getResponse();
response.setAttachmentHeader("export.csv");
response.setContentType("text/csv");
OutputStream out = getResponse().getOutputStream();
try {
c.exportData(dataSource.getListForExport(), columns, out);
} catch (Exception ex) {
System.err.println(ex);
}
}
});
public <T> void exportData(List<T> list, List<IGridColumn<IDataSource<T>, T, String>> columns, OutputStream outputStream)
throws IOException {
long startTime = System.nanoTime();
PrintWriter out = new PrintWriter(new OutputStreamWriter(outputStream, Charset.forName(characterSet)));
try {
if (isExportHeadersEnabled()) {
boolean first = true;
for (IGridColumn<IDataSource<T>, T, String> col : columns) {
if (first) {
first = false;
} else {
out.print(delimiter);
System.out.println(delimiter);
}
if (col.getId().equals("checkBox")) {
continue;
}
out.print(quoteValue(col.getId()));
System.out.println(col.getId());
}
out.print("\r\n");
System.out.println("\r\n");
}
Iterator<? extends T> rowIterator = list.iterator();
while (rowIterator.hasNext()) {
T row = rowIterator.next();
boolean first = true;
for (IGridColumn<IDataSource<T>, T, String> col : columns) {
if (first) {
first = false;
} else {
out.print(delimiter);
System.out.println(delimiter);
}
if (col.getId().equals("checkBox")) {
continue;
}
Object o = (new PropertyModel<>(row, col.getId())).getObject();// ((AbstractColumn<T,
if (o != null) {
Class<?> c = o.getClass();
String s;
IConverter converter = Application.get().getConverterLocator().getConverter(c);
if (converter == null) {
s = o.toString();
} else {
s = converter.convertToString(o, Session.get().getLocale());
}
out.print(quoteValue(s));
System.out.println(quoteValue(s));
}
}
out.print("\r\n");
System.out.println("\r\n");
}
} catch (Exception ex) {
System.out.println(ex);
} finally {
System.out.println(out);
out.close();
// measure
System.out.print(System.nanoTime() - startTime);
}
}
The best way to do this is using dynamic resources. I'll suggest you to read chapter "Resource managment with Wicket" of this magnific free Wicket guide: https://code.google.com/p/wicket-guide/.
Here you have a similar example given in this guide in the section "Custom resources".
public class RSSProducerResource extends AbstractResource {
#Override
protected ResourceResponse newResourceResponse(Attributes attributes) {
ResourceResponse resourceResponse = new ResourceResponse();
resourceResponse.setContentType("text/xml");
resourceResponse.setTextEncoding("utf-8");
resourceResponse.setWriteCallback(new WriteCallback()
{
#Override
public void writeData(Attributes attributes) throws IOException
{
OutputStream outputStream = attributes.getResponse().getOutputStream();
Writer writer = new OutputStreamWriter(outputStream);
SyndFeedOutput output = new SyndFeedOutput();
try {
output.output(getFeed(), writer);
} catch (FeedException e) {
throw new WicketRuntimeException("Problems writing feed to response...");
}
}
});
return resourceResponse;
}
// method getFeed()...
}
And then you need to add the link in the desired page or component:
add(new ResourceLink("rssLink", new RSSProducerResource()));