I'm following this post to do the same as the OP is asking, store in a table of a database the failed records while reading/processing/writing. The only issue I have is with the onSkipInRead that the item is not there, so I'm not able to post the skipped item to the database. How can I get the onSkipInRead item?
Below is my Step configuration and SkipListener:
public Step step() {
return stepBuilderFactory.get("Step")
.<Input, Output> chunk(100)
.reader(reader()).faultTolerant().listener(recordSkipListener()).skipPolicy(fileVerificationSkipper())
.processor(processor())
.writer(writer())
.listener(stepExecutionListener())
.build();
}
public class RecordSkipListener implements SkipListener<Input, Output> {
#Override
public void onSkipInRead(Throwable t) { // <-- item not available here
log.info("Skipping record while reading");
}
#Override
public void onSkipInWrite(Output item, Throwable t) {
// TODO Auto-generated method stub
}
#Override
public void onSkipInProcess(Input item, Throwable t) {
log.info("Skipping record while process with Key: {}", item.getKey());
// Here since I have the record (item) I can update the table in the database with the item details
}
}
Related
I am new to RxJava and finding it very useful for network and database processing within my Android applications.
I have two use cases that I cannot implement completely in RxJava
Use Case 1
Clear down my target database table Table A
Fetch a list of database records from Table B that contain a key field
For each row retrieved from Table B, call a Remote API and persist all the returned data into Table A
The closest I have managed is this code
final AtomicInteger id = new AtomicInteger(0);
DatabaseController.deleteAll(TableA_DO.class);
DatabaseController.fetchTable_Bs()
.subscribeOn(Schedulers.io())
.toObservable()
.flatMapIterable(b -> b)
.flatMap(b_record -> NetworkController.getTable_A_data(b_record.getKey()))
.flatMap(network -> transformNetwork(id, network, NETWORK_B_MAPPER))
.doOnNext(DatabaseController::persistRealmObjects)
.doOnComplete(onComplete)
.doOnError(onError)
.doAfterTerminate(doAfterTerminate())
.doOnSubscribe(compositeDisposable::add)
.subscribe();
Use Case 2
Clear down my target database table Table X
Clear down my target database table Table Y
Fetch a list of database records from Table Z that contain a key field
For each row retrieved from Table B, call a Remote API and persist some of the returned data into Table X the remainder of the data should be persisted into table Y
I have not managed to create any code for use case 2.
I have a number of questions regarding the use of RxJava for these use cases.
Is it possible to achieve both my use cases in RxJava?
Is it "Best Practice" to combine all these steps into an Rx "Stream"
UPDATE
I ended up with this POC test code which seems to work...
I am not sure if its the optimum solution however My API calls return Single and my database operations return Completable so I feel like this is the best solution for me.
public class UseCaseOneA {
public static void main(final String[] args) {
login()
.andThen(UseCaseOneA.deleteDatabaseTableA())
.andThen(UseCaseOneA.deleteDatabaseTableB())
.andThen(manufactureRecords())
.flatMapIterable(x -> x)
.flatMapSingle(record -> NetworkController.callApi(record.getPrimaryKey()))
.flatMapSingle(z -> transform(z))
.flatMapCompletable(p -> UseCaseOneA.insertDatabaseTableA(p))
.doOnComplete(() -> System.out.println("ON COMPLETE"))
.doFinally(() -> System.out.println("ON FINALLY"))
.subscribe();
}
private static Single<List<PayloadDO>> transform(final List<RemotePayload> payloads) {
return Single.create(new SingleOnSubscribe<List<PayloadDO>>() {
#Override
public void subscribe(final SingleEmitter<List<PayloadDO>> emitter) throws Exception {
System.out.println("transform - " + payloads.size());
final List<PayloadDO> payloadDOs = new ArrayList<>();
for (final RemotePayload remotePayload : payloads) {
payloadDOs.add(new PayloadDO(remotePayload.getPayload()));
}
emitter.onSuccess(payloadDOs);
}
});
}
private static Observable<List<Record>> manufactureRecords() {
final List<Record> records = new ArrayList<>();
records.add(new Record("111-111-111"));
records.add(new Record("222-222-222"));
records.add(new Record("3333-3333-3333"));
records.add(new Record("44-444-44444-44-4"));
records.add(new Record("5555-55-55-5-55-5555-5555"));
return Observable.just(records);
}
private static Completable deleteDatabaseTableA() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
System.out.println("deleteDatabaseTableA");
emitter.onComplete();
}
});
}
private static Completable deleteDatabaseTableB() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
System.out.println("deleteDatabaseTableB");
emitter.onComplete();
}
});
}
private static Completable insertDatabaseTableA(final List<PayloadDO> payloadDOs) {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
System.out.println("insertDatabaseTableA - " + payloadDOs);
emitter.onComplete();
}
});
}
private static Completable login() {
return Completable.complete();
}
}
This code doesn't address all my use case requirements. Namely being able to transform the remote payload records into multiple Database record types and insert each type into its own specific target databased table.
I could just call the Remote API twice to get the same remote data items and transform first into one database type then secondly into the second database type, however that seems wasteful.
Is there an operand in RxJava where I can reuse the output from my API calls and transform them into another database type?
You have to index the items yourself in some manner, for example, via external counting:
Observable.defer(() -> {
AtomicInteger counter = new AtomicInteger();
return DatabaseController.fetchTable_Bs()
.subscribeOn(Schedulers.io())
.toObservable()
.flatMapIterable(b -> b)
.doOnNext(item -> {
if (counter.getAndIncrement() == 0) {
// this is the very first item
} else {
// these are the subsequent items
}
});
});
The defer is necessary to isolate the counter to the inner sequence so that repetition still works if necessary.
I am switching from async tasks to rxjava2 and have some issues with my code tests.
I have a room table of elements that have a certain monetary amount. On a usercontrol that is called DisplayCurrentBudget, a sum of all amounts should be displayed. This number must refresh everytime a new element is inserted. I tackled the requirement in two ways, but both produce the same result: My code does not care if the database is updated, it only updates when the fragment is recreated (onCreateView).
My first attempt was this:
//RxJava2 Test
Observable<ItemS> ItemObservable = Observable.create( emitter -> {
try {
List<ItemS> movies = oStandardModel.getItemsVanilla();
for (ItemS movie : movies) {
emitter.onNext(movie);
}
emitter.onComplete();
} catch (Exception e) {
emitter.onError(e);
}
});
DisposableObserver<ItemS> disposable = ItemObservable.
subscribeOn(Schedulers.io()).
observeOn(AndroidSchedulers.mainThread()).
subscribeWith(new DisposableObserver<ItemS>() {
public List<ItemS> BadFeelingAboutThis = new ArrayList<ItemS>();
#Override
public void onNext(ItemS movie) {
// Access your Movie object here
BadFeelingAboutThis.add(movie);
}
#Override
public void onError(Throwable e) {
// Show the user that an error has occurred
}
#Override
public void onComplete() {
// Show the user that the operation is complete
oBinding.DisplayCurrentBudget.setText(Manager.GetBigSum(BadFeelingAboutThis).toString());
}
});
I already was uncomfortable with that code. My second attempt produces the exact same result:
Observable<BigDecimal> ItemObservable2 = Observable.create( emitter -> {
try {
BigDecimal mySum = oStandardModel.getWholeBudget();
emitter.onNext(mySum);
emitter.onComplete();
} catch (Exception e) {
emitter.onError(e);
}
});
DisposableObserver<BigDecimal> disposable = ItemObservable2.
subscribeOn(Schedulers.io()).
observeOn(AndroidSchedulers.mainThread()).
subscribeWith(new DisposableObserver<BigDecimal>() {
#Override
public void onNext(BigDecimal sum) {
// Access your Movie object here
oBinding.DisplayCurrentBudget.setText(sum.toString());
}
#Override
public void onError(Throwable e) {
// Show the user that an error has occurred
}
#Override
public void onComplete() {
// Show the user that the operation is complete
}
});
Any obvious issues with my code?
Thanks for reading, much appreciate it!
Edit:
I was asked what Manager.GetBigSum does, it actually does not do much. It only adds BigDecimal-Values of an Item list.
public static BigDecimal GetBigSum(List<ItemS> ListP){
List<BigDecimal> bigDList = ListP.stream().map(ItemS::get_dAmount).collect(Collectors.toList());
return bigDList.stream()
.reduce(BigDecimal.ZERO, BigDecimal::add);
}
Further, I simplified the query. But it still does not care about DB updates, only about fragment recreation:
Single.fromCallable(() -> oStandardModel.getItemsVanilla())
.map(Manager::GetBigSum)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
e -> oBinding.DisplayCurrentBudget.setText(e.toString())
);
Your rx logic has no error. That should be internal error in your getWholeBudget.
But why you write rx so complex?
For your case, you can just write:
Single.fromCallable(() -> oStandardModel.getItemsVanilla())
.map(Manager::GetBigSum)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
e -> oBinding.DisplayCurrentBudget.setText(sum.toString()),
e -> log.error(e));
I solved it this way:
oStandardModel.getItemJointCatLive().observe(this, new Observer<List<ItemJointCat>>() {
#Override
public void onChanged(#Nullable final List<ItemJointCat> oItemSP) {
Single.fromCallable(() -> oStandardModel.getWholeBudget())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
e -> oBinding.DisplayCurrentBudget.setText(e.toString())
);
}
});
My mistake was that I assumed RXjava2 does not need an onchanged event...now i just use onchanged event of livedata observer to trigger a simple rxjava2 query.
Do you think there is anything wrong with that approach?
I have a REST call returning a collection (the original), this collection is filtered but on the subscribe onSuccess I what to obtain both the original list and the filtered one.
I don't know how to 'pass' this second element, which operator should I use to obtain this result?
I show a simplified version of my code below
Observable.fromCallable(new Callable<List<Integer>>() {
#Override public List<Integer> call() throws Exception {
// dynamic list obtained from REST call
// for simplicity here I return a list
return Arrays.asList(1, 2, 3, 4);
}
})
.flatMap(new Function<List<Integer>, ObservableSource<Integer>>() {
#Override public ObservableSource<Integer> apply(List<Integer> integers) throws Exception {
return Observable.fromIterable(integers);
}
})
.filter(new Predicate<Integer>() {
#Override public boolean test(Integer integer) throws Exception {
return integer > 2;
}
})
.toList()
.subscribe(new SingleObserver<List<Integer>>() {
#Override public void onSubscribe(Disposable d) {}
#Override public void onSuccess(List<Integer> value) {
///////////////////
// here I want both original and filtered list
///////////////////
}
#Override public void onError(Throwable e) {}
});
One way is with ConnectableObservable. You need to share the emissions of your initial stream. Something like this
ConnectableObservable<List<Integer>> connectableObservable
= Observable.fromCallable(() -> {
// dynamic list obtained from REST call
// for simplicity here I return a list
return Arrays.asList(1, 2, 3, 4);
}).publish();
Single.zip(connectableObservable.flatMapIterable(integers -> integers)
.filter(integer -> integer > 2)
.toList(),
connectableObservable.elementAtOrError(0),
(integers, lists) -> combine(integers, lists))
.subscribe(o -> {
///////////////////
// here you ll have a new object containing both the initial list and the filtered list
///////////////////
});
connectableObservable.connect();
I am using activities and places to develop my application. When I click on the link on the left, my page is loaded, and I put the values in the fields. After I put the values, I send a RPC to the server, and get response back. This is also shown. Now the problem is that even if I click on the link on the left, I am getting the result page again. I am not getting the input page.
Impl code
#UiHandler("entInvoiceCompare")
void onClickSubmit(ClickEvent e) {
GWT.log("Going to enterprise compare place");
//listener.goTo(new EnterpriseInvoiceCompareViewPlace());
NewEBRM.getClientFactory().getPlaceController().goTo(new EnterpriseInvoiceCompareViewPlace());
}
Place
public class EnterpriseInvoiceCompareViewPlace extends Place{
public EnterpriseInvoiceCompareViewPlace() {
GWT.log("EnterpriseInvoiceCompareViewPlace: constructor");
}
public static class Tokenizer implements PlaceTokenizer<EnterpriseInvoiceCompareViewPlace> {
#Override
public String getToken(EnterpriseInvoiceCompareViewPlace place {
GWT.log("EnterpriseInvoiceCompareViewPlace: getToken: call");
return "EntInvoiceCompare";
}
#Override
public EnterpriseInvoiceCompareViewPlace getPlace(String token) {
GWT.log("EnterpriseInvoiceCompareViewPlace: getPlace: call");
return new EnterpriseInvoiceCompareViewPlace();
}
}
}
Activity
#Override
public void start(AcceptsOneWidget containerWidget, EventBus eventBus) {
// TODO Auto-generated method stub
GWT.log("EnterpriseInvoiceCompareActivity: start: starting activity");
EnterpriseInvoiceCompareView entInvoiceCompareView = clientFactory.getEnterpriseInvoiceCompareView();
entInvoiceCompareView.setPresenter(this);
containerWidget.setWidget(entInvoiceCompareView.asWidget());
GWT.log("EnterpriseInvoiceCompareActivity: start: ending activity");
}
i need to update a textView from my asynctask. I have an custom adapter for the listview and there i want to have a countdown for each entry. I will start the asynctask for each entry from my Adapter. How can i update the textview each second from the asynctask?
Thanks for help :)
If you post your code, I can give you a better answer. However, a common way to update views periodically is by using Handlers.
private final Handler mHandler = new Handler(); //intialize in main thread
public void test() {
mHandler.postDelayed(new Runnable() {
#Override
public void run() {
mTextView.setText("hello");
}
}, 1000);
}
You can do something like this (this will add an entry to a list view every one second). I have used the normal ArrayAdapter to add a string. You can use your custom adapter to do something similar. The publishProgress() method basically triggers the onProgressUpdate() method which hooks to the UI thread and displays the elements getting added.:
class AddStringTask extends AsyncTask {
#Override
protected Void doInBackground(Void... params) {
for(String item : items) {
publishProgress(item);
SystemClock.sleep(1000);
}
return null;
}
#Override
protected void onProgressUpdate(String... item) {
adapter.add(item[0]);
}
#Override
protected void onPostExecute(Void unused) {
Toast.makeText(getActivity(), "Done adding string item", Toast.LENGTH_SHORT).show();
}
}