Lets suppose that I have class Event with and 10 subclasses (SubEvent1, SubEvent2... and so on) of class Event. I have configured for spring batch ItemReader, ItemProcessor and ItemWriter.
My item processor looks like:
ItemProcessor<Event, Outputclass> {
OutputClass process(Event e) {
if(e instancof SubEvent1) {
return processSubEvent1(e);
} else if(e instanceof SubEvent2) {
return processSubEvent2(e);
} else ...
}
is it possible to avoid those instanceof and process it by class specific Processors?
You should be able to do that using a combination of:
ClassifierCompositeItemProcessor: to process each item with its corresponding ItemProcessor
SubclassClassifier: to classify items according to their subtype.
Hope this helps.
Related
so I have a question on OOP class design. I have read that we should "Tell, don't ask" and not use Exceptions for "Flow control". However in this particular case I see some redundant code being executed!
Lets assume Person have a list of events that he will be attending, and it must be enforced that he cannot attend an event that overlaps with his current schedule. So I have the following Java code
public class Person {
// this arraylist of events must not have overlapping events!
ArrayList<Events> eventsToAttend;
// checks if a person is free to attend a new event by viewing all events he is attending
public boolean canAttendEvent(Event newEvent) {
for(int i = 0; i < eventsToAttend.size(); i++) {
if (newEvent.isSameDayAndTime(eventsToAttend.get(i))) {
return false;
}
}
return true;
}
public void attendEvent(Event newEvent) {
// enforce the validity of the newEvent
if (!canAttendEvent(newEvent)) {
// throw exception and return
}
eventsToAttend.add(newEvent);
}
public static main(String[] args) {
// just an example usage!
Person person = somePersonWithEventsAlready;
Event event = new Event();
if (person.canAttendEvent(event)) {
// !!!
// Notice that canAttendEvent() is called twice!! how do you prevent this?
// !!!
person.attendEvent(event);
}
// Alternatively I could just try - catch around person.attendEvent(), but is that bad practise?
}
}
The issue I am facing in general with this way of doing things, is that "canAttendEvent()" is being called twice. However it is good practice according to OOP design patterns?
What would be a better way to do something like this? Thank you for reading this.
try - catch in the main is the best way to achieve what you are trying to avoid: call twice the function canAttendEvent
I am new to Quarkus. I am trying to write a REST endpoint using quarkus reactive that receives an input, does some validation, transforms the input to a list and then writes a message to kafka. My understanding was converting everything to Uni/Multi, would result in the execution happening on the I/O thread in async manner. In, the intelliJ logs, I could see that the code is getting executed in a sequential manner in the executor thread. The kafka write happens in its own network thread sequentially, which is increasing latency.
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Multi<OutputSample> send(InputSample inputSample) {
ObjectMapper mapper = new ObjectMapper();
//deflateMessage() converts input to a list of inputSample
Multi<InputSample> keys = Multi.createFrom().item(inputSample)
.onItem().transformToMulti(array -> Multi.createFrom().iterable(deflateMessage.deflateMessage(array)))
.concatenate();
return keys.onItem().transformToUniAndMerge(payload -> {
try {
return producer.writeToKafka(payload, mapper);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return null;
});
}
#Inject
#Channel("write")
Emitter<String> emitter;
Uni<OutputSample> writeToKafka(InputSample kafkaPayload, ObjectMapper mapper) throws JsonProcessingException {
String inputSampleJson = mapper.writeValueAsString(kafkaPayload);
return Uni.createFrom().completionStage(emitter.send(inputSampleJson))
.onItem().transform(ignored -> new OutputSample("id", 200, "OK"))
.onFailure().recoverWithItem(new OutputSample("id", 500, "INTERNAL_SERVER_ERROR"));
}
I have been on it for a couple of days. Not sure if doing anything wrong. Any help would be appreciated.
Thanks
mutiny as any other reactive library is designed mainly around data flow control.
That being said, at its heart, it will offer a set of capabilities (generally through some operators) to control flow execution and scheduling. This means that unless you instruct munity objects to go asynchronous, they will simply execute in a sequential (old) fashion.
Execution scheduling is controlled using two operators:
runSubscriptionOn: which will cause the code snippet generating the items (which is generally referred to upstream) to execute on a thread from the specified Executor
emitOn: which will cause subscribing code (which is generally referred to downstream) to execute on a thread from the specified Executor
You can then update your code as follows causing the deflation to go asynchronous:
Multi<InputSample> keys = Multi.createFrom()
.item(inputSample)
.onItem()
.transformToMulti(array -> Multi.createFrom()
.iterable(deflateMessage.deflateMessage(array)))
.runSubscriptionOn(Infrastructure.getDefaultExecutor()) // items will be transformed on a separate thread
.concatenate();
EDIT: Downstream on a separate thread
In order to have the full downstream, transformation and writing to Kafka queue done on a separate thread, you can use the emitOn operator as follows:
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Multi<OutputSample> send(InputSample inputSample) {
ObjectMapper mapper = new ObjectMapper();
return Uni.createFrom()
.item(inputSample)
.onItem()
.transformToMulti(array -> Multi.createFrom().iterable(deflateMessage.deflateMessage(array)))
.emitOn(Executors.newFixedThreadPool(5)) // items will be emitted on a separate thread after transformation
.onItem()
.transformToUniAndConcatenate(payload -> {
try {
return producer.writeToKafka(payload, mapper);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return Uni.createFrom().<OutputSample>nothing();
});
}
Multi is intended to be used when you have a source that emits items continuously until it emits a completion event, which is not your case.
From Mutiny docs:
A Multi represents a stream of data. A stream can emit 0, 1, n, or an
infinite number of items.
You will rarely create instances of Multi yourself but instead use a
reactive client that exposes a Mutiny API.
What you are looking for is a Uni<List<OutputSample>> because your API you return 1 and only 1 item with the complete result list.
So what you need is to send each message to Kafka without immediately waiting for their return but collecting the generated Unis and then collecting it to a single Uni.
#POST
public Uni<List<OutputSample>> send(InputSample inputSample) {
// This could be injected directly inside your producer
ObjectMapper mapper = new ObjectMapper();
// Send each item to Kafka and collect resulting Unis
List<Uni<OutputSample>> uniList = deflateMessage(inputSample).stream()
.map(input -> producer.writeToKafka(input, mapper))
.collect(Collectors.toList());
// Transform a list of Unis to a single Uni of a list
#SuppressWarnings("unchecked") // Mutiny API fault...
Uni<List<OutputSample>> result = Uni.combine().all().unis(uniList)
.combinedWith(list -> (List<OutputSample>) list);
return result;
}
The project uses SpringBoot(2.3.4) and SpringCloud(Hoxton.SR8).
There are three classes: BillController, BillService(interface) and BillServiceImpl (implements BillService), BillController calls function getBillList declared in BillService.
In BillServiceImpl, there are two method, one is getBillList, the other is simulateUnstableService, getBillList calls simulateUnstableService, and in simulateUnstableService just a long sleep(2000).
The strange thing is that if I annoate getBillList with HystrixCommand, then it works as I expect. But if I move HystrixCommand to annoate simulateUnstableService, then there is no break which means timeout does not trigger Circuit Breaker.
#Service
public class BillServiceImpl implements BillService {
#Override
// have effact
#HystrixCommand(
commandProperties = {
#HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")
}
)
public List<Bill> getBillList(long userId) {
return simulateUnstableService(userId);
}
// no effact
// #HystrixCommand(
// commandProperties = {
// #HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")
// }
// )
public List<Bill> simulateUnstableService(long userId) {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ArrayList<>();
}
}
And more, if I just copy simulateUnstableService method content to getBillList, and annoate getBillList with HystrixCommand, the breaker also works.
Why?
Excellent question.
Hystrix uses AOP to wrap the method being called and deliver the Circuit Braking functionality. There is actually an aspect class HystrixCommandAspect.java which defines the around advice used to achieve this.
Now, AOPs don't exactly work if you call a method from within a class. See this answer for more clarity- Spring AOP not working for method call inside another method
When the circuit breaks the fallback method is called. We need to mention fallback method inside the hystrix command. The fallback method has the same signature as the method being annotated by hystrix.
Is simulateUnstableService your fallback method?
#HystrixCommand(fallbackMethod='yourFallbackMethod'
commandProperties = {
#HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")
}
)
public List<Bill> getBillList(long userId) {
return simulateUnstableService(userId);
}
Also, it is good practice to add the hystrix command properties inside the application.properties file instead of providing along with the annotation.
Basically, I am fetching data from another source and creating my db collections. But some of the data has spaces in the end, causing issues in frontend when used later.
Is there a generic way to trim all String fields of all Collections before inserting and updating to MongoDB using spring mongoTemplate configuration/code.
I don't want to write logic specific to each type of collection and each field in it. Also, is it a good practice to put such logic at DB repositories level?
Try to use AbstractMongoEventListener. For example, create create your own implementation of AbstractMongoEventListener:
#Component
class SaveMongoEventListener extends AbstractMongoEventListener<Object> {
#Override
public void onBeforeConvert(BeforeConvertEvent<Object> event) {
Object source = event.getSource();
for (Field field : source.getClass().getFields()) {
if (field.getType().isAssignableFrom(String.class)) {
try {
String value = (String) field.get(source);
field.setAccessible(true);
field.set(value != null ? value.trim(): value, source);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
}
}
}
}
This implementation will trim all strings in your object before converting it to MongoDB object. The listener should work for all your collections. Do not forget to register this listener in Spring context.
To trim string after loading from MongoDB, you should do the same in onAfterConvert event.
I am reading from source table (using JpaPagingItemReader) and passing to ItemProcessor.
My requirement is if Item is processed successfully then it should write to TABLE_A and if processing failed then write to TABLE_B.
I got it working, but I dont feel it as nice way.
My current implementation is
// my processor
public class MyItemProcessor implements ItemProcessor<SourceEntity, BaseOutputEntity>{
#Override
public BaseOutputEntity process(SourceEntity input) {
// NOTE: EntityA, EntityB both extend BaseOutputEntity
try {
EntityA a = callMyBusiness.method(input);
return a;
} catch (MyBusinessException e) {
EntityB b = createMyFailureObj(input)
return b;
}
}
}
// my itemwriter
public class MyItemWriter extends JpaItemWriter<MyBaseOutputEntity> {
// donthing as JpaItemWriter methods will take care
}
It is doing functionally what exactly I want.
One drawback of above is when I see job execution / step execution history, I can't know how many are successful or how many are failure, as it shows e.g. if 100 reads then 100 writes.
Can anyone suggest better approach. Are conditional steps useful here?
You can throw an exception on your processor, and declare this Exception as Skipable (if not, chunck will be broken).
If you implements an ItemProcessListener you can catch the invalid item on the onProcessError(Entry item, Exception t) function and write it on the table B.
(Read the documentation carefully: Some listeners functions are on transactions, others not)
At the end of the batch, writedItemsCount is the number of valids item, skippedItemCount is the number of invalid items.
Other way to write in different tables is use the ClassifierCompositeItemWriter with the BackToBackPatternClassifier but you loose the count of invalid items.