In a quarkus application, We have a complex object with other complex objects as attributes and we need to deserialize this from a consumerRecord. here are the classes examples
'''
#Data
public class Person {
private Hobbies hobbies;
private List<Relatives> relatives;
}
#Data
public class Hobbies {
private Sports sports;
private String principal;
}
#Data
public class Parents {
private Children kids;
private String mom;
}
'''
We have a consumer to receive the message like this
'''
#Incoming("topic_test")
public void process(ConsumerRecord<String, Person> consumerRecord {
log.info("Message key[{}] body [{}], consumerRecord.key(), consumerRecord.value());
final var person = toPerson(consumerRecord::value);
//call service
...
}
private Person toPerson(Callable<Person> callable) {
try{
return callable.call();
} catch Exception(e) {
return null;
}
}
'''
and finally we got our properties file
'''
%prod.mp.messaging.incoming.topic_test.connector=smallrye-kafka
%prod.mp.messaging.incoming.topic_test.topic=test-topic
%prod.mp.messaging.incoming.topic_test.value.serializer=io.quarkus.kafka.client.serialization.ObjectMapperSerializer
%prod.mp.messaging.incoming.topic_test.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
'''
I´m not sure about those deserializer configuration if it´s correct for our scneario, but also I didn´t find any way to deserialize this object 'Person'. We´ve tried to change for ObjectMapperSerializer but got same exception every time. Also tried to change consumeRecord for byte[] and String but it didn´t worked too, when we tried to call readValue using object mapper it couldn´t deserialize the object correctly.
'''
A message sent to channel 'topic_test' has been nacked, fail-stop
A failure has been reported for Kafka topics [test-topic]:
com.faster.jackson.databind.exc.InvalidDefinitionException: Cannot construct instace of 'projectpath'.Person (no Creators, like default constructor, exist): Cannot deserialize from Object value (no delegate- or property-based Creator)
'''
Also, I noticed that now messages are no longer consumed and are stuck on the partition
Can someone help me?
Related
I'm using Postgres as my datasource and I've created a custom Spring converter for a property that holds a list of my custom objects:
#Slf4j
#WritingConverter
#AllArgsConstructor
public class CustomObjectListToStringConverter implements Converter<List<CustomObject>, String> {
#Override
public String convert(#Nonnull List<CustomObject> source) {
try {
return objectMapper.writeValueAsString(source);
} catch (JsonProcessingException e) {
log.error("Error occurred while serializing list of CustomObject to JSON.", e);
}
return "[]";
}
}
Conversion goes smoothly but IllegalArgumentException is raised in getArrayType method of PostgresArrayColumns class because my custom type is not a simple type.
Is there a way to circumvent this guard for some property?
Currently, there is no override possible because DatabaseClient considers collection-typed values as values for Postgres' array fields. Please file a ticket at https://github.com/spring-projects/spring-data-r2dbc/ to fix the issue.
It is not supported intentionally based on the documentation.
Please note that converters get applied on singular properties. Collection properties (e.g. Collection) are iterated and converted element-wise. Collection converters (e.g. Converter<List>, OutboundRow) are not supported.
Source: spring-data-r2dbc mapping reference
Solution:
Create a wrapper class (complex type) as follows:
class CustomObjectList {
List<CustomObject> customObjects;
}
Then, you apply a converter Converter<CustomObjectList, String> and vice versa.
public class CustomObjectListToStringConverter implements Converter<CustomObjectList, String> {
I'm having an issue with some Scala code and am trying to rule out a bad class design (well, constructor design) before I begin to treat it as a networking issue.
So I have a model Scala class called Hail:
class Hail(val handle : String, val message : String) extends BaseMessage {
def this() {
this("default_user", "default_message")
}
}
abstract class BaseMessage extends AbstractMessage(true) {
}
// This is a 3rd party open source class written in Java
public abstract class AbstractMessage implements Message
{
private transient boolean reliable = true;
protected AbstractMessage()
{
}
protected AbstractMessage( boolean reliable )
{
this.reliable = reliable;
}
public Message setReliable(boolean f)
{
this.reliable = f;
return this;
}
public boolean isReliable()
{
return reliable;
}
}
// This is also a 3rd party open source class written in Java
public interface Message
{
public Message setReliable(boolean f);
public boolean isReliable();
}
At runtime, instances of this class get serialized (to binary), sent over the wire to a server, where they are deserialized (back into Hail instances) and processed.
So I have client code that looks like this:
val h1 : Hail = new Hail("user1", "Hello!")
val h2 : Hail = new Hail("user2", "Aloha!")
val h3 : Hail = new Hail("user3", "Bien venu mes amigos")
client.send(h1)
client.send(h2)
client.send(h3)
When the server receives these messages, it prints out their handle/message combos to STDOUT. The messages I receive are as follows:
Server received a Hail: default_user, default_message
Server received a Hail: default_user, default_message
Server received a Hail: default_user, default_message
Instead of what I would be expecting:
Server received a Hail: user1, Hello!
Server received a Hail: user1, Aloha!
Server received a Hail: user3, Bien venu mes amigos
Again, this could be a networking/serialization/server-side issue. But before I go down that route I want to make sure my Hail class has been written correctly.
The serialization framework I'm using requires all messages (such as Hail) to have no-arg constructors (hence the one I provided above). So it seems to me that something is wrong with my other constructor, and perhaps the server defaults to calling the no-arg constructor because it can't use anything else.
I decompiled my Hail class and see the following:
#ScalaSignature(bytes="<lots of bytes here omitted for brevity")
public class Hail
extends BaseMessage
{
private final String handle;
public String handle()
{
return this.handle;
}
public String message()
{
return this.message;
}
public Hail()
{
this("default_user", "default_message");
}
public Hail(String handle, String message) {}
}
Right away several things are curious/suspicious to me:
Although I see a private final String handle (which is desired!), I don't see a reciprocal private final String message field...
Also, the 2nd constructor (public Hail(String handle, String message)) is empty/undefined. This is probably the root of my issues.
So I ask, how can I refactor Hail's source so that the following end result is bytecode that would decompile to:
#ScalaSignature(bytes="<lots of bytes here omitted for brevity")
public class Hail
extends BaseMessage
{
private final String handle;
public String handle()
{
return this.handle;
}
public String message()
{
return this.message;
}
public Hail()
{
this("default_user", "default_message");
}
public Hail(String handle, String message)
{
this.handle = handle;
this.message = message;
}
}
Any ideas?
The problem is that the serializer server-side uses the no-arg constructor, and then change the values of mutable parameters in the class.
However, scala vals are not mutable, so they are compiled as final parameters of your class, and as such cannot be mutated. So the objects are instantiated with the default values, and then keep those values, since they cannot be mutated.
I would recommend using a scala-compatible serializer, but a simpler solution is to allow the properties to be mutated by declaring them as vars instead of vals.
Our problem is, our service GET /services/v1/myobject returns object named Xyz. This representation is used by multiple existing clients.
The new service GET /services/v2/myobject needs to expose exact same object but with different name, say XyzLmn
Now one obvious solution would be to create two classes Xyz and XyzLmn, then copy Xyz into XyzLmn and expose XyzLmn in the v2.
What I am looking for is, how can I keep the same java pojo class Xyz and conditionally serialize it to either XyzLmn or Xyz ?
have you try to add the #JsonIgnoreProperties(ignoreUnknown = true) on your domain object?
One solution is:
Write a customer serializer that emits XyzLmn
Register the customer serializer conditionally
public class XyzWrapperSerializer extends StdSerializer<Item> {
public ItemSerializer() {
this(null);
}
public ItemSerializer(Class<Item> t) {
super(t);
}
#Override
public void serialize(
Item value, JsonGenerator jgen, SerializerProvider provider)
throws IOException, JsonProcessingException {
jgen.writeStartObject();
jgen.writeNumberField("XyzLmn", value.id);
jgen.writeEndObject();
} }
XyzWrapper myItem = new XyzWrapper(1, "theItem", new User(2, "theUser"));
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(XyzWrapper.class, new XyzWrapperSerializer());
mapper.registerModule(module);
String serialized = mapper.writeValueAsString(myItem);
I want to convert Optional<BigDecimal> in morphia. I created BigDecimalConverter, and it works fine. Now I want to create OptionalConverter.
Optional can hold any object type. In my OptionalConverter.encode method I can extract underlying object, and I'd like to pass it to default mongo conversion. So that if there is string, I'll just get string, if there is one of my entities, I'll get encoded entity. How can I do it?
There are two questions:
1. How to call other converters?
2. How to create a converter for a generic class whose type parameters are not statically known?
The first one is possible by creating the MappingMongoConveter and the custom converter together:
#Configuration
public class CustomConfig extends AbstractMongoConfiguration {
#Override
protected String getDatabaseName() {
// ...
}
#Override
#Bean
public Mongo mongo() throws Exception {
// ...
}
#Override
#Bean
public MappingMongoConverter mappingMongoConverter() throws Exception {
MappingMongoConverter mmc = new MappingMongoConverter(
mongoDbFactory(), mongoMappingContext());
mmc.setCustomConversions(new CustomConversions(CustomConverters
.create(mmc)));
return mmc;
}
}
public class FooConverter implements Converter<Foo, DBObject> {
private MappingMongoConverter mmc;
public FooConverter(MappingMongoConverter mmc) {
this.mmc = mmc;
}
public DBObject convert(Foo foo) {
// ...
}
}
public class CustomConverters {
public static List<?> create(MappingMongoConverter mmc) {
List<?> list = new ArrayList<>();
list.add(new FooConverter(mmc));
return list;
}
}
The second one is much more difficult due to type erasure. I've tried to create a converter for Scala's Map but haven't found a way. Unable to get the exact type information for the source Map when writing, or for the target Map when reading.
For very simple cases, e.g. if you don't need to handle all possible parameter types, and there is no ambiguity while reading, it may be possible though.
I am trying to a find a good design for the following scenario.
I have a POST rest service which will be given an array of services as data. And which should in turn be calling them one by one to aggregate results on the server and send them back to the client.
#Path("/resource1")
#Path("/resource2")
#Path("/collection")
Post data to /collection
{["serviceName": "resource1", "data":"test1"], ["serviceName":"resource2","data":"test2"]}
The reason i need the resource1 and resource2 are, because those services can be called standalone also. I want to reuse the same setup if possible.
Is there any way to do this.
I am using jersey with spring.
Not sure what these resources have in common. If the post method has the same signature for all of them, you could have an abstract class or interface they implement defining the post method and can try using ResourceContext.matchResource to do this. E.g. something like this:
public abstract class AbstractResource {
public abstract String post(Object data);
}
#Path("/resource1")
public class Resource1 extends AbstractResource {
#POST
public String post(String data) {
// do something
}
}
#Path("/collection")
public class CollectionResource {
#Context
private ResourceContext rc;
#POST
#Consumes("application/json")
public String post(List<PostRequest> postRequests) {
StringBuilder result = new StringBuilder();
for (PostRequest pr : postRequests) {
// should wrap this in try-catch
AbstractResource ar = rc.matchResource(pr.resource,
AbstractResource.class);
sb.append(ar.post(pr.data));
}
return result.toString();
}
}
#XmlRootElement
public class PostRequest {
public String resource;
public String data;
}
Hopefully you got the idea and will be able to play with it and tweak it to fit your needs.