Returning PostgreSQL aggregations to a HashMap using MyBatis - postgresql

I have a super simple table test e.g.
create table test (
id serial primary key,
status varchar (10)
);
insert into test (status)
values ('ready'), ('ready'),
('steady'),
('go'), ('go'), ('go'),
('new');
To get the aggregated counts I can run: -
1) Simple multi-row result using group by
select status,
count(id) as count
from test
group by status
... which returns ...
-------+-------
status | counts
-------+-------
go | 3
ready | 2
new | 1
steady | 1
-------+-------
2) Single-Row Result using jsonb_object_agg
with stats as (
select status,
count(id) as count
from test
group by status
)
select jsonb_object_agg (status, count) as status_counts from stats
... which returns ...
--------------------------------------------------
status_counts
--------------------------------------------------
{ "go" : 3, "new" : 1, "ready" : 2, "steady" : 1 }
--------------------------------------------------
Mybatis Interface method.
In my Java code (via MyBatis) I have a method: -
public Map<String, Integer> selectStatusCounts();
What I'm keen to figure out is how to map either query to the Map<String, Integer> Java object via MyBatis?
Update (1)
On a_horse_with_no_name advice and this stackover article I've come up with this: -
3) Single-Row Result using hstore
select hstore(array_agg(hs_key), array_agg(hs_value::text))
from (
select
status,
count(id) as count
from test
group by status
) x(hs_key,hs_value)
... which returns ...
--------------------------------------------------
status_counts
--------------------------------------------------
"go"=>"3", "new"=>"1", "ready"=>"2", "steady"=>"1"
--------------------------------------------------
Using something like this could maybe work: -
https://github.com/gbif/checklistbank/blob/master/checklistbank-mybatis-service/src/main/java/org/gbif/checklistbank/service/mybatis/postgres/HstoreCountTypeHandler.java
Will test now! :-)
Update (2)
Thanks a_horse_with_no_name again for your contributions - I'm very close now but still weirdness with MyBatis. Here is a type handler I created (so I can reuse the aggregations elsewhere): -
#MappedTypes(LinkedHashMap.class)
#MappedJdbcTypes(JdbcType.OTHER)
public class MyBatisMapHstoreToStringIntegerMap implements TypeHandler<Map<String, Integer>> {
public MyBatisMapHstoreToStringIntegerMap() {}
public void setParameter(PreparedStatement ps, int i, Map<String, Integer> map, JdbcType jdbcType) throws SQLException {
ps.setString(i, HStoreConverter.toString(map));
}
public Map<String, Integer> getResult(ResultSet rs, String columnName) throws SQLException {
return readMap(rs.getString(columnName));
}
public Map<String, Integer> getResult(ResultSet rs, int columnIndex) throws SQLException {
return readMap(rs.getString(columnIndex));
}
public Map<String, Integer> getResult(CallableStatement cs, int columnIndex) throws SQLException {
return readMap(cs.getString(columnIndex));
}
private Map<String, Integer> readMap(String hstring) throws SQLException {
if (hstring != null) {
Map<String, Integer> map = new LinkedHashMap<String, Integer>();
Map<String, String> rawMap = HStoreConverter.fromString(hstring);
for (Map.Entry<String, String> entry : rawMap.entrySet()) {
map.put(entry.getKey(), Integer.parseInt(entry.getValue())); // convert from <String, String> to <String,Integer>
}
return map;
}
return null;
}
}
... and here's the mapper interface ...
public interface TestMapper {
public Map<String, Integer> selectStatusCounts();
}
... and here is the <select> inside the XML mapper file ...
<select id="selectStatusCounts" resultType="java.util.LinkedHashMap">
select hstore(array_agg(hs_key), array_agg(hs_value::text)) as status_counts
from (
select
status,
count(id) as count
from test
group by status
) x(hs_key,hs_value)
</select>
However, it returns a Map with one entry called status_counts the value of which is the actual map I want i.e. {status_counts={new=1, ready=2, go=3, steady=1}}
The following is my maven dependencies with regards PostgreSQL / MyBatis: -
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis-spring</artifactId>
<version>1.2.2</version>
</dependency>
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>3.3.1</version>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1201-jdbc41</version>
</dependency>

The easiest way is to define a hstore_agg() function:
CREATE AGGREGATE hstore_agg(hstore)
(
SFUNC = hs_concat(hstore, hstore),
STYPE = hstore
);
Then you can do this:
select hstore_agg(hstore(status, cnt::text))
from (
select status, count(*) cnt
from test
group by status
) t;
With the current JDBC driver Statement.getObject() will return a Map<String, String>.
As hstore only stores strings, it can't return a Map<String, Integer>

Posting an answer (which isn't perfect) but keen to see if anyone else has a better solution.
My answer is based on this stackoverflow answer: -
Return HashMap in mybatis and use it as ModelAttribute in spring MVC
I've created a POJO class called KeyValue: -
public class KeyValue<K, V> {
private K key;
private V value;
public KeyValue() {
}
public KeyValue(K key, V value) {
this.key = key;
this.value = value;
}
public K getKey() {
return key;
}
public void setKey(K key) {
this.key = key;
}
public V getValue() {
return value;
}
public void setValue(V value) {
this.value = value;
}
}
... and changed the test mapper method to ...
#MapKey("key")
public Map<String, KeyValue<String, Integer>> selectStatusCounts();
Note use of #MapKey parameter
I'm using the "1) Simple multi-row result using group by" SQL from original question and changed the result columns to key + value (so it maps to the new KeyValue object) as follows: -
<select id="selectStatusCounts" resultType="test.KeyValue">
select status as key,
count(id) as value
from bulk_upload
group by status
</select>
Accessing this in Java is achieved as follows: -
Map<String, KeyValue<String, Integer>> statusCounts = mapper.selectStatusCounts();
And to retrieve e.g. the value from map of new items we simply do: -
int numberOfstatusCounts = statusCounts.get("new").getValue();
I'm fairly happy with this solution but I would still prefer a Map<String, Integer> as opposed to a Map<String, KeyValue<String, Integer>> so not going to accept my solution - it's here purely just to show how I've got something working (for now).

Related

How to get the id value from an EclipseLink ClassDescriptor?

We currently have the following, working soft delete customizer in place:
public class SoftDeleteCustomizer implements DescriptorCustomizer {
#Override
public void customize(ClassDescriptor descriptor) {
descriptor.getQueryManager().setDeleteSQLString(
String.format("UPDATE %s SET record_status = 'D', record_status_time = CURRENT_TIMESTAMP WHERE id = #ID",
descriptor.getTableName()
)
);
}
}
We now want to add the user that deleted the record. I could sanitize the username, but I would prefer to use a parameter / argument.
I rewrote the customizer and did not set an argument for the #ID, as it was already injected correctly somewhere. I then found out that it was not injected when you are using a DeleteObjectQuery (with arguments?). So I have to add an argument for the #ID it seems, but I don't know how to get the id / primary key value of the record / entity to be deleted from a ClassDescriptor.
This is what I have so far:
#Override
public void customize(final ClassDescriptor descriptor) {
final DeleteObjectQuery query = new DeleteObjectQuery();
query.addArgument("DELETED_BY", String.class);
query.addArgument("ID", Long.class);
query.addArgumentValue(SecurityUtils.getUsername());
query.addArgumentValue(...); // How to get the ID of the record to delete?
query.setSQLString(String.format(DELETE_SQL, descriptor.getTableName()));
descriptor.getQueryManager().setDeleteQuery(query);
}
Okay, as a workaround I used our audit listener which we added as one of the EntityListeners. It implements SessionCustomizer. There I was able to do:
#Override
public void postDelete(final DescriptorEvent event) {
final Long id = ((AbstractEntity) event.getObject()).getId();
// Create and execute update query to set the username
}

Implementing pagination and sorting on a ReactiveMongoRepository with a dynamic query

I know pagination is somewhat against reactive principles, but due to requirements I have to make it work somehow. I'm using Spring Data 2.1.6 and I can't upgrade so ReactiveQuerydslSpecification for the dynamic query is out of the question. I figured I could use ReactiveMongoTemplate so I came up with this:
public interface IPersonRepository extends ReactiveMongoRepository<Person, String>, IPersonFilterRepository {
Flux<Person> findAllByCarId(String carId);
}
public interface IPersonFilterRepository {
Flux<Person> findAllByCarIdAndCreatedDateBetween(String carId, PersonStatus status,
OffsetDateTime from, OffsetDateTime to,
Pageable pageable);
}
#Repository
public class PersonFilterRepository implements IPersonFilterRepository {
#Autowired
private ReactiveMongoTemplate reactiveMongoTemplate;
#Override
public Flux<Person> findAllByCarIdAndCreatedDateBetween(String carId, PersonStatus status,
OffsetDateTime from, OffsetDateTime to,
Pageable pageable) {
Query query = new Query(Criteria.where("carId").is(carId));
if (status != null) {
query.addCriteria(Criteria.where("status").is(status));
}
OffsetDateTime maxLimit = OffsetDateTime.now(ZoneOffset.UTC).minusMonths(3).withDayOfMonth(1); // beginning of month
if (from == null || from.isBefore(maxLimit)) {
from = maxLimit;
}
query.addCriteria(Criteria.where("createdDateTime").gte(from));
if (to == null) {
to = OffsetDateTime.now(ZoneOffset.UTC);
}
query.addCriteria(Criteria.where("createdDateTime").lte(to));
// problem is trying to come up with a decent page-ish behavior compatible with Flux
/*return reactiveMongoTemplate.count(query, Person.class)
.flatMap(count -> reactiveMongoTemplate.find(query, Person.class)
.flatMap(p -> new PageImpl<Person>(p, pageable, count))
.collectList()
.map());*/
/* return reactiveMongoTemplate.find(query, Person.class)
.buffer(pageable.getPageSize(), pageable.getPageNumber() + 1)
//.elementAt(pageable.getPageNumber(), new ArrayList<>())
.thenMany(Flux::from);*/
}
I've tried to return a Page<Person> (assuming for once this single method could be non-reactive, for once) and it fails with the following error while running testing (Spring context does not load successfully due to: InvalidDataAccessApiUsageException: 'IDocumentFilterRepository.findAllByCustomerIdAndCreatedDateBetween' must not use sliced or paged execution. Please use Flux.buffer(size, skip). I've also tried returning Mono<Page<Person>> and then fails with "Method has to use a either multi-item reactive wrapper return type or a wrapped Page/Slice type. Offending method: 'IDocumentFilterRepository.findAllByCustomerIdAndCreatedDateBetween', so I guess my only option is returning a Flux, according to Example 133, snippet 3
Turns out you can just add the following to the query object:
query.with(pageable);
reactiveMongoTemplate.find(query, Person.class);
Return Flux<T> and it will work out of the box.

MyBatis ... Get Last insert ID in loop "foreach"

Thank you for help :)
I tried to get last id, and read many post about it, but i don't arrive to apply it in my case.
First Class
private Date date;
private List<AdsEntity> adsDetails;
... getters and setters
Second Class (AdsEntity)
private int id;
private String description;
There is the code where i try to get the last id :
Mapper
#Insert({
"<script>",
"INSERT INTO tb_ads_details (idMyInfo, adDate)"
+ " VALUES"
+ " <foreach item='adsDetails' index='index' collection='adsDetails' separator=',' statement='SELECT LAST_INSERT_ID()' keyProperty='id' order='AFTER' resultType='java.lang.Integer'>"
+ " (#{adsDetails.description, jdbcType=INTEGER}) "
+ " </foreach> ",
"</script>"})
void saveAdsDetails(#Param("adsDetails") List<AdsDetailsEntity> adsDetails);
In debugging mode, when I watch List I see the id still at 0 and don't get any id.
So what I wrote didn't workout :(
Solution Tried with the answer from #Roman Konoval :
#Roman Konoval
I apply what you said, and the table is fully well set :)
Just one problem still, the ID is not fulfill
#Insert("INSERT INTO tb_ads_details SET `idMyInfo` = #{adsDetail.idMyInfo, jdbcType=INTEGER}, `adDate` = #{adsDetail.adDate, jdbcType=DATE}")
#SelectKey(statement = "SELECT LAST_INSERT_ID()", before = false, keyColumn = "id", keyProperty = "id", resultType = Integer.class )
void saveAdsDetails(#Param("adsDetail") AdsDetailsEntity adsDetail);
default void saveManyAdsDetails(#Param("adsDetails") List<AdsDetailsEntity> adsDetails)
{
for(AdsDetailsEntity adsDetail:adsDetails) {
saveAdsDetails(adsDetail);
}
}
Thank for your help :)
Solution add to #Roman Konoval proposal from #Chris advice
#Chris and #Roman Konoval
#Insert("INSERT INTO tb_ads_details SET `idMyInfo` = #{adsDetail.idMyInfo, jdbcType=INTEGER}, `adDate` = #{adsDetail.adDate, jdbcType=DATE}")
#SelectKey(statement = "SELECT LAST_INSERT_ID()", before = false, keyColumn = "id", keyProperty = "adsDetail.id", resultType = int.class )
void saveAdsDetails(#Param("adsDetail") AdsDetailsEntity adsDetail);
default void saveManyAdsDetails(#Param("adsDetails") List<AdsDetailsEntity> adsDetails)
{
for(AdsDetailsEntity adsDetail:adsDetails) {
saveAdsDetails(adsDetail);
}
}
Thanks to all of you, for the 3 suggestions!!!
yes. it doesnt work.
please take a look at mapper.dtd
foreach-tag doesnt support/provide the following properties statement, keyProperty order and resultType
if you need the id for each inserted item please let your DataAccessObject handle iteration and use something like this in your MapperInterface
#Insert("INSERT INTO tb_ads_details (idMyInfo, adDate) (#{adsDetail.idMyInfo, jdbcType=INTEGER}, #{adsDetail.adDate, jdbcType=DATE})")
#SelectKey(before = false, keyColumn = "ID", keyProperty = "id", resultType = Integer.class, statement = { "SELECT LAST_INSERT_ID()" } )
void saveAdsDetails(#Param("adsDetail") AdsDetailsEntity adsDetail);
please ensure AdsDetailsEntity-Class provides the properties idMyInfoand adDate
Edit 2019-08-21 07:25
some explanation
referring to the mentioned dtd the <selectKey>-tag is only allowed as direct child of <insert> and <update>. it refers to a single Object that is passed into the mapper-method and declared as parameterType.
its only executed once and its order property tells myBatis wether to execute it before or after the insert/update statement.
in your case, the <script> creates one single statement that is send to and handled by the database.
it is allowed to combine #Insert with <script> and <foreach> inside and #SelectKey. but myBatis doesnt intercept/observe/watch database handling the given statement. and as mentioned before, #SelectKey gets executed only once, before or after #Insert-execution. so in your particular case #SelectKey returns the id of the very last inserted element. if your script inserts ten elements, only the new generated id of tenth element will be returned. but #SelectKey requires a class-property with getter and setter to put the selected id into - which List<?> doesnt provide.
example
lets say you want to save an Advertisement and its AdvertisementDetails
Advertisement has an id, a date and details
public class Advertisement {
private List<AdvertisementDetail> adDetails;
private Date date;
private int id;
public Advertisement() {
super();
}
// getters and setters
}
AdvertisementDetail has its own id, a description and an id the Advertisementit belongs to
public class AdvertisementDetail {
private String description;
private int id;
private int idAdvertisement;
public AdvertisementDetail() {
super();
}
// getters and setters
}
the MyBatis-mapper could look like this. #Param is not used, so the properties are accessed direct.
#Mapper
public interface AdvertisementMapper {
#Insert("INSERT INTO tb_ads (date) (#{date, jdbcType=DATE})")
#SelectKey(
before = false,
keyColumn = "ID",
keyProperty = "id",
resultType = Integer.class,
statement = { "SELECT LAST_INSERT_ID()" })
void insertAdvertisement(
Advertisement ad);
#Insert("INSERT INTO tb_ads_details (idAdvertisement, description) (#{idAdvertisement, jdbcType=INTEGER}, #{description, jdbcType=VARCHAR})")
#SelectKey(
before = false,
keyColumn = "ID",
keyProperty = "id",
resultType = Integer.class,
statement = { "SELECT LAST_INSERT_ID()" })
void insertAdvertisementDetail(
AdvertisementDetail adDetail);
}
the DataAccessObject (DAO) could look like this
#Component
public class DAOAdvertisement {
#Autowired
private SqlSessionFactory sqlSessionFactory;
public DAOAdvertisement() {
super();
}
public void save(
final Advertisement advertisement) {
try (SqlSession session = this.sqlSessionFactory.openSession(false)) {
final AdvertisementMapper mapper = session.getMapper(AdvertisementMapper.class);
// insert the advertisement (if you have to)
// its new generated id is received via #SelectKey
mapper.insertAdvertisement(advertisement);
for (final AdvertisementDetail adDetail : advertisement.getAdDetails()) {
// set new generated advertisement-id
adDetail.setIdAdvertisement(advertisement.getId());
// insert adDetail
// its new generated id is received via #SelectKey
mapper.insertAdvertisementDetail(adDetail);
}
session.commit();
} catch (final PersistenceException e) {
e.printStackTrace();
}
}
}
What Chris wrote about inability to get ids in the foreach is correct. However there is a way to implement id fetching in mapper without the need to do it externally. This may be helpful if you use say spring and don't have a separate DAO layer and your mybatis mappers are the Repository.
You can use default interface method (see another tutorial about them) to insert the list of items by invoking a mapper method for single item insert and single item insert method does the id selection itself:
interface ItemMapper {
#Insert({"insert into myitem (item_column1, item_column2, ...)"})
#SelectKey(before = false, keyColumn = "ID",
keyProperty = "id", resultType = Integer.class,
statement = { "SELECT LAST_INSERT_ID()" } )
void saveItem(#Param("item") Item item);
default void saveItems(#Param("items") List<Item> items) {
for(Item item:items) {
saveItem(item);
}
}
MyBatis can assign generated keys to the list parameter if your DB/driver supports multiple generated keys via java.sql.Statement#getGeneratedKeys() (MS SQL Server, for example, does not support it, ATM).
The following example is tested with MySQL 5.7.27 + Connector/J 8.0.17 (you should include version info in the question).
Be sure to use the latest version of MyBatis (=3.5.2) as there have been several spec changes and bug fixes recently.
Table definition:
CREATE TABLE le tb_ads_details (
id INT PRIMARY KEY AUTO_INCREMENT,
description VARCHAR(32)
)
POJO:
private class AdsDetailsEntity {
private int id;
private String description;
// getters/setters
}
Mapper method:
#Insert({
"<script>",
"INSERT INTO tb_ads_details (description) VALUES",
"<foreach item='detail' collection='adsDetails' separator=','>",
" (#{detail.description})",
"</foreach>",
"</script>"
})
#Options(useGeneratedKeys = true, keyProperty="adsDetails.id", keyColumn="id")
void saveAdsDetails(#Param("adsDetails") List<AdsDetailsEntity> adsDetails);
Note: You should use batch insert (with ExecutorType.BATCH) instead of multi-row insert (=<foreach/>) when inserting a lot of rows.

Serialization with KStream groupBy operation

I am trying to perform a count operation on a KStream and running into some difficulty in understanding how serialization is working here. I have a stream that is pushing people information e.g. name, age. After consuming this stream, i am trying to create a KTable with a count of people's age.
Input:
{"name" : "abc","age" : "15"}
Output:
30, 10
20, 4
10, 8
35, 22
...
Properties
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "person_processor");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
Processor
KStream<Object, Person> people = builder.stream("people");
people.print(Printed.<Object, Person>toSysOut().withLabel("consumer-1"));
Output
[consumer-1]: null, [B#7e37bab6
Question-1
I understand that data in the topic is in bytes. I am not setting any Serdes for Key or Value to start with. Is KStream converting the input from bytes to Person and printing the address of Person here?
Question-2
When I add the below value Serdes, I get a more meaningful output. Is the byte information here getting converted to String and then to Person? Why is the value now printed correctly?
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
[consumer-1]: null, {"name" : "abc","age" : "15"}
Question-3
Now, when performing the count on the age, I get a runtime error on converting a String to Person. If groupBy is setting the age as the Key and the count as Long, why is the String to Person conversion happening?
KTable<Integer, Long> integerLongKTable = people.groupBy((key, value) -> value.getAge())
.count();
Exception in thread "person_processor-9ff96b38-4beb-4594-b2fe-ae191bf6b9ff-StreamThread-1" java.lang.ClassCastException: java.lang.String cannot be cast to com.example.kafkastreams.KafkaStreamsApplication$Person
at org.apache.kafka.streams.kstream.internals.KStreamImpl$1.apply(KStreamImpl.java:152)
at org.apache.kafka.streams.kstream.internals.KStreamImpl$1.apply(KStreamImpl.java:149)
Edit-1
After reading through the response from #Matthias J. Sax I created a PersonSerde using the Serializer and DeSerializer from this locatio, I get this SerializationException...
https://github.com/apache/kafka/tree/1.0/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview
static class Person {
String name;
String age;
public Person(String name, String age) {
this.name = name;
this.age = age;
}
void setName(String name) {
this.name = name;
}
String getName() {
return name;
}
void setAge(String age) {
this.age = age;
}
String getAge() {
return age;
}
#Override
public String toString() {
return "Person {name:" + this.getName() + ",age:" + this.getAge() + "}";
}
}
public class PersonSerde implements Serde {
#Override
public void configure(Map map, boolean b) {
}
#Override
public void close() {
}
#Override
public Serializer serializer() {
Map<String, Object> serdeProps = new HashMap<>();
final Serializer<Person> personSerializer = new JsonPOJOSerializer<>();
serdeProps.put("JsonPOJOClass", Person.class);
personSerializer.configure(serdeProps, false);
return personSerializer;
}
#Override
public Deserializer deserializer() {
Map<String, Object> serdeProps = new HashMap<>();
final Deserializer<Person> personDeserializer = new JsonPOJODeserializer<>();
serdeProps.put("JsonPOJOClass", Person.class);
personDeserializer.configure(serdeProps, false);
return personDeserializer;
}
}
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, personSerde.getClass());
KTable<String, Long> count = people.selectKey((key, value) -> value.getAge()).groupByKey(Serialized.with(Serdes.String(), personSerde))
.count();
Error
Caused by: org.apache.kafka.common.errors.SerializationException: Error serializing JSON message
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.example.kafkastreams.KafkaStreamsApplication$Person and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:313)
Edit 5
So it appears that when I mapValues to a String, then count works correctly. But when I use it on a custom Object, it fails
KStream<String, Person> people = builder.stream("person-topic", Consumed.with(Serdes.String(), personSerde));
people.print(Printed.<String, Person>toSysOut().withLabel("person-source"));
KStream<String, Person> agePersonKStream = people.selectKey((key, value) -> value.getAge());
agePersonKStream.print(Printed.<String, Person>toSysOut().withLabel("age-person"));
KStream<String, String> stringStringKStream = agePersonKStream.mapValues((person -> person.name));
stringStringKStream.print(Printed.<String, String>toSysOut().withLabel("age-name"));
KTable<String, Long> stringLongKTable = stringStringKStream.groupByKey(Serialized.with(Serdes.String(), Serdes.String())).count();
stringLongKTable.toStream().print(Printed.<String, Long>toSysOut().withLabel("age-count"));
Without the 3 step to mapValues to name, step 4 fails.
Question-1 I understand that data in the topic is in bytes. I am not setting any Serdes for Key or Value to start with. Is KStream converting the input from bytes to Person and printing the address of Person here?
If you don't specify any Serde in StreamsConfig or in builder.stream(..., Consumers.with(/*serdes*/)) the bytes won't be converted into a Person object but the object will be of type byte[]. Thus, print() will call byte[].toString() that results in the cryptic output ([B#7e37bab6) you see.
Question-2 When I add the below value Serdes, I get a more meaningful output. Is the byte information here getting converted to String and then to Person? Why is the value now printed correctly?
As you specify Serde.String() in StreamsConfig the bytes are converted to String type. It seems, that StringSerde is able to deserialize the bytes in a meaningful way -- but this seems to be a coincidence that it works at all. It seems that your data is actually serialized in JSON, what would explain why StringSerde() can convert the bytes into a String.
Question-3 Now, when performing the count on the age, I get a runtime error on converting a String to Person. If groupBy is setting the age as the Key and the count as Long, why is the String to Person conversion happening?
That is expected. Because the bytes are converted into a String object (as you specified Serdes.String()), the cast cannot be performed.
Final remarks:
You don't get a class cast exception if you only use print(), because for this case, no cast operation is performed. Java only inserts a cast operation if required.
For groupBy() you use value.getAge() and thus Java inserts a cast here (it knows that the expected type is Person, because it's specified via KStream<Object, Person> people = .... For print() only toString() is called that is define on Object and thus no cast is required.
Generics in Java a type hints for the compiler and replaced with Object (or casted if required during compile time). Thus, for print() a Object variable can point to an byte[] without problem and toString() is called successfully. For groupBy() case the compiler cast Object to Person to be able to call getAge() -- however, this fails, because the actually type is String.
To get your code working, you need to create a PersonSerde extend Serde<Person> class and specify it as value serde.

CacheBuilder using guava cache for query resultant

To reduce the DB hits to read the data from DB using the query, I am planning to keep resultant in the cache. To do this I am using guava caching.
studentController.java
public Map<String, Object> getSomeMethodName(Number departmentId, String departmentType){
ArrayList<Student> studentList = studentManager.getStudentListByDepartmentType(departmentId, departmentType);
----------
----------
}
StudentHibernateDao.java(criteria query )
#Override
public ArrayList<Student> getStudentListByDepartmentType(Number departmentId, String departmentType) {
Criteria criteria =sessionFactory.getCurrentSession().createCriteria(Student.class);
criteria.add(Restrictions.eq("departmentId", departmentId));
criteria.add(Restrictions.eq("departmentType", departmentType));
ArrayList<Student> studentList = (ArrayList)criteria.list();
return studentList;
}
To cache the criteria query resultant i started off with building CacheBuilder, like below.
private static LoadingCache<Number departmentId, String departmentType, ArrayList<Student>> studentListCache = CacheBuilder
.newBuilder().expireAfterAccess(1, TimeUnit.MINUTES)
.maximumSize(1000)
.build(new CacheLoader<Number departmentId, String departmentType, ArrayList<Student>>() {
public ArrayList<Student> load(String key) throws Exception {
return getStudentListByDepartmentType(departmentId, departmentType);
}
});
Here I dont know where to put CacheBuilder function and how to pass multiple key parameters(i.e departmentId and departmentType) to CacheLoader and call it.
Is this the correct way of caching using guava. Am I missing anything?
Guava's cache only accepts two type parameters, a key and a value type. If you want your key to be a compound key then you need to build a new compound type to encapsulate it. Effectively it would need to look like this (I apologize for my syntax, I don't use Java that often):
// Compound Key type
class CompoundDepartmentId {
public CompoundDepartmentId(Long departmentId, String departmentType) {
this.departmentId = departmentId;
this.departmentType = departmentType;
}
}
private static LoadingCache<CompoundDepartmentId, ArrayList<Student>> studentListCache =
CacheBuilder
.newBuilder().expireAfterAccess(1, TimeUnit.MINUTES)
.maximumSize(1000)
.build(new CacheLoader<CompoundDepartmentId, ArrayList<Student>>() {
public ArrayList<Student> load(CompoundDepartmentId key) throws Exception {
return getStudentListByDepartmentType(key.departmentId, key.departmentType);
}
});