One to Many KStream-KTable join - apache-kafka

Question

I have a kStream of Universities -
when University is -
University(universityId: String, name: String, studentIds: Seq[String])
val universityKStream = builder.stream[String, University](...)
And a kTable of Students,
when Student is -
Student(studentId: String, name: String)
val studentsKtable = builder.table[String, Student](...)
I want to join the two and produce to a topic of ResolvedUniverity objects:
ResolvedUniversity(universityId: String, name: String, students: Seq[Student])
I cant groupBy and aggregate students with universityId, since universityId field doesn't exist in Student object..

Answer 1

Using just the DSL, I think the simplest you can do is (Java):
class Student {
String studentId;
String name;
}
class University {
String universityId;
String name;
List<String> studentIds;
}
class ResolvedUniversity {
String universityId;
String name;
List<Student> students;
}
Serde<String> stringSerde = null;
Serde<Student> studentSerde = null;
Serde<University> universitySerde = null;
Serde<ResolvedUniversity> resolvedUniversitySerde = null;
KStream<String, University> universities = topology
.stream("universities", Consumed.with(stringSerde, universitySerde));
KTable<String, Student> students = topology
.table("students", Consumed.with(stringSerde, studentSerde));
KTable<String, ResolvedUniversity> resolvedUniversities = universities
.flatMap((k, v) -> {
return v.studentIds.stream()
.map(id -> new KeyValue<>(id, v))
.collect(Collectors.toList());
})
.join(students, Pair::pair, Joined.with(stringSerde, universitySerde, studentSerde))
.groupBy((k, v) -> v.left().universityId)
.aggregate(ResolvedUniversity::new,
(k, v, a) -> {
a.universityId = v.left().universityId;
a.name = v.left().name;
a.students.add(v.right());
return a;
},
Materialized.with(stringSerde, resolvedUniversitySerde));
With this type of join, for historical processing your KTable of universities must be "primed" with its data before the KStream is joined against it.