How to print TimeWindowedKStream and KTable in Kafka streams? - apache-kafka

Question

We have a Kafka process that takes a topic as input and writes timed window to the output topic.. the following code is being used. I would like to print TimeWindowedKStream(groupedStream) and KTable(aggregatedTable) and see the output for some debugging purposes..
String intopic = input_topic;
Long window = 60;
String outtopic = output_topic;
final Serde<String> stringSerde = Serdes.String();
Properties property = new Properties();
property.put("bootstrap.servers", "127.0.0.1:9092");
property.put("group.id", "test-consumer-group");
property.put("application.id", "sliding-window-min-bar");
property.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, stringSerde.getClass().getName());
property.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, stringSerde.getClass().getName());
Duration windowSizeMs = Duration.ofMinutes(window);
StreamsBuilder builder = new StreamsBuilder();
System.out.println(intopic);
KStream<String, String> equitybar = builder.stream(intopic, Consumed.with(stringSerde, stringSerde));
System.out.println(equitybar);
equitybar.print(Printed.toSysOut());
// convert string of csv to a double on the mean value
KStream<String, String> transformedbar = equitybar
.map((key, value) -> KeyValue.pair(key, value.substring(1,value.length()-2).split(",")[2]));
System.out.println(transformedbar);
transformedbar.print(Printed.toSysOut());
// group by equity and sliding window
System.out.println(windowSizeMs);
System.out.println(TimeWindows.of(windowSizeMs).advanceBy(advanceMs));
TimeWindowedKStream<String, String> groupedStream = transformedbar.groupByKey().windowedBy(TimeWindows.of(windowSizeMs).advanceBy(advanceMs));
System.out.println(groupedStream);
KTable<Windowed<String>, String> aggregatedTable = groupedStream.aggregate(
() -> "|",
(aggKey, newValue, aggValue) -> aggValue + newValue.trim() + "|") ;
I tried to print it using the the print command that is used for Kafka streams - groupedStream.print(Printed.toSysOut()); - but it doesn't seem to be working.
Thanks.

Answer 1

KGroupedStream and TimeWindowedKStream are "just" helper classes to allow the DSL to present a fluent API to chain operator without too many overloads on a single class.
In the DSL, there are only two main abstractions, KStream and KTable that are actual first-class data-containers. Thus, it's not possible what you want to do.