Karate: Convert string to karate native variable in javascript - apache-kafka

Question

Our test automation needs to interact with kafka and we are looking at how we can achieve this with karate.
We have a java class which reads from kafka and puts records in an internal list. We then ask for these records from karate, filter out all messages from background traffic, and return the first message that matches our filter.
So our consumer looks like this (simplified):
// consume.js
function(bootstrapServers, topic, filter, timeout, interval) {
var KafkaLib = Java.type('kafka.KafkaLib')
var records = KafkaLib.getRecords(bootstrapServers, topic)
for (record_id in records) {
// TODO here we want to convert record to a json (and later xml for xml records) so that
// we can access them as 'native' karate data types and use notation like: cat.cat.scores.score[1]
var record = records[record_id]
if (filter(record)) {
karate.log("Record matched: " + record)
return record
}
}
throw "No records found matching the filter: " + filter
}
Records can be json, xml, or plain text, but looking in the json case now.
In this case given that in kafka there is a message like this:
{"correlationId":"b3e6bbc7-e5a6-4b2a-a8f9-a0ddf435de67","text":"Hello world"}
This is loaded as a string in the record variable above.
We want to convert this to json so that a filter like this would work:
* def uuid = java.util.UUID.randomUUID() + ''
# This is what we are publishing to kafka
* def payload = ({ correlationId: uuid, text: "Hello world" })
* def filter = function(m) { return m.correlationId == uuid }
Is there a way to convert a string to a native karate variable in javascript? Might have missed it looking at https://intuit.github.io/karate/#the-karate-object. By the way var jsonRecord = karate.toJson(record) did not work and jsonRecord.uuid was undefined.
Edit: I have made an example of what I am trying to achieve here:
https://github.com/KostasKgr/karate-issues/blob/java_json_interop/src/test/java/examples/consumption/consumption.feature
Many thanks

Answer 1

Sometime ago I had put together a something that could be used to test Kafka from within Karate. Pls see if https://github.com/Sdaas/karate-kafka helps. Happy to enhance / improve if it helps you.

Answer 2

Can you try,
* json payload = { correlationId: uuid, text: "Hello world" }
ref : Type Conversion
for type conversion within javascript ideally karate.toMap(object) or karate.toJson(object) should.
rather than wrapping up everything into one JS function, I would suggest keeping the record invoking part outside the JS and let karate cast it.
* json records = Java.type('kafka.KafkaLib').getRecords(bootstrapServers, topic)
* consume(records, filter, timeout, interval)

Answer 3

As mentioned in the comments of another answer, there is now an enhancement ticket on karate to achieve what was discussed in this thread, see https://github.com/intuit/karate/issues/1202
Until that is in place, I managed to get most of what I wanted concerning JSON by parsing string to json in Java and returning that to karate.
Map<String,Object> result = new ObjectMapper().readValue(record, HashMap.class);
Not sure if the same can be worked around for xml
You can see the workaround in action here:
https://github.com/KostasKgr/karate-issues/blob/java_json_interop_v2/src/test/java/examples/consumption/consumption.feature

Answer 4

Because of Karate's support for Java inter-op you can easily write some "glue" code to connect your existing Kafka systems to Karate test-suites, see the first link below.
Here are a few references:
how to use Java inter-op to listen and wait for events: https://twitter.com/KarateDSL/status/1417023536082812935
the Karate ActiveMQ example: https://github.com/intuit/karate/tree/master/karate-netty#consumer-provider-example
Walmart Labs blog post (Kafka specific): https://medium.com/walmartglobaltech/kafka-automation-using-karate-6a129cfdc210
Karate Kafka (3rd party project / example): https://github.com/Sdaas/karate-kafka