Kafka Create Topic API Options for Non-Java Languages - apache-kafka

Question

While you can create a topic via Java or Java-based languages (see here), there does not seem to be a clean way to do this without using Java. As a result, pure-language client APIs (like kafka-node, a pure JavaScript client) can't directly create topics. Instead, we have two options:
1) Use a hack like sending a metadata request to a topic -- if auto.create.topics.enable is set to true, then you can create a topic -- but only with the default configuration, no control over partitions, etc.
2) Write a wrapper around a Java-based client just for topic creation. The easiest way to do this is to exec the script bin/kafka-topics.sh with command line arguments, which is ugly, to say the least.
Is there a better way to do this, though? There's a pure-JavaScript client for Zookeeper, node-zookeeper-client, what happens if I manipulate broker / partition info directly in Zookeeper?
Any other thoughts?

Answer 1

You can now use REST Proxy API v3 to create Kafka topics with http requests for non-Java languages.
According to the Confluent REST Proxy API Reference the creation of a topic is possible with the REST Proxy API v3 that is currently available as a preview feature.
"The API v3 can be used for evaluation and non-production testing purposes or to provide feedback to Confluent."
An example of a topic creation request is presented below and documented here:
POST /v3/clusters/cluster-1/topics HTTP/1.1
Host: kafkaproxy.example.com
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json
{
"data": {
"attributes": {
"topic_name": "topic-1",
"partitions_count": 2,
"replication_factor": 3,
"configs": [
{
"name": "cleanup.policy",
"value": "compact"
}
]
}
}
}
Using curl:
curl -X POST -H "Content-Type: application/vnd.api+json" -H "Accept: application/vnd.api+json" \
--data '{"data":{"attributes": {"topic_name": "topic-1", "partitions_count": 2, "replication_factor": 1, "configs": [{"name": "cleanup.policy","value": "compact"}]}}}' \
"http://localhost:8082/v3/clusters/<cluster-id>/topics"
where the cluster-id can be identified using
curl -X GET -H "Accept: application/vnd.api+json" localhost:8082/v3/clusters