I'm following the confluent tutorial to produce messages (Produce and Consume Avro Messages), but when I post in the messages defining the schema registry, it gives the following error and I don't know how to continue, I've looked in several places and none have this error
{
"error_code": 40801,
"message": "Error when registering schema. format = AVRO, subject = teste-value, schema = {\"type\":\"record\",\"name\":\"teste\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"}]}"
}
curl -X POST -H "Content-Type: application/vnd.kafka.avro.v2+json" \
-H "Accept: application/vnd.kafka.v2+json" \
--data '{"value_schema": "{\"type\": \"record\", \"name\": \"teste\", \"fields\": [{\"name\": \"name\", \"type\": \"string\"}]}", "records": [{"value": {"name": "teste"}}]}' \
"http://localhost:38082/topics/teste"
Can you try formatting your query like the following and give it a try.
curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema": "{\"type\":\"record\",\"name\":\"Payment\",\"namespace\":\"io.confluent.examples.clients.basicavro\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"amount\",\"type\":\"double\"}]}"}' http://localhost:8081/subjects/test-value/versions
Related
I have a trouble producing to Kafka topic through Rest API proxy.
I have a running confluent kafka cluster in which I'd like to create a topic and produce a message to it using REST API.
For that purpose I have followed the documentation and created API key and secret.
I manage to create topic:
curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>" \
-H "Content-Type: application/json" \
-d "{\"topic_name\":\"test1\",\"partitions_count\":6,\"configs\":[]}" \
https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics" | jq
-------------------
returns:
{
"kind": "KafkaTopic",
"metadata": {
"self": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1",
"resource_name": "crn:///kafka=lkc-1zoqz/topic=test1"
},
"cluster_id": "lkc-1zoqz",
"topic_name": "test1",
"is_internal": false,
"replication_factor": 0,
"partitions_count": 0,
"partitions": {
"related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/partitions"
},
"configs": {
"related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/configs"
},
"partition_reassignments": {
"related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/partitions/-/reassignment"
},
"authorized_operations": []
}
MY PROBLEM:
I can't produce to that topic (can't produce to ANY topic through Kafka Rest API) :
curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>" \
-H "Content-Type: application/json" \
--data "{"records":[{"value":"S2Fma2E="}]}" \
"https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics/test1"
-----------------
returns:
{"error_code":405,"message":"HTTP 405 Method Not Allowed"}
ALSO TRIED LIKE THIS:
curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>" \
-H "Content-Type: application/json" \
-d "{ \"key\": {\"value\": \"S2Fma2E=\"} }" \
"https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics/test1/records"
----------------
Returns the same:
{"error_code":405,"message":"HTTP 405 Method Not Allowed"}
IDK if this has something to do with ACL management? Looking into that right now....
Are you using Confluent Platform (hosted) or Confluent Cloud API?
If the latter, this API is not supported, here's the documentation on what's supported by Cloud API: https://docs.confluent.io/cloud/current/api.html
I'm trying to retrieve yesterday's transfer for one of our customers. Please check the following curl request
curl --request GET \
--url https://api.stripe.com/v1/transfers \
--header 'Authorization: Basic ourPrivateKeyEncoded' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data destination=acct_anAccount \
--data created.gt=1629676800
I'm able to retrieve all the transfers of my customer, but the created filter is not accepted. Meaning the above request without the created.gt is ok.
With this filter Stripe respond me a 400 with this body
{
"error": {
"code": "parameter_unknown",
"doc_url": "https://stripe.com/docs/error-codes/parameter-unknown",
"message": "Received unknown parameter: created.gt. Did you mean created?",
"param": "created.gt",
"type": "invalid_request_error"
}
}
I'm following this documentation, the created filter seems to exist. Does someone have an idea why Stripe doesn't accept my filter?
Replacing created.gt by created[gt] filter key made my request works.
The documentation seems to be outdated
I am using Kafka rest proxy, but not whole Confluent Platform just Kafka rest with my Kafka brokers. But I am not able to create topics from command line by the following command.
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test.
I want to know is there any other way out.
The command you have tried out is not meant to interact with the REST Proxy Server of Kafka, but rather interacting with the Kafka cluster directly.
According to the Confluent REST Proxy API Reference the creation of a topic is only possible with the REST Proxy API v3 that is currently available as a preview feature.
"The API v3 can be used for evaluation and non-production testing purposes or to provide feedback to Confluent."
An example of a topic creation request is presented below and documented here:
POST /v3/clusters/cluster-1/topics HTTP/1.1
Host: kafkaproxy.example.com
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json
{
"data": {
"attributes": {
"topic_name": "topic-1",
"partitions_count": 2,
"replication_factor": 3,
"configs": [
{
"name": "cleanup.policy",
"value": "compact"
}
]
}
}
}
Using curl:
curl -X POST -H "Content-Type: application/vnd.api+json" -H "Accept: application/vnd.api+json" \
--data '{"data":{"attributes": {"topic_name": "topic-1", "partitions_count": 2, "replication_factor": 1, "configs": [{"name": "cleanup.policy","value": "compact"}]}}}' \
"http://localhost:8082/v3/clusters/<cluster-id>/topics"
where the cluster-id can be identified using
curl -X GET -H "Accept: application/vnd.api+json" localhost:8082/v3/clusters
I try export and import nifi templates via nifi rest api; but when i tried to delete controller-services, connections or import a template i get the response message "The revision must be specified." with '400 bad request' status from nifi rest api. I didn't understand what i have to do.
This my request for delete a control-service,
request "/controller-services/{id}" (id is, The controller service id.)
curl -X DELETE \
http://localhost:8090/nifi-api/controller-services/015b1030-a099-13d3-812c-77772afcaeb0 \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded' \
and the response is "The revision must be specified."
status : "400 Bad Request"
Any idea please?
I finally got some answers about my mistake after reading this article
It seems you have to add two information to request url; version and clientId.
After make a call and get a component form nifi instance, it sends a version number inside a DTO to you as explained here
like this:
and finally i made a call like below and it worked:
curl -X DELETE \
'http://localhost:8090/nifi-api/controller-services/015b1030-a099-13d3-812c-77772afcaeb0?version=0&clientId=5ed15669-015d-1000-99cf-bc9c6f378085' \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded' \
As per Couchbase Sync-Gateway REST API documentation here below mentioned cURL should create a document in the specified database.
Below is the generated cURL from Postman.
curl -X PUT -H "Cache-Control: no-cache" -H "Postman-Token: 498d0fb6-77ac-9335-2379-14258c6731c7" -d '' "http://192.168.244.174:4984/db/"
I also tried adding JSON to the body of the request.
But when I send the put request through Postman, instead of creating a new document, it tries to create a new database and the JSON response is
{
"error": "Precondition Failed",
"reason": "Database already exists"
}
Am I missing something or it was a bug? Is there any other way to create a document to sync gateway?
There is a mistake in the documentation.
As per documentation,
You can either specify the document ID by including the _id object in the request message body, or let the software generate an ID.
But Couchbase REST API does not seem to work like that (may be they are not updating their documentation regularly). You need to provide the id in the URL like /{db}/{id}.
The below cURL worked for me.
curl -X PUT -H "Content-Type: application/json" -H "Cache-Control: no-cache" -H "Postman-Token: 75ab844e-5130-708e-69e9-e87f878108b4" -d '{"name": "xxx",
"full_name": "xxx yyy"}' "http://192.168.244.174:4984/db/123"
JSON response is
{
"id": "123",
"ok": true,
"rev": "1-9324dabc947fc963a754b113d1215ac3"
}