OpenSearch 1.2 - Index pattern programmatically - opensearch

I am trying to create an index_pattern for dashboard in Opensearch programmatically.
Due to the fact that I didn't found anything related to it on Opensearch documentation, I tried the elastic search saved_objects api:
POST /api/saved_objects/index-pattern/my_index_pattern
{
"attributes":{
"title": "my_index"
}
}
But when I call it I got the following error:
{
"error" : "no handler found for uri [/api/saved_objects/index-pattern/my_index_pattern?pretty=true] and method [POST]"
}
How can I solve it? Are there a different call for Opensearch to create an index_pattern?

curl -k -X POST http://<host>:5601/api/saved_objects/index-pattern/logs-* -H "osd-xsrf:true" -H "content-type:application/json" -d '
{
"attributes": {
"title": "logs-*",
"timeFieldName": "#timestamp"
}
}'
this works for me

Related

Setting mirror maker 2 using kafka connect rest api put method not allowed

I am trying to do the setup for mirror maker 2 using my current connect cluster.
Based on this documentation it can be done via connect rest api.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-RunningMirrorMakerinaConnectcluster
I followed the sample sending this PUT request :
PUT /connectors/us-west-source/config HTTP/1.1
{
"name": "us-west-source",
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"source.cluster.alias": "us-west",
"target.cluster.alias": "us-east",
"source.cluster.bootstrap.servers": "us-west-host1:9091",
"topics": ".*"
}
but i am getting a method not allowed response error response.
{
"error_code": 405,
"message": "HTTP 405 Method Not Allowed"
}
The api looks ok if I do a simple GET from the / , returning the version
{
"version": "2.1.0-cp1",
"commit": "bda8715f42a1a3db",
"kafka_cluster_id": "VBo-j1OAQZSN8tO4lMJ0Gg"
}
the PUT method doesnt work, using POST works as the api's documentation shows:
https://docs.confluent.io/current/connect/references/restapi.html#get--connectors
Remove the name of the connector from the url as #cricket_007 suggested, and wrap the config with new element like this:
curl --noproxy "*" -XPOST -H 'Content-Type: application/json' -H 'Accept: application/json' http://localhost:8083/connectors -d'{
"name": "dc-west-source",
"config": {
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"source.cluster.alias": "dc-west",
"target.cluster.alias": "dc-east",
"source.cluster.bootstrap.servers": "dc-west-cp-kafka-0.domain:32721,dc-west-cp-kafka-1.domain:32722,dc-west-cp-kafka-2.dc.domain:32723",
"topics": ".*"
}
}
' | jq .

Request body matching with XML declaration in Wiremock-standalone

I have a stub described in the ./mappings/*.json file.
"request": {
"method": "POST",
"url": "/some/thing",
"bodyPatterns" : [ {
"matchesXPath" : {
"expression": "//nodeA/text()",
"contains": "999"
}
} ]
}
Wiremock (ver.2.26.2) is started in standalone mode.
When I call the service like this:
curl -d "<request><nodeA>999</nodeA></request>" -X POST http://localhost:8888/some/thing
I am getting the response from stub as expected. The problem is that the request has to be sent with XML declaration tag , e.g.
curl -d "<?xml version="1.0" encoding="UTF-8"?><request><nodeA>999</nodeA></request>" -X POST http://localhost:8888/some/thing
and in this case the request isn't matched. I tried to find smth in the documentation regarding it, but so far no luck
I found out that the problem was with curl that I used. It was malformed since I used same double quotes in XML declaration. Now I load the request body from file and everything works just fine
curl -H "Content-Type: application/xml" -d "#path_to_request_file" -X POST http://localhost:8888/some/thing

What is GraphQL query to determine if GitHub package exists within an Organization?

How do I query GitHub Packages to determine whether a package already exists? I want to prevent CI/CD from attempting to publish a Maven package if that package already exists.
I'm trying a query that looks like:
curl -X POST -H "Authorization: bearer <some token>" -H "Accept: application/vnd.github.packages-preview+json" -d <my query> https://api.github.com/graphql
where <my query> looks like:
query {
organization(login: "myorg") {
registryPackagesForQuery(packageType: MAVEN, first: 100) {
edges {
node {
name
id
}
}
}
}
}
I am getting my packages returned:
{
"data": {
"organization": {
"registryPackagesForQuery": {
"edges": [
{
"node": {
"name": "com.example.myorg",
"id": "<some id>"
}
}
]
}
}
}
}
However, there is a query argument (of type String) for registryPackagesForQuery, but I don't know how that query format should look. Any attempt to add in a value seems to result in no data being returned.
What is the correct format of the query, and how does this query vary if I want to check whether a specific version of a package exists?
The base Query docs are here: https://developer.github.com/v4/query/.
Organization: https://developer.github.com/v4/object/organization/
RegistryPackageConnection: https://developer.github.com/v4/object/registrypackageconnection
RegistryPackage: https://developer.github.com/v4/object/registrypackage/
This may be off-topic, but wouldn't it be simpler to use a maven plugin for that? Like https://github.com/chonton/exists-maven-plugin maybe ?

Why does DESCRIBE EXTENDED in Kafka KSQL return error ShowColumns not supported?

I have a simple KTABLE in KSQL called DIMAGE
When I run the following code
{
"ksql": "DESCRIBE EXTENDED DIMAGE ;"
}
I receive the following error
{
"#type": "generic_error",
"error_code": 40000,
"message": "Statement type `io.confluent.ksql.parser.tree.ShowColumns' not supported for this resource",
"stackTrace": []
}
I also receive a similar error message trying to describe a stream. I also receive the same error message if I remove the EXTENDED attribute.
You're using the wrong REST endpoint. If you use query endpoint query you'll get your error:
$ curl -s -X "POST" "http://localhost:8088/query" \
-H "Content-Type: application/vnd.ksql.v1+json; charset=utf-8" \
-d '{
"ksql": "DESCRIBE EXTENDED COMPUTER_T;"
}'
{"#type":"generic_error","error_code":40000,"message":"Statement type `io.confluent.ksql.parser.tree.ShowColumns' not supported for this resource","stackTrace":[]}⏎
If you use the statement endpoint ksql it works fine:
$ curl -s -X "POST" "http://localhost:8088/ksql" \
-H "Content-Type: application/vnd.ksql.v1+json; charset=utf-8" \
-d '{
"ksql": "DESCRIBE EXTENDED COMPUTER_T;"
}'|jq '.'
[
{
"#type": "sourceDescription",
"statementText": "DESCRIBE EXTENDED COMPUTER_T;",
"sourceDescription": {
"name": "COMPUTER_T",
"readQueries": [
{
"sinks": [
"COMP_WATCH_BY_EMP_ID_T"
],
"id": "CTAS_COMP_WATCH_BY_EMP_ID_T_0",
[...]
I've logged #2362 so that we can improve the UX of this.

How to enable Javascript in Druid

I have been using Druid for the past week and wanted to enable javascript for some postAggregations.
I think I followed the outlined steps and updated the common.runtime.properties file in ../con f/druid/_common/ to include druid.javascript.enabled=true. I then stopped the current processes and re-ran the Quickstart procedures, but it still says that JavaScript is disabled:
{
"error" : "Unknown exception",
"errorMessage" : "Instantiation of [simple type, class io.druid.query.aggregation.post.JavaScriptPostAggregator] value failed: JavaScript is disabled. (through reference chain: java.util.ArrayList[0])",
"errorClass" : "com.fasterxml.jackson.databind.JsonMappingException",
"host" : null
}
I am currently running it in the 'Quickstart' configuration - single local machine. Any pointers? Thanks!
JavaScript Query For druid Aggregation. Save the file as .body and hit the curl request.
This is a sample query for Average value.
curl -X POST "http://localhost:8082/druid/v2/?pretty" \ -H
'content-type: application/json' -d #query.body
{
"queryType":"groupBy",
"dataSource":"whirldata",
"granularity":"all",
"dimensions":[],
"aggregations":[{"name":"rows","type":"count","fieldName":"rows"},
{"name":"TargetDOS","type":"doubleSum","fieldName":"Target DOS"}],"postAggregations":[
{
"type": "javascript",
"name": "Target DOS Average",
"fieldNames": ["TargetDOS", "rows"],
"function": "function(TargetDOS, rows) { return Math.abs(TargetDOS) / rows; }"
}], "intervals":[ "2006-01-01T00:00:00.000Z/2020-01-01T00:00:00.000Z" ]}
The part you are missing is likely that the quickstart reads configs from conf-quickstart rather than conf. So try editing conf-quickstart/druid/_common/common.runtime.properties.