Druid Spec to Truncate and Load in druid - druid

Could some please share me the sample spec to truncate and Load the data into druid.
I want to load the data into druid from csv file but the requirement is that the data should be truncate in the druid and then load.

Below command will work:
curl -k --noproxy "*" -X 'POST' -H 'Content-Type:application/json' -H "Authorization: Basic $(echo -n <druid username>:<password> | base64)" -d '{ "interval" : "<start interval>/<end interval>" }' https://<hostname>:<portno.>/druid/coordinator/v1/datasources/<datsource name>/markUnused
or
curl -k --noproxy "*" -X 'DELETE' -H 'Content-Type:application/json' {<druid_ds_url>}/{<DatasourceName>}

Related

Cloudflare DDNS repeat sh script using API v4, with multiple A record in a single sh script, but fail

I am trying to use Cloudflare API v4 to setup DDNS on my server. But I am new in scripting .sh file. I hope to update multiple DNS records in a single .sh file.
I got a script from the internet (script1.sh):
NEW_IP=`curl -s http://ipv4.icanhazip.com`
CURRENT_IP=`cat /Users/foo/Desktop/cloudflare/current_ip.txt`
if [ "$NEW_IP" = "$CURRENT_IP" ]
then
echo "No Change in IP Adddress"
else
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id}/dns_records/{dns_record_id}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"{domain_name}","content":"'$NEW_IP'","ttl":1,"proxied":true}'
echo $NEW_IP > /Users/foo/Desktop/cloudflare/current_ip.txt
fi
The above script is work fine for single DNS record update instead of multiple record update like below (script2.sh):
NEW_IP=`curl -s http://ipv4.icanhazip.com`
CURRENT_IP=`cat /Users/foo/Desktop/cloudflare/current_ip.txt`
if [ "$NEW_IP" = "$CURRENT_IP" ]
then
echo "No Change in IP Adddress"
else
#domain-one
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_one}/dns_records/{dns_record_id_for_domain_one_record_one}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"domain-one.com","content":"'$NEW_IP'","ttl":1,"proxied":true}'
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_one}/dns_records/{dns_record_id_for_domain_one_record_two}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"subdomain.domain-one.com","content":"'$NEW_IP'","ttl":1,"proxied":true}'
#domain-two
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_two}/dns_records/{dns_record_id_for_domain_two_record}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"domain-two.com","content":"'$NEW_IP'","ttl":1,"proxied":true}'
echo $NEW_IP > /Users/foo/Desktop/cloudflare/current_ip.txt
fi
Can you please help to explain and solve the problem? Please tell me what's wrong in the script. Thanks!
[edit] I run it once by sh /Users/foo/Desktop/script-name.sh, for the first example (script1.sh), it is ok; for second example (script2.sh), return -bash: fork: Resource temporarily unavailable. As I use automatic run script like cron, it is same result.
Every curl request done, needed to add & or && to continue the following function.
More explanation here.
NEW_IP=`curl -s http://ipv4.icanhazip.com`
CURRENT_IP=`cat /Users/foo/Desktop/cloudflare/current_ip.txt`
if [ "$NEW_IP" = "$CURRENT_IP" ]
then
echo "No Change in IP Adddress"
else
#domain-one
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_one}/dns_records/{dns_record_id_for_domain_one_record_one}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"domain-one.com","content":"'$NEW_IP'","ttl":1,"proxied":true}' &
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_one}/dns_records/{dns_record_id_for_domain_one_record_two}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"subdomain.domain-one.com","content":"'$NEW_IP'","ttl":1,"proxied":true}' &
#domain-two
curl -X PUT "https://api.cloudflare.com/client/v4/zones/{zone_id_for_domain_two}/dns_records/{dns_record_id_for_domain_two_record}" \
-H "X-Auth-Email: {my_email}" \
-H "X-Auth-Key: {global_api_key}" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"domain-two.com","content":"'$NEW_IP'","ttl":1,"proxied":true}'
echo $NEW_IP > /Users/foo/Desktop/cloudflare/current_ip.txt
fi
After going through many posts on this topic I've ended up taking parts from a number posts and settled for the below.
The below will loop through the entries in the array "dnsrecords" and update each with the machines current external address.
Have had a little more time to look into this, I wanted it to run it on my NGINX server as a cronjob and update multiple records, some to be proxied and others not.
Still want to make some of the below into functions, but for now it is working as I need it to.
And it also logs onto a "log.log" file on every run.
#!/usr/bin/bash
## Cloudflare authentication details
## Keep these private
cloudflare_auth_email=Your_Email
cloudflare_auth_key="Your_API_Key"
zoneid="Your_Zone_ID"
## Cloudflare Proxied DNS Records as Array
dnsrecords_proxied=(
"domain.com"
"www.domain.com"
"sub1.domain.com"
"sub2.domain.com"
"sub3.domain.com"
"sub3.domain.com"
)
## Cloudflare Non-Proxied DNS Records as Array
dnsrecords_not_proxied=(
"vpn01.domain.com"
"vpn02.domain.com"
)
## Files to log to (replace "path/to/" with script path)
log=path/to/log.log
log_ip=path/to/previous_ip
## Getting Date/Time
dt=$(date '+%d/%m/%Y %H:%M:%S')
## Get old IP from file
old_ip=$(cat $log_ip)
## Get the current external IP address
ip=$(curl -s -X GET https://api.ipify.org)
#echo "Current IP is $ip"
## Checking if IP changed since last update
if [ $ip = $old_ip ]; then
echo -en "$dt - Previous IP:$old_ip\n$dt - Current IP:$ip\n$dt - No Changes Required....\n" >> $log
echo "$(tail -n 1000 $log)" > $log
exit
## Exit if IP has not changed
else
## If the IP changed, not match the one on file "previous_ip"
echo -en "$dt - Previous IP:$old_ip\n$dt - Current IP:$ip\n$dt - Starting Updates....\n" >> $log
## Processing Proxied DNS Records
for dnsrecord in "${dnsrecords_proxied[#]}"
do
## For each DNS Record in Array "dnsrecords"
# Getting the DNS Record ID
dnsrecordid=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records?type=A&name=$dnsrecord" \
-H "X-Auth-Email: $cloudflare_auth_email" \
-H "Authorization: Bearer $cloudflare_auth_key" \
-H "Content-Type: application/json" | jq -r '{"result"}[] | .[0] | .id') &&
# Updating the DNS Record
curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records/$dnsrecordid" \
-H "X-Auth-Email: $cloudflare_auth_email" \
-H "Authorization: Bearer $cloudflare_auth_key" \
-H "Content-Type: application/json" \
--data "{\"type\":\"A\",\"name\":\"$dnsrecord\",\"content\":\"$ip\",\"ttl\":1,\"proxied\":true}" | jq
echo -en "$dt - Updated - $dnsrecord \n" >> $log
done
## Processing Non Proxied DNS Records
for dnsrecord in "${dnsrecords_not_proxied[#]}"
do
## For each DNS Record in Array "dnsrecords"
# Getting the DNS Record ID
dnsrecordid=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records?type=A&name=$dnsrecord" \
-H "X-Auth-Email: $cloudflare_auth_email" \
-H "Authorization: Bearer $cloudflare_auth_key" \
-H "Content-Type: application/json" | jq -r '{"result"}[] | .[0] | .id') &&
# Updating the DNS Record
curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records/$dnsrecordid" \
-H "X-Auth-Email: $cloudflare_auth_email" \
-H "Authorization: Bearer $cloudflare_auth_key" \
-H "Content-Type: application/json" \
--data "{\"type\":\"A\",\"name\":\"$dnsrecord\",\"content\":\"$ip\",\"ttl\":1,\"proxied\":false}" | jq
echo -en "$dt - Updated - $dnsrecord \n" >> $log
done
echo $ip > $log_ip
echo -en "$dt - Updates Completed.... \n" >> $log
echo "$(tail -n 1000 $log)" > $log
fi

How to delete grafana dashboard using curl command?

I am trying to delete grafana dashboard using curl command, but it is not deleting, the below curl command I tried to delete the dashboard,
curl -XPOST -d '{"name":"mydashboard"}' http://localhost:3000/api/dashboards/db -u admin:admin -H "Content-Type: application/json"
Run the below command,
curl -k -X DELETE -u admin:admin http://localhost:3000/api/dashboards/db/dashboardslug_name
where dashboardslug_name is the name while save the dashboard and not the dashboard title displaying in the web url.

POST request to google cloud storage using Curl

In the following Curl command, content type and content length and access bearer are attached to my bucket URI to upload a file to google cloud storage.
C:\softwares\curl>curl -X POST -H "Content-Type:application/json" \
-H "Content-Length:100" \
-H "Authorization: Bearer <MY_OAUTH2_TOKEN>" \
"https://www.googleapis.com/upload/storage/v1/b/kids-74096.appspot.com/o?uploadType=media&name=newcurl" \
-d '{"text":"something"}'
But I am getting this error:
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
Make sure to include a Content-Type header, and ensure that your Content-Length matches the length of the data you're sending. Here's an example:
curl -k -v -X POST \
-H "Authorization: Bearer <your_oauth2_token>" -H "Content-Length: 8" \
-H "Content-Type: text/plain" \
'https://www.googleapis.com/upload/storage/v1/b/your-bucket/o?uploadType=media&name=yourobjectname' \
-d 'yourdata'
As suggested in the comments, you might find it easier to use gsutil or the Cloud Storage client libraries to accomplish this.

curl calling REST API response into a file

I am using curl to call a REST API. The REST call returns a file mar21.tar.gz
But the format of the file is data but it has to be in gzip format for me to run tar -xvf on it.
How can I get the file to save in the original format in which it is returned.
curl -u #{user}:#{password} -k -i -H "Content-type: application/json" -o #{tmp_dir}/#{filename} -X GET #{url}
Try this:
curl -u #{user}:#{password} -k -i -H "Content-type: application/json" -o #{tmp_dir}/#{filename} -X GET #{url} > mar21.tar.gz

How to create a connector-connection-pool in Glassfish 3.1.2 with REST interface

For a remote administration of multiple Glassfish 3.1.2.2 instances I want to configure resource adapter connection pool and connector resources. These configuration can only be done after the resource adapter deployment.
All works if I do things with asadmin.
Get Access via REST works as expected.
Example:
curl --user admin:pwd -X GET -H "Accept: application/JSON"
http://localhost:4848/management/domain/resources/connector-connection-pool
Now I want to create a connection pool using the following command with REST
asadmin create-connector-connection-pool --raname MulticastDNS-connector
--connectiondefinition multicastdns.outbound.MulticastDNSRegistry multicastdns/pool
I followed some of the Oracle examples http://docs.oracle.com/cd/E18930_01/html/821-2416/gjipx.html#gjijx or http://docs.oracle.com/cd/E19798-01/821-1751/gjijx/index.html
But all things I tried got 400 Bad Requests.
Example:
curl --user admin:pwd -X POST -H "Accept: application/JSON" -H "Content-Type: application/json"
-d '{"id":"multicastdn%2fspool","connectiondefinitionname":"multicastdns.outbound.MulticastDNSRegistry","resourceAdapterName":"MulticastDNS-connector"}'\
http://localhost:4848/management/domain/resources/connector-connection-pool -v
# other check
curl --user admin:pwd -X POST -H "Accept: application/JSON" \
-d id=multicastdns%2fpool \
-d connectiondefinitionname=multicastdns.outbound.MulticastDNSRegistry \
-d resourceAdapterName=MulticastDNS-connector \
http://localhost:4848/management/domain/resources/connector-connection-pool -v
Simular results if I want to delete a existing resource.
curl --user admin:pwd -X DELETE -H "Accept: application/JSON" \
http://localhost:4848/management/domain/resources/connector-connection-pool/multicastdns%2fpool -v
# other try
curl --user admin:pwd -X DELETE -H "Accept: application/JSON" \
-d id=multicastdns%2fpool \
http://localhost:4848/management/domain/resources/connector-connection-pool -v
Same issue if try the second step with connector resource
asadmin create-connector-resource --poolname multicastdns/pool jca/multicastdns
Get via REST works but add and delete an entry won't work.
Thanx florian