In rundeck (OpenSource v3.3.1 with H2SQL) is there any way to review the details of a Scheduled JOB? For us it is important to know the list of nodes where the job will be executed. I have searched in files and in H2 DB and I canĀ“t get the nodes or the filter used to schedule the job.
You can see that on the "Activity" panel, click on any execution (scheduled or not), and click on "Log Output" you can see all job detailed info. Also, you can check the execution logs, usually at /var/lib/rundeck/logs/rundeck/<projectname>/job/<job-id>/logs path.
You can check more about this here.
UPDATE 1: you can use the forecast API call to check the jobs "to be executed", I leave a script that uses it:
#!/bin/sh
#############################################
echo "Insert the Rundeck Server hostname:"; read -r rundeckServer
echo -e "\nWhich port?:"; read -r rundeckPort
echo -e "\nCopy and Paste the Rundeck API Token:"; read -r rundeckApitoken
#############################################
# to run unatended mode, please comment the code above and uncomment the code below, and replace with your props
#############################################
# rundeckServer="node01.rundeck.local"
# rundeckPort="4440"
# rundeckApitoken="P0K0EvIAoWkAbmvj3JnlelmwaVhF5AJd"
#############################################
# Options Variables
#############################################
rundeckProtocol="http"
rundeckApiversion="31"
rundeckApiformat="json"
curlOptions="-s"
#############################################
echo
echo "The following Jobs are Scheduled:"
echo
for projectName in $(curl -X "GET" "$curlOptions" -H "Accept: application/$rundeckApiformat" -H "Content-Type: application/$rundeckApiformat" -H "X-Rundeck-Auth-Token: $rundeckApitoken" "$rundeckProtocol"://"$rundeckServer":"$rundeckPort"/api/"$rundeckApiversion"/projects|sed 's/,/\n/g'| grep name | cut -d ":" -f2 | tr -d '"')
do
for jobId in $(curl "$curlOptions" -X "GET" -H "Accept: application/$rundeckApiformat" -H "Content-Type: application/$rundeckApiformat" -H "X-Rundeck-Auth-Token: $rundeckApitoken" "$rundeckProtocol"://"$rundeckServer":"$rundeckPort"/api/"$rundeckApiversion"/project/"$projectName"/jobs | sed 's/,/\n/g' | grep id | cut -d ":" -f2 | tr -d '"')
do
output=$(curl "$curlOptions" -X "GET" -H "Accept: application/$rundeckApiformat" -H "Content-Type: application/$rundeckApiformat" -H "X-Rundeck-Auth-Token: $rundeckApitoken" "$rundeckProtocol"://"$rundeckServer":"$rundeckPort"/api/"$rundeckApiversion"/job/"$jobId"/forecast?time=7d | grep "futureScheduledExecutions")
if [ -n "$output" ]
then
echo $output | sed -r 's/.+("href":".+),.+("futureScheduledExecutions":\[.+\]),.+("project":".+),("name":".+).+/\1\n\3\n\4\n\2/g'
echo
fi
done
done
UPDATE 2: I leave more simple forecast API call, (require jq).
curl --location --request GET 'http://localhost:4440/api/35/job/your-job-id/forecast' --header 'Accept: application/json' --header 'X-Rundeck-Auth-Token: your-user-token' --header 'Content-Type: application/json' --data-raw '' | jq
That prints all info:
{
"href": "http://localhost:4440/api/35/job/328789d1-986f-4aa2-9013-de8eb6b6df98",
"id": "328789d1-986f-4aa2-9013-de8eb6b6df98",
"scheduleEnabled": true,
"scheduled": true,
"enabled": true,
"permalink": "http://localhost:4440/project/ProjectEXAMPLE/job/show/328789d1-986f-4aa2-9013-de8eb6b6df98",
"group": null,
"futureScheduledExecutions": [
"2020-08-06T14:33:00Z"
],
"description": "",
"project": "ProjectEXAMPLE",
"name": "ExampleJob"
}
UPDATE 3: Now combining the export job info and forecast API Call, you can get the job info and the nodes, I leave an example script:
#!/bin/sh
#####################################################
# rundeck instance values
server="localhost"
port="4440"
api="35"
jobid="328789d1-986f-4aa2-9013-de8eb6b6df98"
token="MhloqezoV3IygvOIA6yzE9TNlzYCbi4m"
info=$(curl -s --location --request GET "http://$server:$port/api/$api/job/$jobid/forecast" --header "Accept: application/json" --header "X-Rundeck-Auth-Token:$token" --header "Content-Type: application/json" | jq)
nodes=$(curl -s --location --request GET "http://$server:$port/api/$api/job/$jobid" --header "Accept: application/xml" --header "X-Rundeck-Auth-Token:$token" --header "Content-Type: application/xml" --data-raw "" | grep -oPm1 "(?<=<filter>)[^<]+")
echo "######################"
echo "Job information: $info"
echo "######################"
echo "Nodes: $nodes"
echo "######################"
With the following data:
######################
Job information: {
"href": "http://localhost:4440/api/35/job/328789d1-986f-4aa2-9013-de8eb6b6df98",
"id": "328789d1-986f-4aa2-9013-de8eb6b6df98",
"scheduleEnabled": true,
"scheduled": true,
"enabled": true,
"permalink": "http://localhost:4440/project/ProjectEXAMPLE/job/show/328789d1-986f-4aa2-9013-de8eb6b6df98",
"group": null,
"futureScheduledExecutions": [
"2020-08-07T14:33:00Z"
],
"description": "",
"project": "ProjectEXAMPLE",
"name": "ExampleJob"
}
######################
Nodes: localhost
######################
UPDATE 4: The job definition data are "distributed" on the database (H2 or anything else). So, for example, if you configure Rundeck against MySQL and you access the database using any tool, the node filter is saved in scheduled_execution table.
Related
first thanks for your time. I create a really simple powershell code to check periodically gRPC health microservices, seven in a row every two seconds, and it is working since I got the response they are serving, the output is
{
"status": "SERVING"
}
while ($true) {grpcurl -d '{ \"service1\": \"name1\" }' -H "authorization: key key" host1:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service2\": \"name2\" }' -H "authorization: key key" host2:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service3\": \"name3\" }' -H "authorization: key key" host3:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service4\": \"name4\" }' -H "authorization: key key" host4:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service5\": \"name5\" }' -H "authorization: key key" host5:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service6\": \"name6\" }' -H "authorization: key key" host6:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service7\": \"name7\" }' -H "authorization: key key" host7:port grpc.health.v1.Health/Check;start-sleep 2}
Now I wanted to got an email notification if the output is not "SERVING" while the code keeps running in a loop, how can I achieve this?
I see two ways we can do it, the simplest way is creating a job and periodically receive it to see if the status changed:
$block = {
while ($true) {
grpcurl -d '{ \"service1\": \"name1\" }' -H "authorization: key key" host1:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service2\": \"name2\" }' -H "authorization: key key" host2:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service3\": \"name3\" }' -H "authorization: key key" host3:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service4\": \"name4\" }' -H "authorization: key key" host4:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service5\": \"name5\" }' -H "authorization: key key" host5:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service6\": \"name6\" }' -H "authorization: key key" host6:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service7\": \"name7\" }' -H "authorization: key key" host7:port grpc.health.v1.Health/Check
start-sleep 2
}
}
$job = Start-Job -Name 'gRPCMon' -ScriptBlock $block
while ($true) {
$result = Receive-Job -Id $job.Id
if (($result | ConvertFrom-Json).status -ne 'SERVING') {
## Send-MailMessage here
}
}
Or, in a more elegant way, we can run your code in a runspace and raise an event every time the condition is met.
Then, we subscribe to that event:
$block = {
while ($true) {
grpcurl -d '{ \"service1\": \"name1\" }' -H "authorization: key key" host1:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service2\": \"name2\" }' -H "authorization: key key" host2:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service3\": \"name3\" }' -H "authorization: key key" host3:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service4\": \"name4\" }' -H "authorization: key key" host4:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service5\": \"name5\" }' -H "authorization: key key" host5:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service6\": \"name6\" }' -H "authorization: key key" host6:port grpc.health.v1.Health/Check>> grpcurl -d '{ \"service7\": \"name7\" }' -H "authorization: key key" host7:port grpc.health.v1.Health/Check
start-sleep 2
if (($output | ConvertFrom-Json).status -ne 'SERVING') {
$x.Host.Runspace.Events.GenerateEvent('StatusEvent', $null, $null, "Status Event")
}
}
}
$Global:x = [hashtable]::Synchronized(#{})
$x.Host = $Host
$runspace = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace()
$runspace.Open()
$runspace.SessionStateProxy.SetVariable("x",$x)
$powershell = [System.Management.Automation.PowerShell]::Create()
$powershell.Runspace = $runspace
$powershell.AddScript($block)
$IActionResult = $powershell.BeginInvoke()
Register-EngineEvent -SourceIdentifier 'StatusEvent' -Action {
## Send-MailMessage here
}
Let me know if that did the trick.
I have been trying to convert the below cURL command to PowerShell so that I can use it on my script. I converted it but, every time I execute the command, it gives me error like in the screenshot.
cURL command: curl -X POST "https://www.hybrid-analysis.com/api/v2/quick-scan/file?_timestamp=1585901455112" -H "accept: application/json" -H "user-agent: API_KEY" -H "Content-Type: multipart/form-data" -F "scan_type=all" -F "file=#xyz.png;type=image/png" -F "no_share_third_party=false" -F "allow_community_access=false"
converted powershell command:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12;
$fileqq = Get-ChildItem -Path "C:\temp\xyz.jpg"
Invoke-WebRequest "https://www.hybrid-analysis.com/api/v2/quick-scan/file" -Headers #{'accept' = 'application/json'; 'user-agent' = 'Falcon Sandbox' ; 'Content-Type' = 'multipart/form-data' ; 'api-key' = "API_KEY"} -Method Post -Body #{"scan_type" = "all"; "file" = "$fileqq" }
powershell command error
It would be helpful for me if someone pointed out the error or correct my converted command
Thank you very much for the help.
The folowing command works in git bash but not in cmd and powershell
curl -X POST http://localhost:5678/api/findgen -H 'Content-Type: application/json' -d '{"a": "Val 1","b": "Val 2","c": "Val 3","d": "Val 4"}' -o "file.json"
I get error in cmd such as -
curl: (6) Could not resolve host: application
curl: (6) Could not resolve host: Val 1,b
curl: (6) Could not resolve host: Val 2,c
curl: (6) Could not resolve host: Val 3,d
curl: (3) [globbing] unmatched close brace/bracket in column 6
What can be the issue?
Just read the error message:
Invoke-WebRequest Cannot bind parameter 'Headers'. Cannot convert the "Content-Type:
application/json" value of type "System.String" to type
"System.Collections.IDictionary".
In PowerShell, curl is an alias for the Invoke-WebRequest cmdlet. As the error points it out, the Header parameter must be a IDictionary, not a string. This is how it looks like in PowerShell:
#{"Content-Type"= "application/json"}
Some parameters are also different. This is how I would script the request:
Invoke-WebRequest `
-Uri "http://localhost:5678/api/findgen" `
-Headers #{"Content-Type"= "application/json"} `
-Body '{"a": "Val 1","b": "Val 2","c": "Val 3","d": "Val 4"}' `
-OutFile "file.json" `
-Method Post
So here is the adapted syntax as a one liner :
curl.exe "-X" "POST" "https://reqbin.com/echo/post/json" `
"-H" "Content-Type: application/json" `
"-d" '{\"a\": \"Val 1\",\"b\": \"Val 2\",\"c\": \"Val 3\",\"d\": \"Val 4\"}' `
"-o" "file.json"
And a way a little more friendly for JSON :
curl.exe "-X" "POST" "https://reqbin.com/echo/post/json" `
"-H" "Content-Type: application/json" `
"-d" (#{a='Val 1';b='val 2';c='Val 3';d='Val 4'} | ConvertTo-Json -Compress) `
"-o" "file.json"
I see those "Could not resolve host" only with C:\Windows\System32\curl.exe on Windows 10 (build 17063 or later), because of the way the Windows curl.exe native program interprets single and double quotes.
If you inverse/escape those quotes, it does work (on CMD)
C:\Windows\System32\curl -X POST http://localhost:5678/api/findgen \
-H "Content-Type: application/json" \
^^^
double-quotes
-d "{\"a\": \"Val 1\",\"b\": \"Val 2\",\"c\": \"Val 3\",\"d\": \"Val 4\"}" -o "file.json"
^^^
double quotes outside, escaped double-quotes inside: valid JSON
This is similar to "How do I POST JSON with Curl?"
On Powershell, I had to escape double quotes (backtick, as commented), and escape {}: {{}}
PS D:\> C:\Windows\System32\curl -X POST https://reqbin.com/echo/post/json `
-H "Content-Type: application/json" `
-d "{{`"login`":`"my_login`",`"password`":`"my_password`"}}"
Alternatively:
PS D:\> C:\Windows\System32\curl.exe -X POST https://reqbin.com/echo/post/json `
>> -H "Content-Type: application/json" `
>> -d '{{""login"":""my_login"",""password"":""my_password""}}'
{"success":"true"}
From Steven's post:
PS D:\> C:\Windows\System32\curl.exe -X POST -H "Content-Type: application/json" `
-d '{""north"": ""each south""}' https://reqbin.com/echo/post/json
{"success":"true"}
Using curl.exe alone means using the MingW Git for Windows curl.exe
PS D:\> curl.exe -X POST -H "Content-Type: application/json" `
>> -d '{""north"": ""each south""}' https://reqbin.com/echo/post/json
curl: (6) Could not resolve host: each
curl: (3) unmatched close brace/bracket in URL position 6:
south} https://reqbin.com/echo/post/json
You therefore need to escape curly brackets, and remove any space:
PS D:\> curl.exe -X POST -H "Content-Type: application/json" `
>> -d '{{"north":"each south"}}' https://reqbin.com/echo/post/json
{"success":"true"}
Or, using spaces, with single quotes:
PS D:\> curl.exe -X POST -H "Content-Type: application/json" `
>> -d '{\"north\": \"each south\"}' https://reqbin.com/echo/post/json
{"success":"true"}
On CMD If you want to keep those quotes unchanged, used the mingw curl.exe packaged with Git For Windows, at least on CMD.
C:\path\to\git\mingw64\bin\curl.exe -X POST http://localhost:5678/api/findgen \
-H 'Content-Type: application/json' \
-d '{"a": "Val 1","b": "Val 2","c": "Val 3","d": "Val 4"}' -o "file.json"
^^^
You can keep the quotes as-is
As commented, in a Powershell session, curl itself is an alias for Invoke-WebRequest.
I tested it with:
PS D:\> Get-Alias -Name curl
CommandType Name Version Source
----------- ---- ------- ------
Alias curl -> Invoke-WebRequest
PS D:\> curl -uri https://reqbin.com/echo/post/json `
>> -Method 'POST' -ContentType "application/json" `
>> -Body "{""login"":""my_login"",""password"":""my_password""}"
StatusCode : 200
StatusDescription : OK
Content : {"success":"true"}
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
CF-Cache-Status: DYNAMIC
cf-request-id: 0917a97460000053c82aa69000000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/bea...
Forms : {}
Headers : {[Connection, keep-alive], [CF-Cache-Status, DYNAMIC], [cf-request-id, 0917a97460000053c82aa69000000001], [Expect-CT, max-age=604800,
report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml : System.__ComObject
RawContentLength : 19
I've been trying (and failing for hours) to convert this cURL script into PowerShell:
curl -X POST \
https://example.net \
-H 'Authorization: Bearer 1234567890' \
-H 'Content-Type: application/json' \
-H 'cache-control: no-cache' \
-d '{
"data": {
"MID": 33,
"DID": "66666666",
"CID": 10002,
"status": "ACTIVE",
"HID": "11"
}
}'
My PowerShell script looks like this - I suspect it's something to do with the double hash table I've created in $Body but I am really at a loss.:
.. script snipped for simplicity
$Body = #{
'data'= #{
'MID'= 33;
'DID'= "66666666";
'CID'=10002;
'status'="ACTIVE";
'HID'= "11"
}
}
$CurlArgument = '-X', 'POST',
'https://example.net',
'-H', 'Authorization: Bearer 1234567890',
'-H', 'Content-Type: application/json',
'-d',
$Body
$CURLEXE = 'C:\Windows\System32\curl.exe'
& $CURLEXE #CurlArgument
I get the following error message when it executes:
..."message":"Access not allowed","details":{"5011":"A JSONObject text must begin with '{' at 1 [character 2 line 1]"}}]}
I experimented with adding after $Body:
| ConvertTo-Json
but that then gives me this error:
Unexpected character ('d' (code 100)): was expecting double-quote to start field name
My $CurlArgument variable looks like this (which looks the same as the first cURL script):
-X
POST
https://example.net
-H
Authorization: Bearer 1234567890
-H
Content-Type: application/json
-d
{
"data": {
"CID": 10002,
"MID": 33,
"HID": "11",
"DID": "66666666",
"status": "ACTIVE"
}
}
As always, assistance would be greatly appreciated.
Your $Body variable contains a hashtable (#{ ... }), so you must explicitly convert it to JSON with ConvertTo-Json.
Additionally, because you're calling an external program, you must escape the " chars. in the JSON string as \"[1]:
$CurlArgument = '-X', 'POST',
'https://example.net',
'-H', 'Authorization: Bearer 1234567890',
'-H', 'Content-Type: application/json',
'-d',
(($Body | ConvertTo-Json) -replace '"', '\"')
[1] This additional escaping requirement is highly unfortunate, but to date it has been kept around for the sake of backward compatibility.
This GitHub docs issue tells the whole story.
I'm new to Elasticsearch and I'm following the tutorial loading sample data: https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html
When I try to bulk insert the data in powershell
'curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary "#shakespeare.json"'
I get the following error
PS C:\Development\elasticsearch-5.4.1\importdata> curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary "#shakespeare.json"
Invoke-WebRequest : Cannot bind parameter 'Headers'. Cannot convert the "Content-Type: application/x-ndjson" value of type "System.St
ring" to type "System.Collections.IDictionary".
At line:1 char:9
+ curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/s ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I've tried several things, read the bulk api documentation but with no result
Here're some examples of the content of the json file:
{"index":{"_index":"shakespeare","_type":"act","_id":0}}
{"line_id":1,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}
{"index":{"_index":"shakespeare","_type":"scene","_id":1}}
{"line_id":2,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"SCENE I. London. The palace."}
{"index":{"_index":"shakespeare","_type":"line","_id":2}}
{"line_id":3,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"Enter KING HENRY, LORD JOHN OF LANCASTER, the EARL of WESTMORELAND, SIR WALTER BLUNT, and others"}
I found the solution. Because I used curl on windows and with powershell, I had to translate
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary #shakespeare.json
to
PS C:\Development\elasticsearch-5.4.1\importdata> $postParams = #"
>> {"index":{"_index":"shakespeare","_type":"act","_id":0}}
>> {"line_id":1,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}
>>
>> "#
PS C:\Development\elasticsearch-5.4.1\importdata> curl -H #{"Content-Type" = "application/x-ndjson"} -Method POST 'http://localhost:92
00/shakespeare/_bulk?pretty' -body $postParams
When I use the demo file, the bulk insert didn't succeeded but that's another problem
PS C:\Development\elasticsearch-5.4.1\importdata> $postParams = Get-Content "shakespeare.json"