In RunDeck in the bash script I am triggering a Linux error to cause job execution failure
CODE=`echo $RESULT | head -n1 | cut -c2`
if [ $CODE != '0' ];
then
exit 1
fi
Strange thing is that I am getting four records(failed) instead of one under the activity tab
Next I tried to add sleep 20 right above exit 1 and it logged one record only. As I see some processes still running in RunDeck after exit 1 command and it continues adding rows in activities. Any Idea how to wait until everything is finished to add only one row under the activity tab?
EDIT
Here is full definition
#!/bin/bash
set –e
URL="https://my_url"
MY_COMMAND="command_name"
RAW_DATA="[null,\"${MY_COMMAND}\",{}]"
echo "Raw Data ${RAW_DATA}"
RESULT=$(curl "$URL/${MY_COMMAND}" \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'User-Agent: my_user_agent' \
--data-raw $RAW_DATA)
echo $RESULT
CODE=`echo $RESULT | head -n1 | cut -c2`
if [ $CODE != '0' ];
then
exit 1
fi
EDIT 2 - Added Job Definition(YAML)
- defaultTab: output
description: JOB_DESCRIPTION
executionEnabled: true
id: 125cb755-3eaa-49f8-a8b8-dc1004238a44
loglevel: INFO
loglimit: 10MB
loglimitAction: truncate
loglimitStatus: failed
name: JOB_NAME
nodeFilterEditable: false
notification:
onfailure:
email:
attachLog: true
attachLogInFile: true
recipients: someone#mail.com
notifyAvgDurationThreshold: null
plugins:
ExecutionLifecycle: {}
retry: '3'
schedule:
dayofmonth:
day: '*'
month: '*'
time:
hour: '04'
minute: '05'
seconds: '0'
year: '*'
scheduleEnabled: true
schedules: []
sequence:
commands:
- script: "#!/bin/bash\n\n"
keepgoing: false
strategy: node-first
timeout: 1h
uuid: 125cb755-3eaa-49f8-a8b8-dc1004238a44
Your job is configured to retry 3 times in case of error (the original failed execution + 3 retries = 4 failed executions), you can change this behavior by editing your job > Go to the "Other" tab > Retry textbox. More info here.
Related
I am writing a script so that to filter jobs that are failed in past 24hrs and also the failed job is not completed or not in running state again.
rm -rf jobsfailed.txt jobscompleted.txt jobsnotcompleted.txt ## Remove the files which are created
export PGPASSWORD="$PG_PASS"
failed_jobs=`psql -t -U bacula bacula << EOD
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE jobstatus IN ('f','A','E') AND realendtime > NOW() - INTERVAL '24 HOURS';
\q
EOD` ### Collect all the jobs which are in the defined states
echo "$failed_jobs" >> jobsfailed.txt ### redirect the values to a file
sortedlist=$(awk -F'|' '{print $3}' jobsfailed.txt | sort | uniq) ## sort the values based on the jobname
for i in $sortedlist
do
retVal=$?
jobs_notcompleted=`psql -t -U bacula bacula << EOD1
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE name LIKE '$i' AND jobstatus IN ('T','R') AND starttime > NOW() - INTERVAL '24 HOURS' ORDER BY jobid DESC LIMIT 1;
\q
EOD1` ### If the job is in above defined states(T or R) then jobs completed successfully. Any other state apart from above then job not completed
if [[ $retVal -eq 0 ]]; then
echo "$jobs_notcompleted" >> jobscompleted.txt
else
echo "$jobs_notcompleted" >> jobsnotcompleted.txt
fi
exit $retVal
done
But i am not getting desired output. Since if no state is getting matched, then it is producing (0 rows) output. Please let me know, is there any other way if 0 rows are matched then that $jobs_notcompleted value should redirect to the jobsnotcompleted.txt file. jobscompleted.txt file is getting created and working as expected.
For some reason, I feel that you need a "NOT" between "jobstatus" and "in" for your second query, because the ${retVal} would, in my mind, always return 0 unless there was a DB error.
If I am wrong on that point, then the "retVal=$?" needs to be moved to after the 2nd query.
This is my ansible script
- name: show1
debug:
msg: "{{response.stderr_lines}}"
Here is the output
msg:
- Using endpoint [https://us-central1-aiplatform.googleapis.com/]
- CustomJob [projects/123456/locations/us-central1/customJobs/112233445566] is submitted successfully.
- ''
- Your job is still active. You may view the status of your job with the command
- ''
- ' $ gcloud ai custom-jobs describe projects/123456/locations/us-central1/customJobs/112233445566'
- ''
- or continue streaming the logs with the command
- ''
- ' $ gcloud ai custom-jobs stream-logs projects/123456/locations/us-central1/customJobs/112233445566'
Here I want to extract custom Job ID which is 112233445566
I used the select module like below
- name: show
debug:
msg: "{{train_custom_image_unmanaged_response.stderr_lines | select('search', 'describe') | list }}"
and it gives me this output
msg:
- ' $ gcloud ai custom-jobs describe projects/123456/locations/us-central1/customJobs/112233445566'
But I just want the job id as specified above. Any idea about that ?
Thanks.
You selected the line you are interested in. From that now you want to isolate the job id number in the end. You can do that using a regular expression like so:
- set_fact:
line: "{{train_custom_image_unmanaged_response.stderr_lines | select('search', 'describe') | list }}"
- debug:
msg: "{{ line | regex_search('.*/customJobs/(\\d+)', '\\1') }}"
This will give you all the digits in the end of the line after /customJobs/. See https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#searching-strings-with-regular-expressions
Below is the playbook
---
- name: stop agent process
shell: "ps -ef | grep -v grep | grep -w {{ MONGODB_AGENT_PROCESS }} | awk '{print $2}'"
register: running_agent_processes
- name: stop mongod process
shell: "ps -ef | grep -v grep | grep -w {{ MONGODB_SERVER_PROCESS }} | awk '{print $2}'"
register: running_mongod_processes
- name: combine processes
set_fact:
all_processes: "{{ running_agent_processes.stdout_lines + running_mongod_processes.stdout_lines }}"
- name: Kill all processes
shell: "kill {{ item }}"
with_items: "{{ all_processes }}"
when: ansible_facts[ansible_hostname] != primary
- wait_for:
path: "/proc/{{ item }}/status"
state: absent
with_items: "{{ all_processes }}"
ignore_errors: yes
register: killed_processes
when: ansible_facts[ansible_hostname] != primary
- name: Force kill stuck processes
shell: "kill -9 {{ item }}"
with_items: "{{ killed_processes.results | select('failed') | map(attribute='item') | list }}"
when: ansible_facts[ansible_hostname] != primary
I have stored a fact called "primary" which stores the primary of a mongodb replica set in a previous step in the playbook.
I just want to compare the ansible_facts[ansible_hostname] with my primary fact. If they are not equal, I would like to kill processes.
The error I am getting is below:
fatal: [lpdkubpoc01d.phx.aexp.com]: FAILED! => {"msg": "The
conditional check 'ansible_facts[ansible_hostname] != primary' failed.
The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"} fatal: [lpdkubpoc01c.phx.aexp.com]: FAILED! =>
{"msg": "The conditional check 'ansible_facts[ansible_hostname] !=
primary' failed. The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"} fatal: [lpdkubpoc01e.phx.aexp.com]: FAILED! =>
{"msg": "The conditional check 'ansible_facts[ansible_hostname] !=
primary' failed. The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"}
Can someone help me with comparing an ansible_fact with a set_fact fact?
You can comparing use directly ansible facts without write ansible_facts before. Just use as when: ansible_hostname != primary
This is the piece of my playbook file for the question:
- name: Initialize the Kubernetes cluster using kubeadm
command: kubeadm init --config /etc/kubernetes/kubeadminit.yaml
register: init_output
- name: Copy join command to local file
local_action: copy content={{ init_output.stdout }} dest="./join-command"
Currently join-command contains the entire stdout (30+ lines of text) for content. What I want to grab is just the last two lines of init_output.stdout instead of the entire output. I've looked into using index reference (ie. init_output.stdout[#]) but I don't know that the output will always be the same length and I don't know how to use indexes to grab more than one line, but i'm fairly certain that the last two lines will always be the join command. Any suggestions?
Select last 2 lines from the list stdout_lines
- local_action: copy content={{ init_output.stdout_lines[-2:] }} dest="./join-command"
It's possible to format the lines in a block. For example
- local_action:
module: copy
content: |
{{ init_output.stdout_lines[-2] }}
{{ init_output.stdout_lines[-1] }}
dest: "./join-command"
To append the lines in a loop try
- local_action:
module: lineinfile
path: "./join-command"
line: "{{ item }}"
insertafter: EOF
create: true
loop: "{{ init_output.stdout_lines[-2:] }}"
I encountered this kind of issue and did not want to copy the join command to a local file so I did a set_fact instead this way:
- set_fact:
join_cmd: '{{ init_output.stdout_lines[-2][:-2] }}{{ init_output.stdout_lines[-1] }}'
I did this...
- name: kubeadm init
shell: |
kubeadm init --control-plane-endpoint \
localhost \
--control-plane-endpoint kube-api.local >> /tmp/run_kube_init.sh
when: master == "yes"
- name: Get join from master
fetch:
src: "/tmp/run_kube_init.sh"
dest: "/tmp/run_kube_init.sh"
flat: yes
when: ansible_hostname == 'k-master'
- name: Add join file to nodes
copy:
src: "/tmp/run_kube_init.sh"
dest: "/tmp/run_kube_init.sh"
when: master == "no"
- name: Extract join token for nodes
shell: tail -n +2 /tmp/run_kube_init.sh | head -n -1 | awk '{print $5}' | tail -n 1
register: JOIN_TOKEN
- set_fact:
join_token: "{{ JOIN_TOKEN.stdout }}"
- name: join nodes
shell: |
kubeadm join kube-api.local:6443 \
--token {{ JOIN_TOKEN.stdout }} \
--discovery-token-unsafe-skip-ca-verification
when: master == "no"
- name: rm /tmp/run_kube_init.sh
ansible.builtin.file:
path: /tmp/run_kube_init.sh
state: absent
I have a PostgreSQL query as below which is running fine . I am calling it from a shell script as below
Result=$(psql -U username -d database -t -c
$'SELECT round(sum(i.total), 2) AS "ROUND(sum(i.total), 2)"
FROM invoice i
WHERE i.create_datetime = '2019-03-01 00:00:00-06'
AND i.is_review = '1' AND i.user_id != 60;')
now I want the value which I have hard coded as i.create_datetime = '2019-03-01 00:00:00-06' to replace it with a variable date value.
I have tried two ways
way 1:
Result=$(psql -U username -d database -t -c
$'WITH var(reviewMonth) as (values(\'$reviewMonth\'))
SELECT round(sum(i.total),2) AS "ROUND(sum(i.total),2)"
FROM var,invoice i
WHERE i.create_datetime = var.reviewMonth::timestamp
AND i.is_review = \'1\' AND i.user_id != 60;')
and
way 2:
Result=$(psql -U username -d database -t -c
$'SELECT round(sum(i.total),2) AS "ROUND(sum(i.total),2)"
FROM invoice i
WHERE i.create_datetime = \'$reviewMonth\'
AND i.is_review = \'1\' AND i.user_id != 60;')
But both way it's throwing error
way 1 throwing error as :
ERROR: operator does not exist: timestamp with time zone = text
way 2 throwing error as :
ERROR: invalid input syntax for type timestamp with time zone: "$reviewMonth"
Please suggest what should be my approach.
You should try using the psql variables. Here's an example:
# Put the query in a file, with the variable TSTAMP:
> echo "SELECT :'TSTAMP'::timestamp with time zone;" > query.sql
> export TSTAMP='2019-03-01 00:00:00-06'
> RESULT=$(psql -U postgres -t --variable=TSTAMP="$TSTAMP" -f query.sql )
> echo $RESULT
2019-03-01 06:00:00+00
Note how we format the string literal substitution in the query: :'TSTAMP'
You could also do the substitution yourself. Here's an example using a heredoc:
> export TSTAMP='2019-03-01 00:00:01-06'
> RESULT=$(psql -U postgres -t << EOF
SELECT '$TSTAMP'::timestamp with time zone;
EOF
)
> echo $RESULT
2019-03-01 06:00:01+00
In this case, we aren't using psql's variable substitution, so we have to quote the variable like '$TSTAMP' . Using a heredoc makes the quoting much simpler than using -c because you aren't trying to quote the whole command.
EDIT: more examples because it appears this wasn't clear enough. TSTAMP does not have to be hard coded, it's just a bash variable than can be set like any other bash variable.
> TSTAMP=$(date -d 'now' +'%Y-%m-01 00:00:00')
> RESULT=$(psql -U postgres -t << EOF
SELECT '$TSTAMP'::timestamp with time zone;
EOF
)
> echo $RESULT
2019-06-01 00:00:00+00
However, if you're really just looking for the start of the month, there's no need for shell variables at all
> RESULT=$(psql -U postgres -t << EOF
SELECT date_trunc('month', now());
EOF
)
> echo $RESULT
2019-06-01 00:00:00+00