This is my ansible script
- name: show1
debug:
msg: "{{response.stderr_lines}}"
Here is the output
msg:
- Using endpoint [https://us-central1-aiplatform.googleapis.com/]
- CustomJob [projects/123456/locations/us-central1/customJobs/112233445566] is submitted successfully.
- ''
- Your job is still active. You may view the status of your job with the command
- ''
- ' $ gcloud ai custom-jobs describe projects/123456/locations/us-central1/customJobs/112233445566'
- ''
- or continue streaming the logs with the command
- ''
- ' $ gcloud ai custom-jobs stream-logs projects/123456/locations/us-central1/customJobs/112233445566'
Here I want to extract custom Job ID which is 112233445566
I used the select module like below
- name: show
debug:
msg: "{{train_custom_image_unmanaged_response.stderr_lines | select('search', 'describe') | list }}"
and it gives me this output
msg:
- ' $ gcloud ai custom-jobs describe projects/123456/locations/us-central1/customJobs/112233445566'
But I just want the job id as specified above. Any idea about that ?
Thanks.
You selected the line you are interested in. From that now you want to isolate the job id number in the end. You can do that using a regular expression like so:
- set_fact:
line: "{{train_custom_image_unmanaged_response.stderr_lines | select('search', 'describe') | list }}"
- debug:
msg: "{{ line | regex_search('.*/customJobs/(\\d+)', '\\1') }}"
This will give you all the digits in the end of the line after /customJobs/. See https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#searching-strings-with-regular-expressions
Related
In RunDeck in the bash script I am triggering a Linux error to cause job execution failure
CODE=`echo $RESULT | head -n1 | cut -c2`
if [ $CODE != '0' ];
then
exit 1
fi
Strange thing is that I am getting four records(failed) instead of one under the activity tab
Next I tried to add sleep 20 right above exit 1 and it logged one record only. As I see some processes still running in RunDeck after exit 1 command and it continues adding rows in activities. Any Idea how to wait until everything is finished to add only one row under the activity tab?
EDIT
Here is full definition
#!/bin/bash
set –e
URL="https://my_url"
MY_COMMAND="command_name"
RAW_DATA="[null,\"${MY_COMMAND}\",{}]"
echo "Raw Data ${RAW_DATA}"
RESULT=$(curl "$URL/${MY_COMMAND}" \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'User-Agent: my_user_agent' \
--data-raw $RAW_DATA)
echo $RESULT
CODE=`echo $RESULT | head -n1 | cut -c2`
if [ $CODE != '0' ];
then
exit 1
fi
EDIT 2 - Added Job Definition(YAML)
- defaultTab: output
description: JOB_DESCRIPTION
executionEnabled: true
id: 125cb755-3eaa-49f8-a8b8-dc1004238a44
loglevel: INFO
loglimit: 10MB
loglimitAction: truncate
loglimitStatus: failed
name: JOB_NAME
nodeFilterEditable: false
notification:
onfailure:
email:
attachLog: true
attachLogInFile: true
recipients: someone#mail.com
notifyAvgDurationThreshold: null
plugins:
ExecutionLifecycle: {}
retry: '3'
schedule:
dayofmonth:
day: '*'
month: '*'
time:
hour: '04'
minute: '05'
seconds: '0'
year: '*'
scheduleEnabled: true
schedules: []
sequence:
commands:
- script: "#!/bin/bash\n\n"
keepgoing: false
strategy: node-first
timeout: 1h
uuid: 125cb755-3eaa-49f8-a8b8-dc1004238a44
Your job is configured to retry 3 times in case of error (the original failed execution + 3 retries = 4 failed executions), you can change this behavior by editing your job > Go to the "Other" tab > Retry textbox. More info here.
I am writing a script so that to filter jobs that are failed in past 24hrs and also the failed job is not completed or not in running state again.
rm -rf jobsfailed.txt jobscompleted.txt jobsnotcompleted.txt ## Remove the files which are created
export PGPASSWORD="$PG_PASS"
failed_jobs=`psql -t -U bacula bacula << EOD
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE jobstatus IN ('f','A','E') AND realendtime > NOW() - INTERVAL '24 HOURS';
\q
EOD` ### Collect all the jobs which are in the defined states
echo "$failed_jobs" >> jobsfailed.txt ### redirect the values to a file
sortedlist=$(awk -F'|' '{print $3}' jobsfailed.txt | sort | uniq) ## sort the values based on the jobname
for i in $sortedlist
do
retVal=$?
jobs_notcompleted=`psql -t -U bacula bacula << EOD1
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE name LIKE '$i' AND jobstatus IN ('T','R') AND starttime > NOW() - INTERVAL '24 HOURS' ORDER BY jobid DESC LIMIT 1;
\q
EOD1` ### If the job is in above defined states(T or R) then jobs completed successfully. Any other state apart from above then job not completed
if [[ $retVal -eq 0 ]]; then
echo "$jobs_notcompleted" >> jobscompleted.txt
else
echo "$jobs_notcompleted" >> jobsnotcompleted.txt
fi
exit $retVal
done
But i am not getting desired output. Since if no state is getting matched, then it is producing (0 rows) output. Please let me know, is there any other way if 0 rows are matched then that $jobs_notcompleted value should redirect to the jobsnotcompleted.txt file. jobscompleted.txt file is getting created and working as expected.
For some reason, I feel that you need a "NOT" between "jobstatus" and "in" for your second query, because the ${retVal} would, in my mind, always return 0 unless there was a DB error.
If I am wrong on that point, then the "retVal=$?" needs to be moved to after the 2nd query.
Below is the playbook
---
- name: stop agent process
shell: "ps -ef | grep -v grep | grep -w {{ MONGODB_AGENT_PROCESS }} | awk '{print $2}'"
register: running_agent_processes
- name: stop mongod process
shell: "ps -ef | grep -v grep | grep -w {{ MONGODB_SERVER_PROCESS }} | awk '{print $2}'"
register: running_mongod_processes
- name: combine processes
set_fact:
all_processes: "{{ running_agent_processes.stdout_lines + running_mongod_processes.stdout_lines }}"
- name: Kill all processes
shell: "kill {{ item }}"
with_items: "{{ all_processes }}"
when: ansible_facts[ansible_hostname] != primary
- wait_for:
path: "/proc/{{ item }}/status"
state: absent
with_items: "{{ all_processes }}"
ignore_errors: yes
register: killed_processes
when: ansible_facts[ansible_hostname] != primary
- name: Force kill stuck processes
shell: "kill -9 {{ item }}"
with_items: "{{ killed_processes.results | select('failed') | map(attribute='item') | list }}"
when: ansible_facts[ansible_hostname] != primary
I have stored a fact called "primary" which stores the primary of a mongodb replica set in a previous step in the playbook.
I just want to compare the ansible_facts[ansible_hostname] with my primary fact. If they are not equal, I would like to kill processes.
The error I am getting is below:
fatal: [lpdkubpoc01d.phx.aexp.com]: FAILED! => {"msg": "The
conditional check 'ansible_facts[ansible_hostname] != primary' failed.
The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"} fatal: [lpdkubpoc01c.phx.aexp.com]: FAILED! =>
{"msg": "The conditional check 'ansible_facts[ansible_hostname] !=
primary' failed. The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"} fatal: [lpdkubpoc01e.phx.aexp.com]: FAILED! =>
{"msg": "The conditional check 'ansible_facts[ansible_hostname] !=
primary' failed. The error was: error while evaluating conditional
(ansible_facts[ansible_hostname] != primary): 'ansible_facts' is
undefined\n\nThe error appears to have been in
'/idn/home/sanupin/stop-enterprise-mongodb/tasks/stopAutomationAgent.yml':
line 11, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n
all_processes: "{{ running_agent_processes.stdout_lines +
running_mongod_processes.stdout_lines }}"\n- name: Kill all
processes\n ^ here\n"}
Can someone help me with comparing an ansible_fact with a set_fact fact?
You can comparing use directly ansible facts without write ansible_facts before. Just use as when: ansible_hostname != primary
This is the piece of my playbook file for the question:
- name: Initialize the Kubernetes cluster using kubeadm
command: kubeadm init --config /etc/kubernetes/kubeadminit.yaml
register: init_output
- name: Copy join command to local file
local_action: copy content={{ init_output.stdout }} dest="./join-command"
Currently join-command contains the entire stdout (30+ lines of text) for content. What I want to grab is just the last two lines of init_output.stdout instead of the entire output. I've looked into using index reference (ie. init_output.stdout[#]) but I don't know that the output will always be the same length and I don't know how to use indexes to grab more than one line, but i'm fairly certain that the last two lines will always be the join command. Any suggestions?
Select last 2 lines from the list stdout_lines
- local_action: copy content={{ init_output.stdout_lines[-2:] }} dest="./join-command"
It's possible to format the lines in a block. For example
- local_action:
module: copy
content: |
{{ init_output.stdout_lines[-2] }}
{{ init_output.stdout_lines[-1] }}
dest: "./join-command"
To append the lines in a loop try
- local_action:
module: lineinfile
path: "./join-command"
line: "{{ item }}"
insertafter: EOF
create: true
loop: "{{ init_output.stdout_lines[-2:] }}"
I encountered this kind of issue and did not want to copy the join command to a local file so I did a set_fact instead this way:
- set_fact:
join_cmd: '{{ init_output.stdout_lines[-2][:-2] }}{{ init_output.stdout_lines[-1] }}'
I did this...
- name: kubeadm init
shell: |
kubeadm init --control-plane-endpoint \
localhost \
--control-plane-endpoint kube-api.local >> /tmp/run_kube_init.sh
when: master == "yes"
- name: Get join from master
fetch:
src: "/tmp/run_kube_init.sh"
dest: "/tmp/run_kube_init.sh"
flat: yes
when: ansible_hostname == 'k-master'
- name: Add join file to nodes
copy:
src: "/tmp/run_kube_init.sh"
dest: "/tmp/run_kube_init.sh"
when: master == "no"
- name: Extract join token for nodes
shell: tail -n +2 /tmp/run_kube_init.sh | head -n -1 | awk '{print $5}' | tail -n 1
register: JOIN_TOKEN
- set_fact:
join_token: "{{ JOIN_TOKEN.stdout }}"
- name: join nodes
shell: |
kubeadm join kube-api.local:6443 \
--token {{ JOIN_TOKEN.stdout }} \
--discovery-token-unsafe-skip-ca-verification
when: master == "no"
- name: rm /tmp/run_kube_init.sh
ansible.builtin.file:
path: /tmp/run_kube_init.sh
state: absent
I have multiple dict variables in my inventories that start with 'my_var_*'. I would like to combine these into an array of dicts named 'my_var'
In my playbook, I'm using 'set_fact:' to create the 'my_var' variable by attempting to pull the matching variables from "hostvars['localhost']" with a select filter and match regex, but join only works on strings.
variables.yml
my_var_1:
element1: value11
element2: value12
my_var_2:
element1: value21
element2: value22
playbook.yml
- hosts: localhost
connection: local
gather_facts: False
tasks:
- set_fact:
my_var: "{{ hostvars['localhost'] | select('match', '^my_var_*') | join(', ' }}"
- debug:
msg: "{{ my_var }}"
is it possible to join these 'dict' variables into an 'array' like this?
my_var:
- element1: value11
element2: value12
- element1: value21
element2: value22
or possibly even
my_var:
- name: 1
element1: value11
element2: value12
- name: 2
element1: value21
element2: value22
You're very close, but as you point out, the join method on a string is for joining strings. You want to append lists, which you accomplish with the + operator.
There are also a few other issues:
The expression:
hostvars['localhost'] | select('match', '^my_var_*')
Will produce a list that looks like:
[
"my_var_1",
"my_var_2"
]
...which isn't what you want. You want the values of these variables, not the key names. We can use the dict2items filter and the selectattr filter to generate the data we want:
---
- hosts: localhost
gather_facts: false
tasks:
- name: set facts on localhost
set_fact:
my_var_1:
element1: value11
element2: value12
my_var_2:
element1: value21
element2: value22
- hosts: localhost
gather_facts: false
tasks:
- name: merge vars into my_var
set_fact:
my_var: "{{ hostvars['localhost']|dict2items|selectattr('key', 'match', '^my_var_')|map(attribute='value')|list }}"
- name: show content of my_var
debug:
var: my_var
This will produce the following output:
TASK [show content of my_var] ************************************************************************************
ok: [localhost] => {
"my_var": [
{
"element1": "value11",
"element2": "value12"
},
{
"element1": "value21",
"element2": "value22"
}
]
}
If you get rid of the map(attribute='value') filter, you get:
TASK [show content of my_var] *****************************************************************************************
ok: [localhost] => {
"my_var": [
{
"key": "my_var_1",
"value": {
"element1": "value11",
"element2": "value12"
}
},
{
"key": "my_var_2",
"value": {
"element1": "value21",
"element2": "value22"
}
}
]
}
This isn't exactly what you ask for as the second option, but it does include both the key name and values.
Additional notes:
In the above, I've used a separate play running set_fact to set the values of these variables, because this solution will only work if the variables are host vars (aka "facts") rather than global variables. You don't show in your question how you're setting these variables so I don't know if this will all work as written.
In a regular expression, * means "the preceding character zero or more times", so the expression ^my_var_* would match my_var, my_var_1, my_var______________, my_varfoo, and so forth. You can simply write ^my_var_ to select the variable names in which you're interested (this will select anything that begins with the text my_var_).