I'm getting an error when running this command ansible-playbook kubespray-2.12.3/cluster.yml
Error:
k8s-master : ok=12 changed=2 unreachable=0 failed=1
k8s-worker-1 : ok=12 changed=2 unreachable=0 failed=1
k8s-worker-2 : ok=12 changed=2 unreachable=0 failed=1
localhost : ok=1 changed=0 unreachable=0 failed=0
Ansible version :
root#MSI:/home/andrew# ansible --version
ansible 2.7.12
config file = /home/andrew/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
Inventory.ini file
[all]
k8s-master ansible_host=<ext-IP> access_ip=<ext-IP> ip=<int-ip>
k8s-worker-1 ansible_host=<ext-IP>access_ip=<ext-IP> ip=<int-ip>
k8s-worker-2 ansible_host=<ext-IP> access_ip=<ext-IP> ip=<int-ip>
[kube-master]
k8s-master
[etcd]
k8s-master
[kube-node]
k8s-worker-1
k8s-worker-2
[k8s-cluster:children]
kube-master
kube-node
[all:vars]
kube_webhook_token_auth=true
kube_webhook_token_auth_url=https://10.0.0.3
ansible_python_interpreter=/usr/bin/python3
kube_network_plugin=flannel
kubeconfig_localhost=true
ansible.cfg file
[ssh_connection]
pipelining=True
ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null
#control_path = ~/.ssh/ansible-%%r#%%h:%%p
[defaults]
strategy_plugins = plugins/mitogen/ansible_mitogen/plugins/strategy
# https://github.com/ansible/ansible/issues/56930 (to ignore group names with - and .)
force_valid_group_names = ignore
host_key_checking=False
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp
stdout_callback = skippy
library = ./library
callback_whitelist = profile_tasks
roles_path = roles:$VIRTUAL_ENV/usr/local/share/kubespray/roles:$VIRTUAL_ENV/usr/local/share/ansible/roles:/usr/share/kubespray/roles
deprecation_warnings=False
inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo, .creds, .gpg
[inventory]
ignore_patterns = artifacts, credentials
I know that similar errors have been recorded in stack overflow but they don't seem relevant to my case..any help or tips gratefully received.....in fact just give me a clue f you don't want to tell me the actual answer....
many thanks
Related
Inventory file (inventory/k8s.yaml):
plugin: kubernetes.core.k8s
connections:
- kubeconfig: ~/.kube/config
context: 'cluster-2'
Task file (roles/common/tasks/main.yaml):
# Method 1: Using `kubernetes.core` plugin to list the pod names:
- name: Get a list of all pods from any namespace
kubernetes.core.k8s_info:
kind: Pod
register: pod_list
- name: Print pod names
debug:
msg: "pod_list: {{ pod_list | json_query('resources[*].metadata.name') }} "
# Method 2: Using `shell` command to list the pod names:
- name: Get node names
shell: kubectl get pods
register: pod_list2
- name: Print pod names
debug:
msg: "{{ pod_list2.stdout }}"
Ansible config (ansible.cfg):
[inventory]
enable_plugins = host_list, auto, yaml, ini, kubernetes.core.k8s
Main file (main.yaml):
---
- hosts: localhost
gather_facts: false
collections:
- azure.azcollection
- kubernetes.core
roles:
- "common"
Running command to execute task: ansible-playbook main.yaml -i cluster-2/k8s.yaml -e role=common -e cluster_name=cluster-2
Question:
I am running the above configs to run get the pods from the remote cluster mentioned in the inventory file. But, the problem is, I am still getting the pod names from the local cluster and not the cluster-2 in Method 1 and 2.
k8s plugin should get the list of pods from cluster-2 as described in the inventory file. How can I connect to remote kubernetes cluster?
I also checked output with -vvvv:
ansible-playbook [core 2.14.0]
config file = /Users/test/u/apps/ansible.cfg
configured module search path = ['/Users/test/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/test/Library/Python/3.9/lib/python/site-packages/ansible
ansible collection location = /Users/test/.ansible/collections:/usr/share/ansible/collections
executable location = /Users/test/Library/Python/3.9/bin/ansible-playbook
python version = 3.9.12 (main, Mar 26 2022, 15:52:10) [Clang 13.0.0 (clang-1300.0.29.30)] (/usr/local/opt/python#3.9/bin/python3.9)
jinja version = 3.1.2
libyaml = True
Using /Users/test/u/apps/ansible.cfg as config file
setting up inventory plugins
Loading collection kubernetes.core from /Users/test/.ansible/collections/ansible_collections/kubernetes/core
You're trying to use both the kubernetes inventory plugin and the k8s_info module, and because of that you're getting conflicting results. The two don't have anything to do with each other.
The inventory module
The kubernetes inventory module is -- I think -- a weird beast; it produces an ansible inventory in which the pods in your cluster are presented as Ansible hosts. To see a list of all the pod names in your cluster, you could write a playbook like this:
- hosts: all
gather_facts: false
tasks:
- name: Print pod names
debug:
msg: "{{ inventory_hostname }}"
This will respect the context you've configured in your kubernetes inventory plugin configuration. For example, if I have in inventory/k8s.yaml the following:
plugin: kubernetes.core.k8s
connections:
- kubeconfig: ./kubeconfig
context: 'kind-cluster2'
Then the above playbook will list the pod names from kind-cluster2, regardless of the current-context setting in my kubeconfig file. In my test environment, this produces:
PLAY [all] *********************************************************************
TASK [Print pod names] *********************************************************
ok: [kubernetes] => {
"msg": "kubernetes"
}
ok: [coredns-565d847f94-2shl6_coredns] => {
"msg": "coredns-565d847f94-2shl6_coredns"
}
ok: [coredns-565d847f94-md57c_coredns] => {
"msg": "coredns-565d847f94-md57c_coredns"
}
ok: [kube-dns] => {
"msg": "kube-dns"
}
ok: [etcd-cluster2-control-plane_etcd] => {
"msg": "etcd-cluster2-control-plane_etcd"
}
ok: [kube-apiserver-cluster2-control-plane_kube-apiserver] => {
"msg": "kube-apiserver-cluster2-control-plane_kube-apiserver"
}
ok: [kube-controller-manager-cluster2-control-plane_kube-controller-manager] => {
"msg": "kube-controller-manager-cluster2-control-plane_kube-controller-manager"
}
ok: [kube-scheduler-cluster2-control-plane_kube-scheduler] => {
"msg": "kube-scheduler-cluster2-control-plane_kube-scheduler"
}
ok: [kindnet-nc27b_kindnet-cni] => {
"msg": "kindnet-nc27b_kindnet-cni"
}
ok: [kube-proxy-9chgt_kube-proxy] => {
"msg": "kube-proxy-9chgt_kube-proxy"
}
ok: [local-path-provisioner-684f458cdd-925v5_local-path-provisioner] => {
"msg": "local-path-provisioner-684f458cdd-925v5_local-path-provisioner"
}
PLAY RECAP *********************************************************************
coredns-565d847f94-2shl6_coredns : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
coredns-565d847f94-md57c_coredns : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
etcd-cluster2-control-plane_etcd : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kindnet-nc27b_kindnet-cni : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kube-apiserver-cluster2-control-plane_kube-apiserver : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kube-controller-manager-cluster2-control-plane_kube-controller-manager : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kube-dns : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kube-proxy-9chgt_kube-proxy : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kube-scheduler-cluster2-control-plane_kube-scheduler : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
kubernetes : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
local-path-provisioner-684f458cdd-925v5_local-path-provisioner : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The key point here is that your inventory will consist of a list of pods. I've never found this particularly useful.
The k8s_info module
The k8s_info queries a kubernetes cluster for a list of objects. It doesn't care about your inventory configuration -- it will run on whichever target host you've defined for your play (probably localhost) and perform the rough equivalent of kubectl get <whatever>. If you want to use an explicit context, you need to set that as part of your module parameters. For example, to see a list of pods in kind-cluster2, I could use the following playbook:
- hosts: localhost
gather_facts: false
tasks:
- kubernetes.core.k8s_info:
kind: pod
kubeconfig: ./kubeconfig
context: kind-cluster2
register: pods
- debug:
msg: "{{ pods.resources | json_query('[].metadata.name') }}"
Which in my test environment produces as output:
PLAY [localhost] ***************************************************************
TASK [kubernetes.core.k8s_info] ************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": [
"coredns-565d847f94-2shl6",
"coredns-565d847f94-md57c",
"etcd-cluster2-control-plane",
"kindnet-nc27b",
"kube-apiserver-cluster2-control-plane",
"kube-controller-manager-cluster2-control-plane",
"kube-proxy-9chgt",
"kube-scheduler-cluster2-control-plane",
"local-path-provisioner-684f458cdd-925v5"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
In conclusion: you probably want to use k8s_info rather than the inventory plugin, and you'll need to configure the module properly by setting the context (and possibly the kubeconfig) parameters when you call the module.
Is there any way I can define context and kubeconfig outside of the tasks (globally) if I am using k8s_info module?
According to the documentation, you could set the K8S_AUTH_KUBECONFIG and K8S_AUTH_CONTEXT environment variables if you want to globally configure the settings for the k8s_info module. You could also write your task like this:
- kubernetes.core.k8s_info:
kind: pod
kubeconfig: "{{ k8s_kubeconfig }}"
context: "{{ k8s_context }}"
register: pods
And then define the k8s_kubeconfig and k8s_context variables somewhere else in your Ansible configuration (e.g., as group vars). This makes it easy to retarget things to a different cluster with only a single change.
I have an offline host node which includes (compute node, control node and storage node). This host node was shutdown by accident and can't recover to online. All services about that node are down and enable but I can't set to disable.
So I can't remove the host by:
kolla-ansible -i multinode stop --yes-i-really-really-mean-it --limit node-17
I get this error:
TASK [Gather facts] ********************************************************************************************************************************************************************************************************************
fatal: [node-17]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host node-17 port 22: Connection timed out", "unreachable": true}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
node-17 : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0
How can I remove that offline host node? THX.
PS: Why I remove that offline host?
node-14(online) : **manage node which kolla-ansible installed**; compute node, control node and storage node
node-15(online) : compute node, control node and storage node
node-17(offline) : compute node, control node and storage node
osc99 (adding) : compute node, control node and storage node
Because when I deploy a new host(osc99) with (the multinode file had comment the node-17 line):
kolla-ansible -i multinode deploy --limit osc99
kolla-ansible will report error:
TASK [keystone : include_tasks] ********************************************************************************************************************************************************************************************************
included: .../share/kolla-ansible/ansible/roles/keystone/tasks/init_fernet.yml for osc99
TASK [keystone : Waiting for Keystone SSH port to be UP] *******************************************************************************************************************************************************************************
ok: [osc99]
TASK [keystone : Initialise fernet key authentication] *********************************************************************************************************************************************************************************
ok: [osc99 -> node-14]
TASK [keystone : Run key distribution] *************************************************************************************************************************************************************************************************
fatal: [osc99 -> node-14]: FAILED! => {"changed": true, "cmd": ["docker", "exec", "-t", "keystone_fernet", "/usr/bin/fernet-push.sh"], "delta": "0:00:04.006900", "end": "2021-07-12 10:14:05.217609", "msg": "non-zero return code", "rc": 255, "start": "2021-07-12 10:14:01.210709", "stderr": "", "stderr_lines": [], "stdout": "Warning: Permanently added '[node.15]:8023' (ECDSA) to the list of known hosts.\r\r\nssh: connect to host node.17 port 8023: No route to host\r\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\r\nrsync error: unexplained error (code 255) at io.c(235) [sender=3.1.2]", "stdout_lines": ["Warning: Permanently added '[node.15]:8023' (ECDSA) to the list of known hosts.", "", "ssh: connect to host node.17 port 8023: No route to host", "", "rsync: connection unexpectedly closed (0 bytes received so far) [sender]", "rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.2]"]}
NO MORE HOSTS LEFT *********************************************************************************************************************************************************************************************************************
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
osc99 : ok=120 changed=55 unreachable=0 failed=1 skipped=31 rescued=0 ignored=1
How could I fixed this error, this is the main point whether or not I can remove the offline host.
Maybe I could fixed that by change the init_fernet.yml file:
node-14:~$ cat .../share/kolla-ansible/ansible/roles/keystone/tasks/init_fernet.yml
---
- name: Waiting for Keystone SSH port to be UP
wait_for:
host: "{{ api_interface_address }}"
port: "{{ keystone_ssh_port }}"
connect_timeout: 1
register: check_keystone_ssh_port
until: check_keystone_ssh_port is success
retries: 10
delay: 5
- name: Initialise fernet key authentication
become: true
command: "docker exec -t keystone_fernet kolla_keystone_bootstrap {{ keystone_username }} {{ keystone_groupname }}"
register: fernet_create
changed_when: fernet_create.stdout.find('localhost | SUCCESS => ') != -1 and (fernet_create.stdout.split('localhost | SUCCESS => ')[1]|from_json).changed
until: fernet_create.stdout.split()[2] == 'SUCCESS' or fernet_create.stdout.find('Key repository is already initialized') != -1
retries: 10
delay: 5
run_once: True
delegate_to: "{{ groups['keystone'][0] }}"
- name: Run key distribution
become: true
command: docker exec -t keystone_fernet /usr/bin/fernet-push.sh
run_once: True
delegate_to: "{{ groups['keystone'][0] }}"
by changing the delegate_to: "{{ groups['keystone'][0] }}? But I can't implement that.
I have used this code it only enter with new ipaddress
sed -r 's/(\b[0-9]{1,3}.){3}[0-9]{1,3}\b'/$newip/
But i need to enter with system ipaddress to the file directly,can anyone help it out with this
Thanks in advance
The command you have already does this. Just need to add the -i option to the sed:
$ cat file
{ "MONGODB_CONNECTION_URL" : "mongodb://127.0.0.1/genieacs",
"CWMP_INTERFACE" : "0.0.0.0",
"CWMP_PORT" : 7547,
"CWMP_SSL" : false,
"NBI_INTERFACE" : "0.0.0.0",
"NBI_PORT" : 7557,
"FS_INTERFACE" : "0.0.0.0",
"FS_PORT" : 7567,
"FS_HOSTNAME" : "acs.example.com",
"DEBUG" : false
}
Set the IP address in a variable:
$ newip="192.168.0.20"
Run the sed command:
$ sed -i -r 's/(\b[0-9]{1,3}.){3}[0-9]{1,3}\b'/$newip/ file
File has updated IP:
$ cat file
{ "MONGODB_CONNECTION_URL" : "mongodb://255.255.25.0/genieacs",
"CWMP_INTERFACE" : "255.255.25.0",
"CWMP_PORT" : 7547,
"CWMP_SSL" : false,
"NBI_INTERFACE" : "255.255.25.0",
"NBI_PORT" : 7557,
"FS_INTERFACE" : "255.255.25.0",
"FS_PORT" : 7567,
"FS_HOSTNAME" : "acs.example.com",
"DEBUG" : false
}
sed -E 's/([^/])\b([0-9]{1,3}\.){3}[0-9]{1,3}\b([^/])'/\1192.168.0.28\2/g config.json
if as variable e.g. obtained from ifconfig etc, after testing do real editing, add -i option
$ newip=192.168.0.28
$ sed -Ei 's/([^/])\b([0-9]{1,3}\.){3}[0-9]{1,3}\b([^/])'/\1$newip\2/g config.json
I'm trying to deploy Kolla in AIO.
I build images using the command:
kolla-build -p default -b ubuntu -t binary
I am deploying it in my local system.
I'm using ubuntu 16.04, built images. I'm not using a local registry.
kolla-ansible precheck runs fine
kolla-ansible deploy gives me an error while starting rabbitmq
My host name is DESKTOP
The output of hosts file
cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 DESKTOP
::1 ip6-localhost ip6-loopback <br>
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
The error is as follows
TASK: [rabbitmq | fail msg="Hostname has to resolve to IP address of api_interface"] ***
failed: [localhost] => (item={u'cmd': [u'getent', u'ahostsv4', u'DESKTOP'], u'end': u'2017-02-26 00:45:10.399323', u'stderr': u'', u'stdout': u'127.0.1.1 STREAM DESKTOP\n127.0.1.1 DGRAM \n127.0.1.1 RAW ', u'changed': False, u'rc': 0, 'item': 'localhost', u'warnings': [], u'delta': u'0:00:00.001585', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'getent ahostsv4 DESKTOP'}, 'stdout_lines': [u'127.0.1.1 STREAM DESKTOP', u'127.0.1.1 DGRAM ', u'127.0.1.1 RAW '], u'start': u'2017-02-26 00:45:10.397738'}) => {"failed": true, "item": {"changed": false, "cmd": ["getent", "ahostsv4", "DESKTOP"], "delta": "0:00:00.001585", "end": "2017-02-26 00:45:10.399323", "invocation": {"module_args": "getent ahostsv4 DESKTOP", "module_complex_args": {}, "module_name": "command"}, "item": "localhost", "rc": 0, "start": "2017-02-26 00:45:10.397738", "stderr": "", "stdout": "127.0.1.1 STREAM DESKTOP\n127.0.1.1 DGRAM \n127.0.1.1 RAW ", "stdout_lines": ["127.0.1.1 STREAM DESKTOP", "127.0.1.1 DGRAM ", "127.0.1.1 RAW "], "warnings": []}}
msg: Hostname has to resolve to IP address of api_interface
FATAL: all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit #/home/ravichandran/site.retry
localhost : ok=84 changed=11 unreachable=0 failed=1
Please help. Also let me know if additional information is required.
Make sure your hosts file has a mapping of DESKTOP to 127.0.0.1. Your formatting is a little off but this appears to be what's missing.
I am installing a ceph-cluster with one monitor node and one osd.
I am following the document: http://docs.ceph.com/docs/v0.86/start/quick-ceph-deploy/
During the step 5: Add the initial monitor(s) and gather the keys (new in ceph-deploy v1.1.3),
I am getting the following exception:
**[ceph-mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory**
[ceph-mon1][WARNIN] monitor: mon.ceph-mon1, might not be running yet
[ceph-mon1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-mon1.asok mon_status
**[ceph-mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph-mon1][WARNIN] monitor ceph-mon1 does not exist in monmap**
Just for reference my **ceph.conf** is as follows:
*[global]
fsid = 351948ba-9716-4a04-802d-28b5510bfeb0
mon_initial_members = ceph-mon1,ceph-admin,ceph-osd1
mon_host = xxx.yyy.zzz.78,xxx.yyy.zzz.147,xxx.yyy.zzz.135
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
osd_pool_default_size = 2
public_addr = xxx.yyy.zzz.0*
I tried to understand all the questions related to sane on ceph user mailing list but there is no precise solution I found for this problem.
Can anyone help me on this?
Thanks in advance.
I faced the same errors was able to resolve the issue by adding my other ceph node's hostname & IpAdrress and by adding "public_network ="
The sections which I tweaked in ceph.conf are:
mon_initial_members =
mon_host =
public_network =
cat /etc/ceph/ceph.conf
[global]
fsid = 33cb5c76-a685-469e-8cdd-fee7c98c3f4d
mon_initial_members = ceph1,ceph2
mon_host = 192.168.61.39,192.168.61.40
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
public_network = 192.168.61.0/24
And the running the command:
$ ceph-deploy --overwrite-conf mon create <ceph-node>
I had a similar issue...
My problem was that the alias hostname in my /etc/hosts on my deployment server and my target server was with a different hostname ....
Always make sure your hostname on the server is the same in the ceph.conf and the correct IP- HOSTNAME are the same in /etc/hosts on your deployment box ...