I have a task in my ansible-playbook script to open TCP port on a remote machine. but when I run my ansible playbook it throws an error. But when i run "firewall-cmd --permanent --zone=public --add-port=1234/tcp" and "firewalld-cmd --reload" I can see port is added in public zone.
Environment
Ansible local: OS x El Capitan
Ansible remote: AWS Centos 7 minimum version
Ansible version: 2.1.1.0
Remote python version: 2.7.5
My task
- name: open management console port
firewalld: port=1234/tcp zone=public permanent=true state=enabled immediate=yes
The error I am getting
fatal: [X.X.X.X]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_MojhHQ/ansible_module_firewalld.py\", line 605, in <module>\r\n main()\r\n File \"/tmp/ansible_MojhHQ/ansible_module_firewalld.py\", line 456, in main\r\n is_enabled = get_port_enabled_permanent(zone, [port, protocol])\r\n File \"/tmp/ansible_MojhHQ/ansible_module_firewalld.py\", line 170, in get_port_enabled_permanent\r\n fw_zone = fw.config().getZoneByName(zone)\r\n File \"<string>\", line 2, in getZoneByName\r\n File \"/usr/lib/python2.7/site-packages/slip/dbus/polkit.py\", line 103, in _enable_proxy\r\n return func(*p, **k)\r\n File \"<string>\", line 2, in getZoneByName\r\n File \"/usr/lib/python2.7/site-packages/firewall/client.py\", line 52, in handle_exceptions\r\n return func(*args, **kwargs)\r\n File \"/usr/lib/python2.7/site-packages/firewall/client.py\", line 1505, in getZoneByName\r\n path = dbus_to_python(self.fw_config.getZoneByName(name))\r\n File \"/usr/lib64/python2.7/site-packages/dbus/proxies.py\", line 70, in __call__\r\n return self._proxy_method(*args, **keywords)\r\n File \"/usr/lib/python2.7/site-packages/slip/dbus/proxies.py\", line 50, in __call__\r\n return dbus.proxies._ProxyMethod.__call__(self, *args, **kwargs)\r\n File \"/usr/lib64/python2.7/site-packages/dbus/proxies.py\", line 145, in __call__\r\n **keywords)\r\n File \"/usr/lib64/python2.7/site-packages/dbus/connection.py\", line 651, in call_blocking\r\n message, timeout)\r\ndbus.exceptions.DBusException: org.fedoraproject.slip.dbus.service.PolKit.NotAuthorizedException.org.fedoraproject.FirewallD1.config: \r\n", "msg": "MODULE FAILURE", "parsed": false}
- name: Install firewalld
yum:
name: firewalld
state: latest
notify:
- start firewalld
- name: start firewalld
service:
name: firewalld
state: started
enabled: yes
become: yes
- name: enable 1234
firewalld:
zone: public
port: 1234/tcp
permanent: true
state: enabled
become: yes
Do it this way . It will work
dbus.exceptions.DBusException: org.fedoraproject.slip.dbus.service.PolKit.NotAuthorizedException.org.fedoraproject.FirewallD1.config indicates there's some sort of permissions error. The task probably needs to elevate its privileges with become: yes.
See the become documentation for more details.
Related
I'm having trouble working my code in docker. Could you please help me?
I'm putting my application in docker together with mongo in docker, but when I run the file inside docker it doesn't connect with the mongo of the other docker and accuses Timeout.
My Docker:
version: "3.4"
services:
mongo_db:
image: mongo:6.0
ports:
- "27017:27017"
volumes:
- ./mongo_db:/data/db
container_name: mongo_db
mongo_app:
image: mongo_img_db:latest
links:
- mongo_db
command: python3 /app/main.py
container_name: mongo_app
Error:
### Connection: Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'app')
Traceback (most recent call last):
File "/app/main.py", line 216, in <module>
db_calc.update_storage_stats(db_calc.calculate_artifacts_size())
File "/app/main.py", line 43, in calculate_artifacts_size
artifacts_size = self.db_client.builds.aggregate(
File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 2428, in aggregate
with self.__database.client._tmp_session(session, close=False) as s:
File "/usr/local/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1757, in _tmp_session
s = self._ensure_session(session)
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1685, in __start_session
self._topology._check_implicit_session_support()
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
self._check_session_support()
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 554, in _check_session_support
self._select_servers_loop(
File "/usr/local/lib/python3.9/site-packages/pymongo/topology.py", line 238, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 63542415a868649d12f2a966, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 111] Connection refused')>]>
I'm having trouble getting flask to connect to the mongo database. I was able to connect to the base via Pycharm - Database, additionally I downloaded MongoDB Compass to check if I can also connect from there and I did it. I installed the mongo server in containers, not locally.
This is my docker and docker-compose:
dockerfile:
FROM python:3.10
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
WORKDIR /code
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
CMD python main.py
docker-compose.yaml:
version: "3.8"
services:
db_mongo:
image: mongo:5.0
container_name: mongo
ports:
- "27018:27017"
volumes:
- ./init-mongo.js:/docker-entrypoint-initdb.d/init-mongo.js:ro
- ./mongo-volume:/data/db
env_file:
- ./env/.env_database
backend:
build: .
volumes:
- .:/code/
ports:
- '8100:5000'
container_name: flask_api_container
depends_on:
- db_mongo
init-mongo.js:
db.log.insertOne({"message": "Database created."});
db = db.getSiblingDB('admin');
db.auth('mk', 'adminPass')
db = db.getSiblingDB('flask_api_db');
db.createUser(
{
user: 'flask_api_user',
pwd: 'dbPass',
roles: [
{
role: 'dbOwner',
db: 'flask_api_db'
}
]
}
)
db.createCollection('collection_test');
I copied the uri address from MongoDB Compass (I connected to the base with this address). I have tried various combinations of this address.
main.py:
import pymongo
from flask import Flask, Response, jsonify
from flask_pymongo import PyMongo
app = Flask(__name__)
uri = 'mongodb://flask_api_user:dbPass#db_mongo:27018/?authMechanism=DEFAULT&authSource=flask_api_db'
client = pymongo.MongoClient(uri)
client.admin.command('ismaster') # to check if the connection has been established - show errors in terminal
# try:
# mongodb_client = PyMongo(
# app=app,
# uri='mongodb://flask_api_user:dbPass#db_mongo:27018/?authMechanism=DEFAULT&authSource=flask_api_db'
# # uri='mongodb://flask_api_user:dbPass#localhost:27018/?authMechanism=DEFAULT&authSource=flask_api_db'
# )
# db = mongodb_client.db
# print('DB: ', db, flush=True)
# # client.admin.command('ismaster')
# # print('OK', flush=True)
# except:
# print('Error', flush=True)
#app.route('/')
def index():
return 'Test2'
# #app.route("/add_one/")
# def add_one():
# db.my_collection.insert_one({'title': "todo title", 'body': "todo body"})
# return jsonify(message="success")
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=8100)
Errors:
when I did this, it returned None:
# db = mongodb_client.db
# print('DB: ', db, flush=True)
below errors from this method:
uri = 'mongodb://flask_api_user:dbPass#db_mongo:27018/?authMechanism=DEFAULT&authSource=flask_api_db'
client = pymongo.MongoClient(uri)
client.admin.command('ismaster') # to check if the connection has been established - show errors in terminal
flask_api_container | Traceback (most recent call last):
flask_api_container | File "/code/main.py", line 9, in <module>
flask_api_container | client.admin.command('ismaster')
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/database.py", line 721, in command
flask_api_container | with self.__client._socket_for_reads(read_preference, session) as (
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1235, in _socket_for_reads
flask_api_container | server = self._select_server(read_preference, session)
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1196, in _select_server
flask_api_container | server = topology.select_server(server_selector)
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/topology.py", line 251, in select_server
flask_api_container | servers = self.select_servers(selector, server_selection_timeout, address)
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/topology.py", line 212, in select_servers
flask_api_container | server_descriptions = self._select_servers_loop(selector, server_timeout, address)
flask_api_container | File "/usr/local/lib/python3.10/site-packages/pymongo/topology.py", line 227, in _select_servers_loop
flask_api_container | raise ServerSelectionTimeoutError(
flask_api_container | pymongo.errors.ServerSelectionTimeoutError: db_mongo:27018: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 62a9baa2cc5e740091e0a60b, topology_type: Unknown, servers: [<ServerDescription ('db_mongo', 27018) server_type: Unknown, rtt: None, error=AutoReconnect('db_mongo:27018: [Errno 111] Connection refused')>]>
flask_api_container exited with code 1
None of the uri used worked.
Thanks for any help in resolving this issue.
main.py
MongoDB Compass
PyCharm - database
main.py 2
The mapped port (27018) is used when you connect from the host to the database. Containers on the same bridge network as the database connect directly to the container and should use the container port (27017).
So your connection string should be
uri = 'mongodb://flask_api_user:dbPass#db_mongo:27017/?authMechanism=DEFAULT&authSource=flask_api_db'
I am having some trouble with building the Yocto project, hope I can find some help. Is there any way to fix the following issue? Let me know if you need any more information. Thanks.
My goal
I am building the default image from this guide : https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html
This build takes up a lot of space, so I want to build it on a network drive.
Current situation
I am able to finish the build normally if I am using a normal folder.
If I use the the shared drive that is mounted in the system, the build never starts. The error looks like this :
$ bitbake core-image-sato
NOTE: Bitbake server didn't start within 5 seconds, waiting for 90
ERROR: Error parsing configuration files
Traceback (most recent call last):
File "/mnt/NetworkShare/yocto/poky/bitbake/lib/bb/persist_data.py", line 45, in SQLTable.wrap_func(*args=('CREATE TABLE IF NOT EXISTS BB_URI_HEADREVS(key TEXT PRIMARY KEY NOT NULL, value TEXT);',), **kwargs={}):
if self.connection is None and reconnect:
> self.reconnect()
File "/mnt/NetworkShare/yocto/poky/bitbake/lib/bb/persist_data.py", line 105, in SQLTable.reconnect():
self.connection.text_factory = str
> self._setup_database()
File "/mnt/NetworkShare/yocto/poky/bitbake/lib/bb/persist_data.py", line 50, in SQLTable.wrap_func(*args=(), **kwargs={}):
try:
> return f(self, *args, **kwargs)
except sqlite3.OperationalError as exc:
File "/mnt/NetworkShare/yocto/poky/bitbake/lib/bb/persist_data.py", line 79, in SQLTable.wrap_func(*args=(), **kwargs={}):
with contextlib.closing(self.connection.cursor()) as cursor:
> return f(self, cursor, *args, **kwargs)
return wrap_func
File "/mnt/NetworkShare/yocto/poky/bitbake/lib/bb/persist_data.py", line 93, in SQLTable._setup_database(cursor=<sqlite3.Cursor object at 0x7f3d59c5dab0>):
def _setup_database(self, cursor):
> cursor.execute("pragma synchronous = off;")
# Enable WAL and keep the autocheckpoint length small (the default is
sqlite3.OperationalError: disk I/O error
Details
The /etc/fstab line to mount the drive is :
NetworkShare /mnt/NetworkShare 9p trans=virtio,version=9p2000.L,_netdev,rw 0 0
The host is Ubuntu server 20.04 running in a VM inside UnRAID. I don't think the VM is the issue (It's possible that I am very wrong) because I get the same error if I mount an external share on my own computer (OpenSuse Tumbleweed) and try to build in it.
The bitbake-cookerdaemon.log :
1221 13:38:18.293775 --- Starting bitbake server pid 1221 at 2022-01-19 13:38:18.293689 ---
1221 13:38:18.333537 Started bitbake server pid 1221
1221 13:38:18.339125 Entering server connection loop
1221 13:38:18.340399 Accepting [<socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([])
1221 13:38:18.341382 Processing Client
1221 13:38:18.342099 Connecting Client
1221 13:38:18.343689 Running command ['setFeatures', [2]]
1221 13:38:18.344805 Command Completed
1221 13:38:18.346085 Running command ['updateConfig', {'abort': True, 'force': False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, {'SHELL': '/bin/bash', 'PWD': '/mnt/NetworkShare/yocto/poky/build', 'LOGNAME': 'metics', 'HOME': '/home/metics', 'BBPATH': '/mnt/NetworkShare/yocto/poky/build', 'BB_ENV_EXTRAWHITE': 'ALL_PROXY BBPATH_EXTRA BB_LOGCONFIG BB_NO_NETWORK BB_NUMBER_THREADS BB_SETSCENE_ENFORCE BB_SRCREV_POLICY DISTRO FTPS_PROXY FTP_PROXY GIT_PROXY_COMMAND HTTPS_PROXY HTTP_PROXY MACHINE NO_PROXY PARALLEL_MAKE SCREENDIR SDKMACHINE SOCKS5_PASSWD SOCKS5_USER SSH_AGENT_PID SSH_AUTH_SOCK STAMPS_DIR TCLIBC TCMODE all_proxy ftp_proxy ftps_proxy http_proxy https_proxy no_proxy ', 'USER': 'metics', 'PATH': '/mnt/NetworkShare/yocto/poky/scripts:/mnt/NetworkShare/yocto/poky/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LC_ALL': 'en_US.UTF-8', 'TERMCAP': 'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#51:co#110:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d#:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Km=\\E[M:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:#1=\\E[1~:kH=\\E[4~:#7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:', 'WINDOW': '0', 'XDG_SESSION_TYPE': 'tty', 'MOTD_SHOWN': 'pam', 'LANG': 'en_US.UTF-8', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'SSH_CONNECTION': '10.0.0.12 60522 10.0.0.19 22', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'XDG_SESSION_CLASS': 'user', 'PYTHONPATH': '/mnt/NetworkShare/yocto/poky/bitbake/lib:', 'TERM': 'screen.xterm-256color', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'SHLVL': '2', 'XDG_SESSION_ID': '1', 'XDG_RUNTIME_DIR': '/run/user/1000', 'SSH_CLIENT': '10.0.0.12 60522 22', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'STY': '1116.pts-0.ubuntuserver', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'BUILDDIR': '/mnt/NetworkShare/yocto/poky/build', 'SSH_TTY': '/dev/pts/0', 'OLDPWD': '/mnt/NetworkShare/yocto/poky', '_': '/mnt/NetworkShare/yocto/poky/bitbake/bin/bitbake'}, ['/mnt/NetworkShare/yocto/poky/bitbake/bin/bitbake', 'core-image-sato']]
1221 13:38:33.830099 Command Completed
1221 13:38:33.831731 Processing Client
1221 13:38:33.832344 Disconnecting Client
1221 13:38:33.833129 No timeout, exiting.
1221 13:38:33.933875 Exiting
1221 13:38:33.942717 Original lockfile contents: ['1221\n']
1221 13:38:33.954461 Exiting as we could obtain the lock
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/mnt/NetworkShare/yocto/poky/build/bitbake-cookerdaemon.log' mode='a+' encoding='UTF-8'>
It means your hard disk is full. You should delete some files before re-running job to create a new image.
I am trying to make a reboot to a pod on a Rancher Cluster through Ansible and I am having this error:
ERROR! couldn't resolve module/action 'k8s_exec'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/ansible/ansible/GetKubectlPods': line 27, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
-
name: Reboot Machine
^ here
I don't know why I am getting an error since I am also using k8s_info and he is working be fine.
Here is the playbook I am running:
---
- hosts: localhost
connection: local
remote_user: root
vars:
ansible_python_interpreter: '{{ ansible_playbook_python }}'
tasks:
-
name: Get the pods in the specific namespace
k8s_info:
kubeconfig: '/etc/ansible/RCCloudConfig'
kind: Pod
namespace: redmine
register: pod_list
-
name: Print pod names
debug:
msg: "pod_list: {{ pod_list | json_query('resources[*].status.podIP') }} "
- set_fact:
pod_names: "{{pod_list|json_query('resources[*].metadata.name')}}"
-
name: Reboot Machine
k8s_exec:
kubeconfig: '/etc/ansible/RCCloudConfig'
namespace: redmine
pod: redmine_quick-testing-6c57cc5d65-lwkww #pod name
command: reboot
Ansible Version:
ansible 2.9.9
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.6/site-packages/ansible
executable location = /usr/bin/ansible
python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
In case I edit my playbook and add the community.kubernetes collection, I get the following error:
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/__init__.py:12: DeprecationWarning: The package kubernetes.client.apis is renamed and deprecated, use kubernetes.client.api instead (please note that the trailing s was removed).\n DeprecationWarning\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py\", line 296, in websocket_call\n client = WSClient(configuration, get_websocket_url(url), headers, capture_all)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py\", line 94, in __init__\n self.sock.connect(url, header=header)\n File \"/usr/local/lib/python3.6/site-packages/websocket/_core.py\", line 226, in connect\n self.handshake_response = handshake(self.sock, *addrs, **options)\n File \"/usr/local/lib/python3.6/site-packages/websocket/_handshake.py\", line 80, in handshake\n status, resp = _get_resp_headers(sock)\n File \"/usr/local/lib/python3.6/site-packages/websocket/_handshake.py\", line 165, in _get_resp_headers\n raise WebSocketBadStatusException(\"Handshake status %d %s\", status, status_message, resp_headers)\nwebsocket._exceptions.WebSocketBadStatusException: Handshake status 404 Not Found\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1592530746.4685414-46123-84159696554463/AnsiballZ_k8s_exec.py\", line 102, in <module>\n _ansiballz_main()\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1592530746.4685414-46123-84159696554463/AnsiballZ_k8s_exec.py\", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1592530746.4685414-46123-84159696554463/AnsiballZ_k8s_exec.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.community.kubernetes.plugins.modules.k8s_exec', init_globals=None, run_name='__main__', alter_sys=True)\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_k8s_exec_payload_q6oom26c/ansible_k8s_exec_payload.zip/ansible_collections/community/kubernetes/plugins/modules/k8s_exec.py\", line 148, in <module>\n File \"/tmp/ansible_k8s_exec_payload_q6oom26c/ansible_k8s_exec_payload.zip/ansible_collections/community/kubernetes/plugins/modules/k8s_exec.py\", line 135, in main\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/stream/stream.py\", line 35, in stream\n return func(*args, **kwargs)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/client/api/core_v1_api.py\", line 841, in connect_get_namespaced_pod_exec\n (data) = self.connect_get_namespaced_pod_exec_with_http_info(name, namespace, **kwargs) # noqa: E501\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/client/api/core_v1_api.py\", line 941, in connect_get_namespaced_pod_exec_with_http_info\n collection_formats=collection_formats)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py\", line 345, in call_api\n _preload_content, _request_timeout)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py\", line 176, in __call_api\n _request_timeout=_request_timeout)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/stream/stream.py\", line 30, in _intercept_request_call\n return ws_client.websocket_call(config, *args, **kwargs)\n File \"/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py\", line 302, in websocket_call\n raise ApiException(status=0, reason=str(e))\nkubernetes.client.rest.ApiException: (0)\nReason: Handshake status 404 Not Found\n\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
Solution:
Find your k8s_exec.py file and edit your import settings from Kubernetes.apis to Kubernetes.api
Thank you in advance!
I am using using https://github.com/helm/charts/tree/master/stable/airflow helm chart and building v1.10.8 puckle/docker-airflow image with kubernetes installed on it and using that image in the helm chart,
But I keep getting
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 1140, in initdb
db.initdb(settings.RBAC)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 332, in initdb
dagbag = models.DagBag()
File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 95, in __init__
executor = get_default_executor()
File "/usr/local/lib/python3.7/site-packages/airflow/executors/__init__.py", line 48, in get_default_executor
DEFAULT_EXECUTOR = _get_executor(executor_name)
File "/usr/local/lib/python3.7/site-packages/airflow/executors/__init__.py", line 87, in _get_executor
return KubernetesExecutor()
File "/usr/local/lib/python3.7/site-packages/airflow/contrib/executors/kubernetes_executor.py", line 702, in __init__
self.kube_config = KubeConfig()
File "/usr/local/lib/python3.7/site-packages/airflow/contrib/executors/kubernetes_executor.py", line 283, in __init__
self.kube_client_request_args = json.loads(kube_client_request_args)
File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.7/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
In my scheduler, also as various sources advise,
I tried setting :
AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS: {"_request_timeout" : [60,60] }
in my helm values. that also didn't work any one have any ideas what am I missing?
Here's my values.yaml
airflow:
image:
repository: airflow-docker-local
tag: 1.10.8
executor: Kubernetes
service:
type: LoadBalancer
config:
AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY: airflow-docker-local
AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG: 1.10.8
AIRFLOW__KUBERNETES__WORKER_CONTAINER_IMAGE_PULL_POLICY: Never
AIRFLOW__KUBERNETES__WORKER_SERVICE_ACCOUNT_NAME: airflow
AIRFLOW__KUBERNETES__DAGS_VOLUME_CLAIM: airflow
AIRFLOW__KUBERNETES__NAMESPACE: airflow
AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS: {"_request_timeout" : [60,60] }
AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://postgres:airflow#airflow-postgresql:5432/airflow
persistence:
enabled: true
existingClaim: ''
workers:
enabled: false
postgresql:
enabled: true
redis:
enabled: false
EDIT :
Various attempts to set environment variable in helm values.yaml didn't work, after that I added (pay attention to double and single quotes)
ENV AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS='{"_request_timeout" : [60,60] }'
to Dockerfile here : https://github.com/puckel/docker-airflow/blob/1.10.9/Dockerfile#L19
after that my airflow-scheduler pod starts but then I keep getting following error on my scheduler pod.
Process KubernetesJobWatcher-9: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 313,
in recv_into return self.connection.recv_into(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py",
line 1840, in recv_into self._raise_ssl_error(self._ssl, result) File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py",
line 1646, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError
For the helm value, the template uses a loop that places the airflow.config map into double quotes ". This means any " in a value needs to be escaped for the output templated YAML to be valid.
airflow:
config:
AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS: '{\"_request_timeout\":60}'
That deploys and runs (but I haven't completed an end to end test)
According to this github issue, the python scheduler SSL timeout may not be a problem as the watcher starts again after the 60 second connection timeout.