parallel-python error: RuntimeError("Socket connection is broken") - sockets

I am using a simple program to send a function:
import pp
nodes=('mosura02','mosura03','mosura04','mosura05','mosura06',
'mosura09','mosura10','mosura11','mosura12')
nodes=('miner:60001',)
def pptester():
js=pp.Server(ppservers=nodes)
js.set_ncpus(0)
tmp=[]
for i in range(200):
tmp.append(js.submit(ppworktest,(),(),('os',)))
return tmp
def ppworktest():
import os
return os.system("uname -a")
the result is:
wkerzend#mosura:/home/wkerzend/tmp/ppython_test>ssh miner "source ~/coala_python_setup.sh;ppserver.py -d -p 60001"
2010-04-12 00:50:48,162 - pp - INFO - Creating server instance (pp-1.6.0)
2010-04-12 00:50:52,732 - pp - INFO - pp local server started with 32 workers
2010-04-12 00:50:52,732 - pp - DEBUG - Strarting network server interface=0.0.0.0 port=60001
Exception in thread client_socket:
Traceback (most recent call last):
File "/usr/lib64/python2.6/threading.py", line 525, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.6/threading.py", line 477, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/wkerzend/python_coala/bin/ppserver.py", line 161, in crun
ctype = mysocket.receive()
File "/home/wkerzend/python_coala/lib/python2.6/site-packages/pptransport.py", line 178, in receive
raise RuntimeError("Socket connection is broken")
RuntimeError: Socket connection is broken

Related

kubernetes.client.exceptions.ApiException: (0) Reason: Handshake status 500 Internal Server Error

getting the above exception while deploying dags in the pipeline.
The log is as follows:
************************************************************************************************************
* *
* *
* Deploying 'dags'... *
* *
* *
************************************************************************************************************
[2022-12-16 14:09:48,076] {io} INFO - Current directory: /artifacts/dags
[2022-12-16 14:09:48,076] {copy_deploy_tool} INFO - Deploy 'dags' by copying files...
[2022-12-16 14:09:48,083] {deploy_tool} INFO - saving values.yaml...
[2022-12-16 14:09:48,162] {copy_deploy_tool} INFO - Removing files from 'development:airflow-5db795dd7c-d586h:/root/airflow/dags'
[2022-12-16 14:09:48,264] {deploy} ERROR - Execution failed for project: dags
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/kubernetes/stream/ws_client.py", line 296, in websocket_call
client = WSClient(configuration, get_websocket_url(url), headers, capture_all)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/stream/ws_client.py", line 94, in __init__
self.sock.connect(url, header=header)
File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 253, in connect
self.handshake_response = handshake(self.sock, *addrs, **options)
File "/usr/local/lib/python3.6/dist-packages/websocket/_handshake.py", line 57, in handshake
status, resp = _get_resp_headers(sock)
File "/usr/local/lib/python3.6/dist-packages/websocket/_handshake.py", line 143, in _get_resp_headers
raise WebSocketBadStatusException("Handshake status %d %s", status, status_message, resp_headers)
websocket._exceptions.WebSocketBadStatusException: Handshake status 500 Internal Server Error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/commands/deploy/deploy.py", line 65, in __try_execute
deployment=project[DEPLOYMENT],
File "/usr/local/lib/python3.6/dist-packages/tools/deploy/copy_deploy_tool.py", line 50, in run
namespace, container, pod_name, command, api_client=api_client
File "/usr/local/lib/python3.6/dist-packages/helpers/kubernetes.py", line 133, in run_pod_command
stdout=True,
File "/usr/local/lib/python3.6/dist-packages/kubernetes/stream/stream.py", line 35, in stream
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/client/api/core_v1_api.py", line 841, in connect_get_namespaced_pod_exec
(data) = self.connect_get_namespaced_pod_exec_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/usr/local/lib/python3.6/dist-packages/kubernetes/client/api/core_v1_api.py", line 941, in connect_get_namespaced_pod_exec_with_http_info
collection_formats=collection_formats)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/client/api_client.py", line 345, in call_api
_preload_content, _request_timeout)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/client/api_client.py", line 176, in __call_api
_request_timeout=_request_timeout)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/stream/stream.py", line 30, in _intercept_request_call
return ws_client.websocket_call(config, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/kubernetes/stream/ws_client.py", line 302, in websocket_call
raise ApiException(status=0, reason=str(e))
kubernetes.client.rest.ApiException: (0)
Reason: Handshake status 500 Internal Server Error
[2022-12-16 14:09:49,631] {shell} INFO - doi: Deployment record uploaded successfully
[2022-12-16 14:09:49,631] {shell} INFO - OK
[2022-12-16 14:09:49,635] {io} INFO - Current directory: /artifacts
[2022-12-16 14:09:49,635] {pretty_info} INFO - ```
Usually, it happens when pod becomes into the Running state, but no running containers on it (0/1). And if you run execute command to this pod and container you will get 500 Internal server error instead of errors related to the real issue(the container is not running).
Check that all containers are running.
if all([ p.status.phase == "Running" for p in my_pods]) \
and all([c.state.running for p in my_pods for c in p.status.container_statuses]):
Also refer to this Stackpost and Github issue.

ConnectionRefusedError: [Errno 111] Connection refused - MQTT

I am very new to socket and communication using MQTT. Already spent hours but dont know what is wrong. Kindly help me to fix this error.
I am using the below code,
import socket
import json
from random import randint
from inference import Network
import paho.mqtt.client as mqtt
HOSTNAME = socket.gethostname()
IPADDRESS = socket.gethostbyname(HOSTNAME)
MQTT_HOST = IPADDRESS
MQTT_PORT = 1883 ### TODO: Set the Port for MQTT
MQTT_KEEPALIVE_INTERVAL = 60
client = mqtt.Client()
client.connect(MQTT_HOST,MQTT_PORT,MQTT_KEEPALIVE_INTERVAL)
And the error and traceback is shown below,
(venv) root#119d16139409:/home/workspace# python app.py | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 1280x720 -framerate 24 -i - http://0.0.0.0:3004/fac.ffm
Traceback (most recent call last):
File "app.py", line 145, in <module>
main()
File "app.py", line 141, in main
infer_on_video(args, model)
File "app.py", line 74, in infer_on_video
client.connect(MQTT_HOST,MQTT_PORT,MQTT_KEEPALIVE_INTERVAL)
File "/opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/venv/lib/python3.5/site-packages/paho/mqtt/client.py", line 937, in connect
return self.reconnect()
File "/opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/venv/lib/python3.5/site-packages/paho/mqtt/client.py", line 1071, in reconnect
sock = self._create_socket_connection()
File "/opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/venv/lib/python3.5/site-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection
return socket.create_connection(addr, source_address=source, timeout=self._keepalive)
File "/usr/lib/python3.5/socket.py", line 711, in create_connection
raise err
File "/usr/lib/python3.5/socket.py", line 702, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Odoo 12 in WSL - Psycopg2 doesn't get the right Port for PostgreSQL

I have installed Odoo 12 in Ubuntu 18.04 LTS using WSL for windows 10 Pro.
Everything seems fine and service is running, but when I access the localhost I get Internal Server Error, and I get this error:
2019-09-05 06:52:07,596 309 ERROR ? werkzeug: Error on request:
Traceback (most recent call last):
File "/opt/odoo/.local/lib/python3.6/site-packages/werkzeug/serving.py", line 303, in run_wsgi
execute(self.server.app)
File "/opt/odoo/.local/lib/python3.6/site-packages/werkzeug/serving.py", line 291, in execute
application_iter = app(environ, start_response)
File "/opt/odoo/odoo/odoo/service/server.py", line 409, in app
return self.app(e, s)
File "/opt/odoo/odoo/odoo/service/wsgi_server.py", line 128, in application
return application_unproxied(environ, start_response)
File "/opt/odoo/odoo/odoo/service/wsgi_server.py", line 117, in application_unproxied
result = odoo.http.root(environ, start_response)
File "/opt/odoo/odoo/odoo/http.py", line 1320, in __call__
return self.dispatch(environ, start_response)
File "/opt/odoo/odoo/odoo/http.py", line 1293, in __call__
return self.app(environ, start_wrapped)
File "/opt/odoo/.local/lib/python3.6/site-packages/werkzeug/middleware/shared_data.py", line 220, in __call__
return self.app(environ, start_response)
File "/opt/odoo/odoo/odoo/http.py", line 1453, in dispatch
self.setup_db(httprequest)
File "/opt/odoo/odoo/odoo/http.py", line 1376, in setup_db
httprequest.session.db = db_monodb(httprequest)
File "/opt/odoo/odoo/odoo/http.py", line 1537, in db_monodb
dbs = db_list(True, httprequest)
File "/opt/odoo/odoo/odoo/http.py", line 1504, in db_list
dbs = odoo.service.db.list_dbs(force)
File "/opt/odoo/odoo/odoo/service/db.py", line 375, in list_dbs
with closing(db.cursor()) as cr:
File "/opt/odoo/odoo/odoo/sql_db.py", line 657, in cursor
return Cursor(self.__pool, self.dbname, self.dsn, serialized=serialized)
File "/opt/odoo/odoo/odoo/sql_db.py", line 171, in __init__
self._cnx = pool.borrow(dsn)
File "/opt/odoo/odoo/odoo/sql_db.py", line 540, in _locked
return fun(self, *args, **kwargs)
File "/opt/odoo/odoo/odoo/sql_db.py", line 608, in borrow
**connection_info)
File "/opt/odoo/.local/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? - - -
PostgreSQL is installed using port 5433:
Ver Cluster Port Status Owner Data directory Log file
10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
I changed the port in the conf file \etc\odoo-server.conf from False to 5433:
[options]
; This is the password that allows database operations:
; admin_passwd = admin
db_host = False
db_port = 5433
db_user = odoo
db_password = False
logfile = /var/log/odoo/odoo-server.log
addons_path = /opt/odoo/addons,/opt/odoo/odoo/addons
However, I get the same error. Seems Psycopg2 take the value as default port 5432 even I specified it in the .conf file.
Any help?
Thanks
Your distribution (like mine) probably makes the key in the /tmp directory.
A quick workaround I've been using is:
mkdir /var/run/postgresql
ln -s /tmp/.s.PGSQL.5432(or 5433) /var/run/postgresql/

KeyError: 'found' on Elastic2DocManager when syncing data from MongoDB

From MongoDB to Elastic Search(5.6.5), I sync the database with Mongo-Connector using Elastic2DocManager:
mongo-connector -m localhost:27017 -t localhost:9200 -d elastic2_doc_manager
After seeing some update on docs.deleted of mongodb_meta on Elastic Search:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open mongodb_meta 3wd6OjTT6tD3f6ZGezZw 5 1 1337173 8372 192.9mb 192.9mb
mongo-connector stops working with below error:
2018-07-11 07:16:41,977 [WARNING] elasticsearch:97 - POST http://localhost:9200/_bulk [status:N/A request:10.003s]
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "c:\programdata\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "c:\programdata\anaconda3\lib\http\client.py", line 1331, in getresponse
response.begin()
File "c:\programdata\anaconda3\lib\http\client.py", line 297, in begin
version, status, reason = self._read_status()
File "c:\programdata\anaconda3\lib\http\client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "c:\programdata\anaconda3\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
...
Exception in thread Thread-1:
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 150, in run
self._docman.send_buffered_operations()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 482, in send_buffered_operations
action_buffer = self.BulkBuffer.get_buffer()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 696, in get_buffer
self.update_sources()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\util.py", line 35, in wrapped
return f(*args, **kwargs)
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 628, in update_sources
if ES_doc['found']:
KeyError: 'found'
What is the reason for this error?
Make sure the versions are compatible.
The required python packages are installed, this is my requirements.txt look like.
astroid==1.6.5
autopep8==1.3.5
certifi==2018.8.24
elasticsearch==6.3.1
elasticsearch-dsl==6.2.1
isort==4.3.4
lazy-object-proxy==1.3.1
mccabe==0.6.1
pycodestyle==2.4.0
pylint==1.9.2
pymongo==3.7.1
rope==0.11.0
six==1.11.0
urllib3==1.23
wrapt==1.10.11

celery failing on dotcloud deployment with IO Error

Celery is failing on one of my dotcloud deployments, and I'm not sure how to fix. The deployment is almost identical to an existing dotcloud deployment (verified via doing a file diff) which seems to be working ok.
The error I get in djcelery log:
dotcloud#hack-default-www-0:/var/log/supervisor$ more djcelery_error.log
/home/dotcloud/env/lib/python2.6/site-packages/django/conf/__init__.py:75: Depre
cationWarning: The ADMIN_MEDIA_PREFIX setting has been removed; use STATIC_URL i
nstead.
"use STATIC_URL instead.", DeprecationWarning)
/home/dotcloud/env/lib/python2.6/site-packages/djcelery/loaders.py:108: UserWarn
ing: Using settings.DEBUG leads to a memory leak, never use this setting in prod
uction environments!
warnings.warn("Using settings.DEBUG leads to a memory leak, never "
[2012-06-04 03:27:32,139: WARNING/MainProcess] -------------- celery#hack-defaul
t-www-0 v2.5.3
---- **** -----
--- * *** * -- [Configuration]
-- * - **** --- . broker: amqp://root#hack-OQVADQ2K.dotcloud.com:29210//
- ** ---------- . loader: djcelery.loaders.DjangoLoader
- ** ---------- . logfile: [stderr]#INFO
- ** ---------- . concurrency: 2
- ** ---------- . events: ON
- *** --- * --- . beat: OFF
-- ******* ----
--- ***** ----- [Queues]
-------------- . celery: exchange:celery (direct) binding:celery
[Tasks]
. experiments.tasks.pushMessageToIphone
. experiments.tasks.sendTestMessage
[2012-06-04 03:27:32,172: INFO/PoolWorker-1] child process calling self.run()
[2012-06-04 03:27:32,185: INFO/PoolWorker-2] child process calling self.run()
[2012-06-04 03:27:32,188: WARNING/MainProcess] celery#hack-default-www-0 has sta
rted.
[2012-06-04 03:27:35,315: ERROR/MainProcess] Consumer: Connection Error: Socket
closed. Trying again in 2 seconds...
[2012-06-04 03:27:40,374: ERROR/MainProcess] Consumer: Connection Error: Socket
closed. Trying again in 4 seconds...
[2012-06-04 03:27:47,479: ERROR/MainProcess] Consumer: Connection Error: Socket
closed. Trying again in 6 seconds...
[2012-06-04 03:27:56,509: ERROR/MainProcess] Consumer: Connection Error: Socket
Interestingly, the error log of celery cam shows something a bit different. I'm not sure if this is a red herring..
/home/dotcloud/env/lib/python2.6/site-packages/django/conf/__init__.py:75: Depre
cationWarning: The ADMIN_MEDIA_PREFIX setting has been removed; use STATIC_URL i
nstead.
"use STATIC_URL instead.", DeprecationWarning)
[2012-06-04 03:27:31,373: INFO/MainProcess] -> evcam: Taking snapshots with djce
lery.snapshot.Camera (every 1.0 secs.)
Traceback (most recent call last):
File "hack/manage.py", line 14, in
execute_manager(settings)
File "/home/dotcloud/env/lib/python2.6/site-packages/django/core/management/__
init__.py", line 459, in execute_manager
utility.execute()
File "/home/dotcloud/env/lib/python2.6/site-packages/django/core/management/__
init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/dotcloud/env/lib/python2.6/site-packages/djcelery/management/base.
py", line 74, in run_from_argv
return super(CeleryCommand, self).run_from_argv(argv)
File "/home/dotcloud/env/lib/python2.6/site-packages/django/core/management/ba
se.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/dotcloud/env/lib/python2.6/site-packages/djcelery/management/base.
py", line 67, in execute
super(CeleryCommand, self).execute(*args, **options)
File "/home/dotcloud/env/lib/python2.6/site-packages/django/core/management/ba
se.py", line 232, in execute
output = self.handle(*args, **options)
File "/home/dotcloud/env/lib/python2.6/site-packages/djcelery/management/comma
nds/celerycam.py", line 26, in handle
ev.run(*args, **options)
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/bin/celeryev.py",
line 38, in run
detach=detach)
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/bin/celeryev.py",
line 70, in run_evcam
return cam()
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/events/snapshot.py
", line 116, in evcam
recv.capture(limit=None)
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/events/__init__.py
", line 204, in capture
list(self.itercapture(limit=limit, timeout=timeout, wakeup=wakeup))
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/events/__init__.py
", line 193, in itercapture
with self.consumer(wakeup=wakeup) as consumer:
File "/usr/lib/python2.6/contextlib.py", line 16, in __enter__
return self.gen.next()
File "/home/dotcloud/env/lib/python2.6/site-packages/celery/events/__init__.py
", line 185, in consumer
queues=[self.queue], no_ack=True)
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/messaging.py", line
279, in __init__
self.revive(self.channel)
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/messaging.py", line
286, in revive
channel = channel.default_channel
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/connection.py", lin
e 581, in default_channel
self.connection
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/connection.py", lin
e 574, in connection
self._connection = self._establish_connection()
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/connection.py", lin
e 533, in _establish_connection
conn = self.transport.establish_connection()
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/transport/amqplib.p
y", line 279, in establish_connection
connect_timeout=conninfo.connect_timeout)
File "/home/dotcloud/env/lib/python2.6/site-packages/kombu/transport/amqplib.p
y", line 89, in __init__
super(Connection, self).__init__(*args, **kwargs)
File "/home/dotcloud/env/lib/python2.6/site-packages/amqplib/client_0_8/connec
tion.py", line 144, in __init__
(10, 30), # tune
File "/home/dotcloud/env/lib/python2.6/site-packages/amqplib/client_0_8/abstra
ct_channel.py", line 95, in wait
self.channel_id, allowed_methods)
File "/home/dotcloud/env/lib/python2.6/site-packages/amqplib/client_0_8/connec
tion.py", line 202, in _wait_method
self.method_reader.read_method()
File "/home/dotcloud/env/lib/python2.6/site-packages/amqplib/client_0_8/method
_framing.py", line 221, in read_method
raise m
IOError: Socket closed
My supervisord file:
[program:djcelery]
directory = /home/dotcloud/current/
command = /home/dotcloud/env/bin/python hack/manage.py celeryd -E -l info -c 2
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log
[program:celerycam]
directory = /home/dotcloud/current/
command = /home/dotcloud/env/bin/python hack/manage.py celerycam
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log
As mentioned, I have nearly identical code deployed under a different dotcloud account that is working fine.
Status of the rabbitmq broker:
$ ./dotcloud info hack.broker
aliases:
- hackxxxx.dotcloud.com
config:
password: xxxx
rabbitmq_management: true
user: root
created_at: 1338702527.075196
datacenter: Amazon-us-east-1c
image_version: 924a079b622a (latest)
memory: 49M/512M (9%)
ports:
- name: ssh
url: ssh://dotcloud#hackxxx.dotcloud.com:29209
- name: amqp
url: amqp://root:xxxx#hackxxxx.dotcloud.com:29210
- name: http
url: http://root:xxx#hack1-xxxx.dotcloud.com/
state: running
type: rabbitmq
It looks like it is having an issue connection to your broker. Have you confirmed that you can connect to your broker, and it is up and running?
What are you using for a broker?