I'm trying to make a socket connection between two python files for testing. My server should be uploading some data to clients that are listening. I'm trying to test it by creating some dummy client. After client connects, I'm getting
websocket._exceptions.WebSocketBadStatusException: Handshake status 404 NOT FOUND
Unfortunately I couldn't find any solution online for this error
import time
from threading import Thread
from flask import Flask
from flask_socketio import SocketIO
from flask_sockets import Sockets
from websocket import create_connection
app = Flask(__name__)
socketio = SocketIO(app)
sockets = Sockets(app)
#sockets.route('/socket_test')
def update_time(ws):
while not ws.closed:
ws.send('hello world')
time.sleep(1)
class Client(Thread):
def __init__(self):
super().__init__()
def run(self):
time.sleep(0.5)
ws = create_connection('ws://localhost:5000/socket_test')
while True:
ws.recv()
if __name__ == '__main__':
k = Client()
k.start()
socketio.run(app)
I would like client to receive hello world messages from server
Related
I'm running Python Flask with Waitress. I'm starting the server using the following code:
from flask import Flask, render_template, request
from waitress import serve
#app.route("/get")
def first_method():
...
#app.route("/second")
def second_method():
...
app = Flask(__name__)
app.static_folder = 'static'
serve(app, host="ip_address", port=8080)
I'm calling the server from a Webpage and also from Unity. From the webpage, I'm using the following example get request in jQuery:
$.get("/get", { variable1: data1, variable2: data2 }).done(function (data) {
...
}
In Unity I'm using the following call:
http://ip_address/get?msg=data1?data2
Unfortuantely, after some time I'm getting the error on the server total open connections reached the connection limit, no longer accepting new connections. This especially happens with Unity. I assume that for each get request a new channel/connection is established.
How can this be fixed, i.e. how can channels/connections be reused?
Minimal reproducible example:
import asyncio
import aiopg
from fastapi import FastAPI, WebSocket
dsn = "dbname=aiopg user=aiopg password=passwd host=127.0.0.1"
app = FastAPI()
class ConnectionManager:
self.count_connections = 0
# other class functions and variables are taken from FastAPI docs
...
manager = ConnectionManager()
async def send_and_receive_data(websocket: WebSocket):
data = await websocket.receive_json()
await websocket.send_text('Thanks for the message')
# then process received data
# taken from official aiopg documentation
# the function listens to PostgreSQL notifications
async def listen(conn):
async with conn.cursor() as cur:
await cur.execute("LISTEN channel")
while True:
msg = await conn.notifies.get()
async def postgres_listen():
async with aiopg.connect(dsn) as listenConn:
listener = listen(listenConn)
await listener
#app.get("/")
def read_root():
return {"Hello": "World"}
#app.websocket("/")
async def websocket_endpoint(websocket: WebSocket):
await manager.connect(websocket)
manager.count_connections += 1
if manager.count_connections == 1:
await asyncio.gather(
send_and_receive_data(websocket),
postgres_listen()
)
else:
await send_and_receive_data(websocket)
Description of the problem:
I am building an app with Vue.js, FastAPI and PostgreSQL. In this example I attempt to use listen/notify from Postgres and implement it in the websocket. I also use a lot of usual http endpoints along with the websocket endpoint.
I want to run a permanent background asynchronous function at the start of the FastAPI app that will then send messages to all websocket clients/connections. So, when I use uvicorn main:app it should not only run the FastAPI app but also my background function postgres_listen(), which notifies all websocket users, when a new row is added to the table in the database.
I know that I can use asyncio.create_task() and place it in the on_* event, or even place it after the manager = ConnectionManager() row, but it will not work in my case! Because after any http request (for instance, read_root() function), I will get the same error described below.
You see that I use a strange way to run my postgres_listen() function in my websocket_endpoint() function only when the first client connects to the websocket. Any subsequent client connection does not run/trigger this function again. And everything works fine... until the first client/user disconnects (for example, closes browser tab). When it happens, I immediately get the GeneratorExit error caused by psycopg2.OperationalError:
Future exception was never retrieved
future: <Future finished exception=OperationalError('Connection closed')>
psycopg2.OperationalError: Connection closed
Task was destroyed but it is pending!
task: <Task pending name='Task-18' coro=<Queue.get() done, defined at
/home/user/anaconda3/lib/python3.8/asyncio/queues.py:154> wait_for=<Future cancelled>>
The error comes from the listen() function. After this error, I will not get any notification from the database as the asyncio's Task is cancelled. There is nothing wrong with the psycopg2, aiopg or asyncio. The problem is that I don't understand where to put the postgres_listen() function so it will not be cancelled after the first client disconnects. From my understanding, I can easily write a python script that will connect to the websocket (so I will be the first client of the websocket) and then run forever so I will not get the psycopg2.OperationalError exception again, but it does not seem right to do so.
My question is: where should I put postgres_listen() function, so the first connection to websocket may be disconnected with no consequences?
P.S. asyncio.shield() also does not work
I have answered this on Github as well, so I am reposting it here.
A working example can be found here:
https://github.com/JarroVGIT/fastapi-github-issues/tree/master/5015
# app.py
import queue
from typing import Any
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from asyncio import Queue, Task
import asyncio
import uvicorn
import websockets
class Listener:
def __init__(self):
#Every incoming websocket conneciton adds it own Queue to this list called
#subscribers.
self.subscribers: list[Queue] = []
#This will hold a asyncio task which will receives messages and broadcasts them
#to all subscribers.
self.listener_task: Task
async def subscribe(self, q: Queue):
#Every incoming websocket connection must create a Queue and subscribe itself to
#this class instance
self.subscribers.append(q)
async def start_listening(self):
#Method that must be called on startup of application to start the listening
#process of external messages.
self.listener_task = asyncio.create_task(self._listener())
async def _listener(self) -> None:
#The method with the infinite listener. In this example, it listens to a websocket
#as it was the fastest way for me to mimic the 'infinite generator' in issue 5015
#but this can be anything. It is started (via start_listening()) on startup of app.
async with websockets.connect("ws://localhost:8001") as websocket:
async for message in websocket:
for q in self.subscribers:
#important here: every websocket connection has its own Queue added to
#the list of subscribers. Here, we actually broadcast incoming messages
#to all open websocket connections.
await q.put(message)
async def stop_listening(self):
#closing off the asyncio task when stopping the app. This method is called on
#app shutdown
if self.listener_task.done():
self.listener_task.result()
else:
self.listener_task.cancel()
async def receive_and_publish_message(self, msg: Any):
#this was a method that was called when someone would make a request
#to /add_item endpoint as part of earlier solution to see if the msg would be
#broadcasted to all open websocket connections (it does)
for q in self.subscribers:
try:
q.put_nowait(str(msg))
except Exception as e:
raise e
#Note: missing here is any disconnect logic (e.g. removing the queue from the list of subscribers
# when a websocket connection is ended or closed.)
global_listener = Listener()
app = FastAPI()
#app.on_event("startup")
async def startup_event():
await global_listener.start_listening()
return
#app.on_event("shutdown")
async def shutdown_event():
await global_listener.stop_listening()
return
#app.get('/add_item/{item}')
async def add_item(item: str):
#this was a test endpoint, to see if new items where actually broadcasted to all
#open websocket connections.
await global_listener.receive_and_publish_message(item)
return {"published_message:": item}
#app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
q: asyncio.Queue = asyncio.Queue()
await global_listener.subscribe(q=q)
try:
while True:
data = await q.get()
await websocket.send_text(data)
except WebSocketDisconnect:
return
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)
As I didn't have access to a stream of message I could have subscribed to, I created a quick script that produces a websocket, so that the app.py above could listen to that (indefinitely) to mimic your use case.
# generator.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
import asyncio
import uvicorn
app = FastAPI()
#app.websocket("/")
async def ws(websocket: WebSocket):
await websocket.accept()
i = 0
while True:
try:
await websocket.send_text(f"Hello - {i}")
await asyncio.sleep(2)
i+=1
except WebSocketDisconnect:
pass
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8001)
The app.py will listen to a websocket and publishes all incoming messages to all connections to the websockets in app.py.
The generator.py is a simple FastAPI app that has a websocket (that our example app.py above listens to) that emits a message every 2 seconds to every connection it gets.
To try this out:
Start generator.py (e.g. python3 generator.py on your command line when in your working folder)
Start app.py (either debug mode in VScode or same as above)
Listen to http://localhost:8000/ws (= endpoint in app.py) with several clients, you will see that they will all join in the same message streak.
NOTE: lots of this logic was inspired by Broadcaster (a python module)
I'm trying to set up a secure socket server on esp32 with micropython. I used/tried the latest bulid (esp32-idf3-20200117-v1.12-68-g3032ae115.bin) with a self-signed certificate.
I saw a lot of memory leak related problem with ssl.wrap_socket() on esp32/esp8266 but what I got is different:
mbedtls_ssl_handshake error: -4310
Traceback (most recent call last):
File "boot.py", line 100, in <module>
OSError: [Errno 5] EIO
and the connection fails of course. I try to connect from my laptop. The exact same code of client side works if I start a secure socket server on the laptop itself (127.0.0.1) so I suppose that the client side is OK and the problem is on the server side.
I could not find any solution for this problem yet. I tried certificate and key both in 'der' and 'pem' format the result is the same.
The toy server code I tried:
import esp
esp.osdebug(None)
import gc
gc.collect()
import usocket as socket
import ssl
import machine
import network
KEY_PATH = 'server_key.der'
CERT_PATH = 'server_cert.der'
ssid= "AP_local"
pw="passwd"
ap = network.WLAN(network.AP_IF) # create access-point interface
ap.config(essid=ssid, password=pw, authmode=4) # set the ESSID of the access point
ap.ifconfig(('192.168.3.1', '255.25.255.0', '192.168.3.1', '8.8.8.8'))
ap.active(True)
with open(KEY_PATH, 'rb') as f:
key = f.read()
print(key)
print(" ")
with open(CERT_PATH, 'rb') as f:
cert = f.read()
#s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ad = ap.ifconfig()
addr = socket.getaddrinfo(ad[0], 8443)[0][-1]
s.bind(addr)
s.listen(5)
import gc
gc.collect()
while True:
print("ssl connection started")
cl, addr = s.accept()
scl = ssl.wrap_socket(cl, server_side=True, cert=cert, key=key)
print(gc.mem_free())
l = 0
while True:
req = scl.read(1024)
print(req)
if not req or b'\r\n' in req:
break
response = '\r\n'.join(['HTTP/1.1 200 OK',
'Content-Type: text/plain',
'OK',
'Connection: close', '\r\n'])
scl.write(response.encode("utf-8"))
scl.close()
I hope someone could help me with this, thanks!
OK, after spending a few days on searching and reading and testing I got a solution and a possible reason for this issue. I hope it will help others a little.
I want to put forward that I didn't dig into the source code of mbedtls so the reasoning is somewhat phenomenological.
First, I tried to connect from a python program with default settings which failed as shown above. Then I tried with the openssl cli, which also failed.
I tried with the example given at https://github.com/micropython/micropython/blob/master/examples/network/http_server_ssl.py and failed again but with different error codes.
Finally I found a very useful page:
https://tls.mbed.org/api/ssl_8h.html#a55ed67b6e414f9b381ff536d9ea6b9c0
which helped to understand where the problem occures.
Now depending on requirements different errors occured from -7900,-7780, -7380, -7d00.
It turned out that although in the documentation the cipher suit being used is automatically agreed during handshake but there is some bug in it or the implementation of some of them in micropython is different.
I didn't tested all the available ciphers on my laptop but e.g.: AES_256_GCM_SHA384
cipher works.
For me it is enough now.
I have a Web application that is using Flask-socketIO.
__init__.py
import flask
import flask_socketio
app = flask.Flask(__name__)
socketio = flask_socketio.SocketIO()
socketio.init_app(app, manage_session=False)
I have this in my manage.py
socketio.run(app, host='0.0.0.0', port=80)
I have this in my socket_IO.py
#WebApp.socketio.on('disconnect', namespace='/all')
def disconnect():
"""Socket disconnected."""
print('Disconnected')
If i now set the system clock ( Ubuntu 16.04 ) for example 1 day forward i get socket disconnect. I can set the clock backwards to whatever i want it does not effect the socket.
What i expect to happen is for the socket not to fire the disconnect function on a change in the system clock.
Does flask-socketIO use the system clock and because of that makes the socket disconnect if it's set forward and not backwards?
Thanks in advance
Probably doing something very silly here, but I'm having some trouble authenticating automatically through Tor.
I'm using 32 bit ubuntu 12.04 with obfuscated bridges.
This should be all the relevant code, but let me know if there's something else that would be useful in debugging this issue:
import socket
import socks
import httplib
def connectTor():
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050, True)
#9050 is the Tor proxy port
socket.socket = socks.socksocket
def newIdentity():
socks.setdefaultproxy() #Disconnect from Tor network
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 46594))
s.send("AUTHENTICATE\r\n")
response = s.recv(128)
#128 bytes of data for now, just to see how Tor responds
print response
if response.startswith("250"): #250 is the code for a positive response from Tor
s.send("SIGNAL NEWNYM\r\n") #Use a new identity
s.close()
connectTor() #Just to make sure we're still connected to Tor
Whenever I run this I get the following error:
515 Authentication failed: Password did not match HashedControlPassword value from configuration. Maybe you tried a plain text password
I tried using the --hash-password option and pasting that in the place of the AUTHENTICATE string, but that just caused the script to hang. Thoughts?
That error means that you set the HashedControlPassword option in your torrc. I would suggest option for CookieAuthentication 1 instead then using a controller library rather than doing this from scratch.
What you're trying to do here (issue a NEWNYM) is a very, very common request (1, 2) so I just added a FAQ entry for it. Here's an example using stem...
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)