I have some basic Server and Client Side code with FastAPI, but I am trying to find the best way to read in that data for a live application that does some audio processing. I am not sure how to approach this, as FastAPI can only read the data from the client as bytes. Should I even bother converting it to a Blob after recording? Maybe use HTTP for simplicity (just building a prototype so it does not need to be optimal)? I also read about RTSP but I can't seem to find any good resources for it, so I would appreciate some way to point me in the right direction. Thanks for any advice!
Client
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<h1>helloworld</h1>
<script>
var ws = new WebSocket("ws://localhost:8000/ws", 'echo-protocol');
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.start(5000);
mediaRecorder.addEventListener("dataavailable", event => {
console.log("sending audio");
const audioBlob = new Blob([event.data], {type: 'audio/mp3'});
ws.send(audioBlob);
console.log("sent audio");
});
});
</script>
</body>
</html>
Server
from fastapi import FastAPI, WebSocket, Request
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.get("/")
def read_root(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
#app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
while True:
data = await websocket.receive_bytes()
Related
I am pulling my hair out trying to figure out what I am doing wrong.
Server shows emit is being sent, but the browser never gets it. After about a minute, the client disconnects and re-connects.
However, emit after on.connect works perfect. It's almost like the client has to send the server a request and immediately get a response to work.
For some reason, StackOverflow wants me to type more info because it looks like all code, but I'm not really sure what else I can add. I googled and tried different versions and tutorials to get this working, and I believe it should be good. It must be a typo somewhere.
Okay, that still isn't enough detail, so let's go through the entire process.
I have a Raspberry Pi 3 with an SD card, running Bullseye x64 that is up to date.
I am also using a logic level shifter to drive a few 5v transistors that turn on 4 different LEDs on 4 different buttons. The buttons are grounded when pressed, and using the Raspberry Pi's internal pull up resistor. There are also 2 relays that operate solenoid air valves, the relays are separated from the Raspberry Pi/ logic level with an optocoupler.
Essentially, I am trying to use flask-socket io to display a question on a tablet. The answer is then entered using 1 of the 4 colored buttons. If the answer is wrong then relays/transistors are activated to let air out. If the answer is correct then different relays/transistors are activated to trigger the compressor and build pressure again.
The LEDs, buttons presses, and compressors all work. I just can't seem to get socketio to receive emit messages outside of an on event.
As a workaround, I may just have Javascript poll for button status every second, until I can figure out what I'm doing wrong. At least I can get that part to work.
Sorry for the long intro. Maybe it helps, to me it seems like a lot of boring text to read through while I'm sure someone with more knowledge only needed to look at my code to see what was wrong.
Nope, still not enough text.... so let's see. I'm new to Python, and not sure what else to put in here. Actually, I'll write more boring stuff at the bottom so you don't actually have to read any of this.
Raspberry Pi3 Bullseye x64
Dependancies
bidict==0.22.0
cachelib==0.9.0
click==8.1.3
Flask==2.2.1
Flask-Login==0.6.2
Flask-Session==0.4.0
Flask-SocketIO
importlib-metadata==4.12.0
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.1
python-engineio
python-socketio
six==1.11.0
Werkzeug==2.2.1
zipp==3.8.1
RPi.GPIO==0.7.1
Python code:
from flask import Flask, render_template, request
from flask_socketio import SocketIO, emit
from random import random
from threading import Lock
from datetime import datetime
import RPi.GPIO as GPIO
import time
"""
define gpio class
this holds all info for pins
"""
class rio:
def __init__(self, label, pin, state, btnPin=None):
self.label = label
self.pin = pin
self.state = state
self.btnPin = btnPin
self.checking = False
self.timeLast = time.time()
GPIO.setup(self.pin, GPIO.OUT)
if (btnPin!=None):
GPIO.setup(btnPin,GPIO.IN,pull_up_down=GPIO.PUD_UP)
GPIO.add_event_detect(self.btnPin, GPIO.FALLING, callback=self.stateChange)
GPIO.output(self.pin, self.state)
def stateChange(self,channel):
global count
duration = time.time() - self.timeLast
self.timeLast = time.time()
print(self.label + " was pressed")
count+=1
print("Sending counter event...")
socketio.emit('updateSensorData', {'value': count, "date": get_current_datetime()})
"""
Define Globals
"""
count = 0
"""
label and define what gpio
we want to use
"""
GPIO.setmode(GPIO.BCM) #enable GPIO BCM Module
yellow = rio("yellow", 22, GPIO.LOW, 14)
green = rio("green", 10, GPIO.LOW, 16)
blue = rio("blue", 9, GPIO.LOW, 20)
red = rio("red", 11, GPIO.LOW, 21)
compressor = rio("compressor", 13, GPIO.LOW)
LogicShift = rio("shifter", 26, GPIO.HIGH)
valve1 = rio("valve in", 27, GPIO.HIGH)
valve2 = rio("valve out", 17, GPIO.HIGH)
"""
create websocket
"""
app = Flask(__name__)
app.config['SECRET_KEY'] = 'donsky!'
socketio = SocketIO(app, cors_allowed_origins='*', logger=True)
"""
Get current date time
"""
def get_current_datetime():
now = datetime.now()
return now.strftime("%m/%d/%Y %H:%M:%S")
"""
Serve root index file
"""
#app.route('/')
def index():
return render_template('index.html')
"""
Decorator for connect
"""
#socketio.on('connect')
def connect():
global thread
print('Client connected')
emit('after connect', {'data':'Lets dance'})
"""
Decorator for disconnect
"""
#socketio.on('disconnect')
def disconnect():
print('Client disconnected', request.sid)
if __name__ == '__main__':
# Access app in all IP not just localhost
socketio.run(app, host="0.0.0.0")
HTML:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Synchronized</title>
<link rel="stylesheet" href='/static/style.css' />
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.1/css/bootstrap.min.css">
<script src="https://code.jquery.com/jquery-3.5.1.js"></script>
<!--script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.5.3/socket.io.js"></script-->
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/3.0.4/socket.io.js" integrity="sha512-aMGMvNYu8Ue4G+fHa359jcPb1u+ytAF+P2SCb+PxrjCdO3>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/3.7.1/chart.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){
// sending a connect request to the server.
var socket = io.connect('http://10.100.180.15:5000');
// An event handler for a change of value
$('#btn1').on('input', function(event) {
console.log('button changed');
socket.emit('Button value changed', {
who:$(this).attr('id'),
data: $(this).val()
});
return false;
});
socket.on('after connect', function(msg){
console.log('After connect', msg);
});
socket.on('update value', function(msg) {
console.log('Slider value updated');
$('#'+msg.who).val(msg.data);
});
socket.on('message', function(msg) {
console.log(msg);
});
socket.on('updateSensorData', function (msg) {
console.log('Received sensorData :: ' + msg.date + ' :: ' + msg.value);
$('#current-count').text(msg.value);
});
});
</script>
</head>
<body>
<h1>Data:</h1>
<div class="">
<h1>Demo</h1>
<form class="mt-5">
<div class="form-group">
<label for"formControlRange">Btn1</label>
<input type="checkbox" class"form-control-range sync" id="btn1" value="{{btn1}}">
</div>
</form>
</div>
<button class="btn-close btn btn-sm">×</button>
<div class="container">
<div class="hero">
<h1 class="title">Real-Time Conveyor Counter Using Raspberry Pi</h1>
<div class="counter">
<h1 id="current-count">0</h1>
</div>
</div>
</div>
</body>
</html>
I tried different dependencies, and different JavaScript versions of socketio, but all of them respond the same way. The browser is Chrome 107.0.5304.110. Safari responds the same way, as well.
I have installed multiple times Flask-socketio on my mac, closely reading the instructions and installing the requirements (eventlet/gevent). Athough when i run my simple code to test, it either says that i have not imported the modules or show nothing until i open index.html in my browser where it then displays :
The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
Here is my app.py code:
from flask import Flask
from flask_socketio import SocketIO, send
app = Flask(__name__)
app.config['SECRET_KEY'] = 'hello'
socketio = SocketIO(app, cors_allowed_origins='*')
#socketio.on('message')
def handle(msg):
print("message: "+msg)
send(msg, bradcast=True)
if __name__ == '__main__':
socketio.run(app)
And here is my terminal window:
Here is my index.html code (if needed):
<html>
<head>
<title>Chat Room</title>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.4.8/socket.io.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.4/jquery.min.js"></script>
</head>
<body>
<script type="text/javascript">
$(document).ready(function() {
var socket = io.connect('http://127.0.0.1:5000');
socket.on('connect', function() {
socket.send('User has connected!');
});
socket.on('message', function(msg) {
$("#messages").append('<li>'+msg+'</li>');
console.log('Received message');
});
$('#sendbutton').on('click', function() {
socket.send($('#myMessage').val());
$('#myMessage').val('');
});
});
</script>
<ul id="messages"></ul>
<input type="text" id="myMessage">
<button id="sendbutton">Send</button>
</body>
</html>
Thank you for your help
Check the Flask-SocketIO docs for information about version compatibility.
You have installed Flask-SocketIO version 5, so you need version 3 of the JavaScript client, but you have 1.4.8.
Use this CDN URL instead for version 3.0.5: https://cdnjs.cloudflare.com/ajax/libs/socket.io/3.0.5/socket.io.min.js
I have this on my server
app.get('*', function(req, res) {
match({ routes, location: req.url }, (error, redirectLocation, renderProps) => {
const body = renderToString(<RouterContext {...renderProps} />)
res.send(`
<!DOCTYPE html>
<html>
<head>
<link href="//cdn.muicss.com/mui-0.6.5/css/mui.min.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="root">${body}</div>
<script defer src="assets/app.js"></script>
</body>
</html>
`)
})
})
And this on the client side
import { Router, hashHistory, browserHistory, match } from 'react-router'
let history = browserHistory
//client side, will become app.js
match({ routes, location, history }, (error, redirectLocation, renderProps) => {
render(<Router {...renderProps} />, document.getElementById('root'))
})
the problem
It works only when I remove the (let history = browserHistory), but it adds the /#/ hash prefix to my url(which I don't want to happen).
When I leave the let (history = browserHistory) there, it throws an error
Warning: React attempted to reuse markup in a container but the checksum was invalid. This generally means that you are using server rendering and the markup generated on the server was not what the client was expecting. React injected new markup to compensate which works but you have lost many of the benefits of server rendering. Instead, figure out why the markup being generated is different on the client or server:
(client) < ! -- react-empty: 1 -
(server) < section data-reactro
The error message is pretty clear, however, I don't understand why it works with the hashHistory but fails with the browserHistory
version incompatibility issue
solution
{
"history": "^2.1.2",
"react-router": "~2.5.2"
}
links:
https://github.com/reactjs/react-router/issues/3003
I'm trying to getting started with socket.io and node.js.
Following the first example on the socket.io's site I'm getting the following error in the browser's console:
Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost:3001/socket.io/socket.io.js
Uncaught ReferenceError: io is not defined
This is my server.js
var app = require('express').createServer()
, io = require('socket.io').listen(app);
app.listen(3001);
app.get('/', function (req, res) {
res.sendfile(__dirname + '/index.html');
});
io.sockets.on('connection', function (socket) {
socket.emit('news', { hello: 'world' });
socket.on('my other event', function (data) {
console.log(data);
});
});
And this is my index.html
<!DOCTYPE html>
<html>
<head>
<title></title>
<meta charset="UTF-8" />
</head>
<body>
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io.connect('http://localhost');
socket.on('news', function (data) {
console.log(data);
socket.emit('my other event', { my: 'data' });
});
</script>
</body>
</html>
I've already installed socket.io..
The Issues
First of all you need to be looking at the server port that the server is bound on (app.listen(3001);) on the client side in order to reach the server at all.
As for socket.io, adding http://localhost:3001 before the rest of the source in the link tag solves this problem. This is apparently due to the way the network binds ports to localhost, however I will try to find some more information on the cause;
What to change:
The port binding for the server:
var socket = io.connect('http://localhost');
should be change to
var socket = io.connect('http://localhost:3001');
Making socket.io behave:
<script src="/socket.io/socket.io.js"></script>
should be change to
<script src="http://localhost:3001/socket.io/socket.io.js"></script>
If you are using express version 3.x there are Socket.IO compatibility issues that require a bit of fine tuning to migrate:
Socket.IO's .listen() method takes an http.Server instance as an argument.
As of 3.x, the return value of express() is not an http.Server instance. To get Socket.IO working with Express 3.x, make sure you manually create and pass your http.Server instance to Socket.IO's .listen() method.
Here is a quick example:
var app = express()
, http = require('http')
, server = http.createServer(app)
, io = require('socket.io').listen(server);
server.listen(3000);
Firstly, the Socket.io "How To Use" documentation is vague (perhaps misleading); especially when documenting code for the Socket.io client.
Secondly the answers you've been given here on StackOverflow are too specific. You shouldn't have to manually hardcode the protocol, hostname, and port number for the Socket.io client; that's not a scalable solution. Javascript can handle this for you with the location object - window.location.origin.
Getting started with Socket.io
Installation
Socket.io requires the installation of a server and client.
To install the server
npm install socket.io
To use the client, add this script to your document (index.html)
<script src="https://cdn.socket.io/socket.io-1.4.5.js"></script>
Implementation with Express 3/4
Create the following files:
Server (app.js)
var app = require('express')();
var server = require('http').Server(app);
var io = require('socket.io')(server);
server.listen(80);
app.get('/', function (req, res) {
res.sendfile(__dirname + '/index.html');
});
io.on('connection', function (socket) {
socket.emit('news', { hello: 'world' });
socket.on('my other event', function (data) {
console.log(data);
});
});
Client (index.html)
<script src="https://cdn.socket.io/socket.io-1.4.5.js"></script>
<script>
// origin = http(s)://(hostname):(port)
// The Socket.io client needs an origin
// with an http(s) protocol for the initial handshake.
// Web sockets don't run over the http(s) protocol,
// so you don't need to provide URL pathnames.
var origin = window.location.origin;
var socket = io.connect(origin);
socket.on('news', function (data) {
console.log(data);
socket.emit('my other event', { my: 'data' });
});
</script>
change;
<script src="/socket.io/socket.io.js"></script>
to;
<script src="https://cdn.socket.io/4.5.0/socket.io.min.js" integrity="sha384-7EyYLQZgWBi67fBtVxw60/OWl1kjsfrPFcaU0pp0nAh+i8FD068QogUvg85Ewy1k" crossorigin="anonymous"></script>
and drop between to head tags.
And Try it!
If you're trying to make this run on a different computer and you are getting this error, you may find this useful.
var host = window.location.hostname;
var socket = io.connect('http://' + host);
If you're using https, then obviously you need to change the protocol as well. In my case I needed to create a server that would run on several local computers, so putting here a fixed address would not work, as the socket would need to connect to a different address depending on what server you are loading the page from. Adding this here as it may help someone else too.
Try this
var app = express()
,http = require('http');
var server = http.createServer(app);
server.listen(3001);
io = require('socket.io').listen(server);
io.set('log level', 1);
Hope it helps
The following changes finally worked for me
const app = express(); const http = require('http'); const server = http.createServer(app); const io = require('socket.io')(server)
I know this is very very late, but I found a solution for those who might have previously set node.js up. My machine needed get hardware replaced, when it came back, I noticed quite a few settings were back to factory default. I was also getting the error discussed above.
Running
sudo apachectl start
from the terminal fixed up my issue.
I'm running Sinatra with Backbone.js. I'm trying to split up my models, views, etc so they aren't all lumped into a single JS file. Right now I have the following.
index.html
<html>
<head>
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
<script src="scripts/underscore-min.js"></script>
<script src="scripts/jquery-1.5.min.js"></script>
<script src="scripts/backbone-min.js"></script>
<script src="scripts/models.js"></script>
...
models.js
Models = {
var Event = Backbone.Model.extend({
});
var Events = Backbone.Collection.extend({
url: '/events',
model: Event
});
};
So models.js expects that Backbone.js has been loaded, which it should have been based on index.html, however, I'm getting a JavaScript error in models.js where I reference Backbone.Model.
Any ideas on what I'm missing here?
That isn't valid javascript. Something like this is more likely to work :
Models = {}
Models.Event = Backbone.Model.extend({
});
Models.Events = Backbone.Collection.extend({
url: '/events',
model: Event
});