I am trying to do the simplest thing ever but for some reason that I cannot see (maybe because of looking for too long) is not working.
I have 2 zeromq sockets , the connection with the client thread is REQ and the connection with the broker node is ROUTER.
I removed everything not related just to find the issue , the code looks like that:
import zmq
import json
from time import sleep
from threading import Thread
def client(client_name):
print('|CLIENT {}| starting'.format(client_name))
# Prepare our context and sockets
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:5559")
socket.send(b'HI')
message = socket.recv()
print("Received reply %s" % (message))
socket.close()
context.term()
def broker():
print('|BROKER| starting')
# Prepare our context and sockets
context = zmq.Context()
frontend = context.socket(zmq.ROUTER)
frontend.bind("tcp://*:5559")
# Initialize poll set
poller = zmq.Poller()
poller.register(frontend, zmq.POLLIN)
received = frontend.recv()
# to_send = [[{'from': '1', 'value': 'yooooo'}, {'from': '1', 'value': 'mamamia'}], [{'from': '2', 'value': 'yooooo'}, {'from': '2', 'value': 'mamamia'}], ]
# to_send = [0,12,6,8,3,2,10]
frontend.send(b'OK')
print('broker sent the message')
if __name__ == '__main__':
broker = Thread(target=broker)
client = Thread(target=client, args=['1', ])
broker.start()
client.start()
broker.join()
client.join()
And my problem is that the client never receives a message from the broker node.
I experimented with many formats, like "send_string" , "send_multipart", etc. but the problem seems to be that the client node is not receiving anything at all.
I would really appreciate some help because I am stuck at the moment with that.
Related
I am using micropython on my esp32. I want to send data from esp32 to AWS IoT MQTT Broker Endpoint.
But in micropython socket module, getaddrinfo(hostname, port) method return empty list every time. How can I solve it?
It is working when I use IP address instead of host name. But AWS MQTT broker endpoint has no static IP for its broker endpoint.
# My code:
sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.connect("wifiname", "password")
KEY_PATH = "/619e3d582c-private.pem.key"
CERT_PATH = "/619e3d582c-certificate.pem"
with open(KEY_PATH, 'r') as f:
PVT_KEY = f.read()
with open(CERT_PATH, 'r') as f:
CERT_KEY = f.read()
client = MQTTClient(client_id="esp32_micropython_shafik",
server="xxxxxxxxxxx.iot.eu-west-1.amazonaws.com",
port = 8883,
keepalive = 10000,
ssl = True,
ssl_params = {
"cert": CERT_KEY,
"key": PVT_KEY,
"server_side":False
} )
def checkwifi():
while not sta_if.isconnected():
time.sleep_ms(500)
print(".")
sta_if.connect()
def publish():
while True:
checkwifi()
msg = b'hello shafik'
client.publish(b"weather", msg)
time.sleep(1)
print("type", type(PVT_KEY))
client.connect()
publish()`
I am getting continuously this error:
umqtt/simple.py in 57 line,
IndexError: list index out of range.
How can I solve it?
I have solved this problem just changing my wifi network.
Actually this problem occurred for my ISP network issue, that's why socket module didn't work properly for fetching IP address properly.
I have two services: one that sends stream data and the second one receives it using akka-grpc for communication. When source data is provided Service one is called to process and send it to service two via grpc client. It's possible that multiple instances of server one runs at the same time when multiple source data are provided at the same time.In long running test of my application. I see below error in service one:
ERROR i.a.g.application.actors.DbActor - GraphStage [akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1#59d40805] terminated abruptly, caused by for example materializer or act
akka.stream.AbruptStageTerminationException: GraphStage [akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1#59d40805] terminated abruptly, caused by for example materializer or actor system termination.
I have never shutdown actor systems but only kill actors after doing their job. Also I used proto3 and http2 for request binding. Here is a piece of my code in service one:
////////////////////server http binding /////////
val service: HttpRequest => Future[HttpResponse] =
ServiceOneServiceHandler(new ServiceOneServiceImpl(system))
val bound = Http().bindAndHandleAsync(
service,
interface = config.getString("akka.grpc.server.interface"),
port = config.getString("akka.grpc.server.default-http-port").toInt,
connectionContext = HttpConnectionContext(http2 = Always))
bound.foreach { binding =>
logger.info(s"gRPC server bound to: ${binding.localAddress}")
}
////////////////////client /////////
def send2Server[A](data: ListBuffer[A]): Future[ResponseDTO] = {
val reply = {
val thisClient = interface.initialize()
interface.call(client = thisClient, req = data.asInstanceOf[ListBuffer[StoreRequest]].toList)
}
reply
}
///////////////// grpc communication //////////
def send2GrpcServer[A](data: ListBuffer[A]): Unit = {
val reply = send2Server(data)
Await.ready(reply, Duration.Inf) onComplete {
case util.Success(response: ResponseDTO) =>
logger.info(s"got reply message: ${res.description}")
//////check response content and stop application if desired result not found in response
}
case util.Failure(exp) =>
//////stop application
throw exp.getCause
}
}
Error occurred exactly after waiting for service 2 response :
Await.ready(reply, Duration.Inf)
I can't catch the cause of error.
UPDATE
I found that some stream is missed such that service one sends an stream an indefinitely wait for the response and service two does not receive any thing to reply to service one but still don't know why stream is missed
I also updated akka grpc plugin but has no sense:
addSbtPlugin("com.lightbend.akka.grpc" % "sbt-akka-grpc" % "0.6.1")
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.4")
I am working with an asyncio.Protocol server where the purpose is for the client to call the server, but wait until the server has responded and data is returned before stopping the client loop.
Based on the asyncio doc Echo Client and Server here: https://docs.python.org/3/library/asyncio-protocol.html#protocol-example-tcp-echo-server-and-client , results of transport.write(...) are returned immediately when called.
Through experience, calling loop.run_until_complete(coroutine) fails with RuntimeError: Event loop is running.
Running asyncio.sleep(n) in the data_received() method of the server doesn't have any effect either.
yield from asyncio.sleep(n) and yield from asyncio.async(asyncio.sleep(n)) in data_received() both hang the server.
My question is, how do I get my client to wait for the server to write a response before giving back control?
I guess to never use transport/protocol pair directly.
asyncio has Streams API for high-level programming.
Client code can look like:
#asyncio.coroutine
def communicate():
reader, writer = yield from asyncio.open_connection(HOST, PORT)
writer.write(b'data')
yield from writer.drain()
answer = yield from reader.read()
# process answer, maybe send new data back to server and wait for answer again
writer.close()
You don't have to change the client code.
echo-client.py
#!/usr/bin/env python3.4
import asyncio
class EchoClient(asyncio.Protocol):
message = 'Client Echo'
def connection_made(self, transport):
transport.write(self.message.encode())
print('data sent: {}'.format(self.message))
def data_received(self, data):
print('data received: {}'.format(data.decode()))
def connection_lost(self, exc):
print('server closed the connection')
asyncio.get_event_loop().stop()
loop = asyncio.get_event_loop()
coro = loop.create_connection(EchoClient, '127.0.0.1', 8888)
loop.run_until_complete(coro)
loop.run_forever()
loop.close()
The trick is to place your code (including self.transport methods) into a coroutine and use the wait_for() method, with the yield from statement in front of the statements that require their values returned, or ones which take a while to complete:
echo-server.py
#!/usr/bin/env python3.4
import asyncio
class EchoServer(asyncio.Protocol):
def connection_made(self, transport):
peername = transport.get_extra_info('peername')
print('connection from {}'.format(peername))
self.transport = transport
def data_received(self, data):
print('data received: {}'.format(data.decode()))
fut = asyncio.async(self.sleeper())
result = asyncio.wait_for(fut, 60)
#asyncio.coroutine
def sleeper(self):
yield from asyncio.sleep(2)
self.transport.write("Hello World".encode())
self.transport.close()
loop = asyncio.get_event_loop()
coro = loop.create_server(EchoServer, '127.0.0.1', 8888)
server = loop.run_until_complete(coro)
print('serving on {}'.format(server.sockets[0].getsockname()))
try:
loop.run_forever()
except KeyboardInterrupt:
print("exit")
finally:
server.close()
loop.close()
Call echo-server.py and then echo-client.py, the client will wait 2 seconds as determined by asyncio.sleep, then stop.
I've been working on a client for this chat server but I am running into a bit of a challenge. The server uses Python's 3.4RC1 asyncio module.
Behavior:
My client connects. My second client connects. Either can send messages to the server BUT, the server is not broadcasting them as it should in a normal public chat room.
User1: Hello. Presses Enter.
User2 does not see it.
User2: Anyone there? Presses Enter.
User2 sees User1: Hello. and User2: Anyone there?
Just... strange. Not sure what I'm missing.
Here are the files. Give it a try.
Server:
from socket import socket, SO_REUSEADDR, SOL_SOCKET
from asyncio import Task, coroutine, get_event_loop
class Peer(object):
def __init__(self, server, sock, name):
self.loop = server.loop
self.name = name
self._sock = sock
self._server = server
Task(self._peer_handler())
def send(self, data):
return self.loop.sock_send(self._sock, data.encode('utf-8'))
#coroutine
def _peer_handler(self):
try:
yield from self._peer_loop()
except IOError:
pass
finally:
self._server.remove(self)
#coroutine
def _peer_loop(self):
while True:
buf = yield from self.loop.sock_recv(self._sock, 1024)
if buf == b'':
break
self._server.broadcast('%s: %s' % (self.name, buf.decode('utf-8')))
class Server(object):
def __init__(self, loop, port):
self.loop = loop
self._serv_sock = socket()
self._serv_sock.setblocking(0)
self._serv_sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
self._serv_sock.bind(('',port))
self._serv_sock.listen(5)
self._peers = []
Task(self._server())
def remove(self, peer):
self._peers.remove(peer)
self.broadcast('Peer %s quit!' % (peer.name,))
def broadcast(self, message):
for peer in self._peers:
peer.send(message)
#coroutine
def _server(self):
while True:
peer_sock, peer_name = yield from self.loop.sock_accept(self._serv_sock)
peer_sock.setblocking(0)
peer = Peer(self, peer_sock, peer_name)
self._peers.append(peer)
self.broadcast('Peer %s connected!' % (peer.name,))
def main():
loop = get_event_loop()
Server(loop, 1234)
loop.run_forever()
if __name__ == '__main__':
main()
Client:
# import socket
from socket import *
# form socket import socket, bind, listen, recv, send
HOST = 'localhost' #localhost / 192.168.1.1
# LAN - 192.168.1.1
PORT = 1234
s = socket(AF_INET, SOCK_STREAM)# 98% of all socket programming will use AF_INET and SOCK_STREAM
s.connect((HOST, PORT))
while True:
message = input("Your Message: ")
encoded_msg = message.encode('utf-8')
s.send(encoded_msg)
print('Awaiting Reply..')
reply = s.recv(1024)
decoded_reply = reply.decode('utf-8')
decoded_reply = repr(decoded_reply)
print('Received ', decoded_reply)
s.close()
Here's the non threaded server code I wrote. works great but ONLY between 2 people. How could this code be updated to broadcast every message received to all clients connected?
# import socket
from socket import *
# form socket import socket, bind, listen, recv, send
HOST = 'localhost' #localhost / 192.168.1.1
# LAN - 192.168.1.1
PORT = 1234
s = socket(AF_INET, SOCK_STREAM) # 98% of all socket programming will use AF_INET and SOCK_STREAM
s.bind((HOST, PORT))
s.listen(5) # how many connections it can receive at one time
conn, addr = s.accept() # accept the connection
print('Connected by', addr) # print the address of the person connected
while True:
data = conn.recv(1024)
decoded_data = data.decode('utf-8')
data = repr(decoded_data)
print('Received ', decoded_data)
reply = input("Reply: ")
encoded_reply = reply.encode('utf-8')
conn.sendall(encoded_reply)
print('Server Started')
conn.close()
Okay, let’s think about what your client does. You ask for a message to send, blocking for user input. Then you send that message and receive whatever there is at the server. Afterwards, you block again, waiting for another message.
So when client A sends a text, client B is likely blocking for user input. As such, B won’t actually check if the server sent anything. It will only display what’s there after you have sent something.
Obviously, in a chat, you don’t want to block on user input. You want to continue receiving new messages from the server even if the user isn’t sending messages. So you need to separate those, and run both asynchronously.
I haven’t really done much with asyncio yet, so I don’t really know if this can be nicely done with it, but you essentially just need to put the reading and sending into two separate concurrent tasks, e.g. using threads or concurrent.futures.
A quick example of what you could do, using threading:
from socket import *
from threading import Thread
HOST = 'localhost'
PORT = 1234
s = socket(AF_INET, SOCK_STREAM)
s.connect((HOST, PORT))
def keepReading ():
try:
while True:
reply = s.recv(1024).decode()
print('Received ', reply)
except ConnectionAbortedError:
pass
t = Thread(target=keepReading)
t.start()
try:
while True:
message = input('')
s.send(message.encode())
except EOFError:
pass
finally:
s.close()
So I am working on an iPhone app that requires a socket to handle multiple clients for online gaming. I have tried Twisted, and with much effort, I have failed to get a bunch of info to be sent at once, which is why I am now going to attempt socket.
My question is, using the code below, how would you be able to have multiple clients connected? I've tried lists, but I just can't figure out the format for that. How can this be accomplished where multiple clients are connected at once and I am able to send a message to a specific client?
Thank you!
#!/usr/bin/python # This is server.py file
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 50000 # Reserve a port for your service.
print 'Server started!'
print 'Waiting for clients...'
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
c, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
while True:
msg = c.recv(1024)
print addr, ' >> ', msg
msg = raw_input('SERVER >> ')
c.send(msg);
#c.close() # Close the connection
Based on your question:
My question is, using the code below, how would you be able to have multiple clients connected? I've tried lists, but I just can't figure out the format for that. How can this be accomplished where multiple clients are connected at once and I am able to send a message to a specific client?
Using the code you gave, you can do this:
#!/usr/bin/python # This is server.py file
import socket # Import socket module
import thread
def on_new_client(clientsocket,addr):
while True:
msg = clientsocket.recv(1024)
#do some checks and if msg == someWeirdSignal: break:
print addr, ' >> ', msg
msg = raw_input('SERVER >> ')
#Maybe some code to compute the last digit of PI, play game or anything else can go here and when you are done.
clientsocket.send(msg)
clientsocket.close()
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 50000 # Reserve a port for your service.
print 'Server started!'
print 'Waiting for clients...'
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
print 'Got connection from', addr
while True:
c, addr = s.accept() # Establish connection with client.
thread.start_new_thread(on_new_client,(c,addr))
#Note it's (addr,) not (addr) because second parameter is a tuple
#Edit: (c,addr)
#that's how you pass arguments to functions when creating new threads using thread module.
s.close()
As Eli Bendersky mentioned, you can use processes instead of threads, you can also check python threading module or other async sockets framework. Note: checks are left for you to implement how you want and this is just a basic framework.
accept can continuously provide new client connections. However, note that it, and other socket calls are usually blocking. Therefore you have a few options at this point:
Open new threads to handle clients, while the main thread goes back to accepting new clients
As above but with processes, instead of threads
Use asynchronous socket frameworks like Twisted, or a plethora of others
Here is the example from the SocketServer documentation which would make an excellent starting point
import SocketServer
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print "{} wrote:".format(self.client_address[0])
print self.data
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
Try it from a terminal like this
$ telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello
HELLOConnection closed by foreign host.
$ telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Sausage
SAUSAGEConnection closed by foreign host.
You'll probably need to use A Forking or Threading Mixin too
This program will open 26 sockets where you would be able to connect a lot of TCP clients to it.
#!usr/bin/python
from thread import *
import socket
import sys
def clientthread(conn):
buffer=""
while True:
data = conn.recv(8192)
buffer+=data
print buffer
#conn.sendall(reply)
conn.close()
def main():
try:
host = '192.168.1.3'
port = 6666
tot_socket = 26
list_sock = []
for i in range(tot_socket):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind((host, port+i))
s.listen(10)
list_sock.append(s)
print "[*] Server listening on %s %d" %(host, (port+i))
while 1:
for j in range(len(list_sock)):
conn, addr = list_sock[j].accept()
print '[*] Connected with ' + addr[0] + ':' + str(addr[1])
start_new_thread(clientthread ,(conn,))
s.close()
except KeyboardInterrupt as msg:
sys.exit(0)
if __name__ == "__main__":
main()
def get_clients():
first_run = True
startMainMenu = False
while True:
if first_run:
global done
done = False
Thread(target=animate, args=("Waiting For Connection",)).start()
Client, address = objSocket.accept()
global menuIsOn
if menuIsOn:
menuIsOn = False # will stop main menu
startMainMenu = True
done = True
# Get Current Directory in Client Machine
current_client_directory = Client.recv(1024).decode("utf-8", errors="ignore")
# beep on connection
beep()
print(f"{bcolors.OKBLUE}\n***** Incoming Connection *****{bcolors.OKGREEN}")
print('* Connected to: ' + address[0] + ':' + str(address[1]))
try:
get_client_info(Client, first_run)
except Exception as e:
print("Error data received is not a json!")
print(e)
now = datetime.now()
current_time = now.strftime("%D %H:%M:%S")
print("* Current Time =", current_time)
print("* Current Folder in Client: " + current_client_directory + bcolors.WARNING)
connections.append(Client)
addresses.append(address)
if first_run:
Thread(target=threaded_main_menu, daemon=True).start()
first_run = False
else:
print(f"{bcolors.OKBLUE}* Hit Enter To Continue.{bcolors.WARNING}\n#>", end="")
if startMainMenu == True:
Thread(target=threaded_main_menu, daemon=True).start()
startMainMenu = False
#!/usr/bin/python
import sys
import os
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
port = 50000
try:
s.bind((socket.gethostname() , port))
except socket.error as msg:
print(str(msg))
s.listen(10)
conn, addr = s.accept()
print 'Got connection from'+addr[0]+':'+str(addr[1]))
while 1:
msg = s.recv(1024)
print +addr[0]+, ' >> ', msg
msg = raw_input('SERVER >>'),host
s.send(msg)
s.close()