I am working on an event-driven server in Rust and want it able to connect it to maximum no of clients possible.
But only up to 700 clients I am able to connect and send data at the same time. If I try to increase the clients, client connections from client side start breaking. I am using threads to create multiple clients, it starts giving the following error:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value:
Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }',
src/libcore/result.rs
If data sending is disabled it is able to connect 1017 clients and can be increased by allowing a process to open more files than 1024.
I checked the max. no of threads that can be created in my os is 62096. Memory usage of the server program is also around 15KB.
I tried to increase the event loop capacity to 20,480 still it is not supporting 800 clients.
Server.rs:
extern crate mio;
use mio::net::{TcpListener, TcpStream};
use mio::*;
use mio::{Poll, PollOpt, Ready, Token};
use std::collections::HashMap;
use std::env;
use std::io::{Error, Read, Write};
fn process_events(
e: Event,
listener: &TcpListener,
poll: &Poll,
clients: &mut HashMap<Token, TcpStream>,
mut count: usize,
) {
if e.token() == Token(0) {
match listener.accept() {
Ok((mut stream, addr)) => {
let new_token = Token(count);
println!("new token {:?}", new_token);
poll.register(
&stream,
new_token,
Ready::readable(),
PollOpt::edge() | PollOpt::oneshot(),
)
.unwrap();
println!("Got a client: {:?}", stream.peer_addr().unwrap());
clients.insert(new_token, stream);
}
Err(e) => panic!("Error during connection{}", e),
}
}
if e.token() != Token(0) && e.readiness().is_readable() {
let mut buf = [0; 512];
let mut t = e.token();
let reader = clients.get_mut(&e.token()).unwrap().read(&mut buf);
match reader {
Ok(_) => {
let bytes_no = reader.unwrap();
println!("No of bytes read : {:?}, {:?}", bytes_no, e.token());
}
Err(e) => {
println!("could not read: {}", e);
}
}
poll.reregister(
&clients[&e.token()],
e.token(),
Ready::readable(),
PollOpt::edge() | PollOpt::oneshot(),
)
.unwrap();
}
}
fn main() {
const server: Token = Token(0);
let args: Vec<String> = env::args().collect();
if args.len() != 2 {
eprintln!("Provide the argument");
std::process::exit(1);
}
let listen = &args[1];
let addr = listen.parse().unwrap();
let mut listener = TcpListener::bind(&addr).expect("Could not bind");
println!("Server listening on : {}", addr);
let mut clients = HashMap::new();
let mut eve = Events::with_capacity(20480);
let poll = Poll::new().unwrap();
let mut count = 1;
poll.register(
&listener,
server,
Ready::readable(),
PollOpt::edge() | PollOpt::oneshot(),
)
.unwrap();
loop {
poll.poll(&mut eve, None);
for e in &eve {
process_events(e, &listener, &poll, &mut clients, count);
if e.token() == Token(0) {
count += 1;
}
}
poll.reregister(
&listener,
server,
Ready::readable(),
PollOpt::edge() | PollOpt::oneshot(),
)
.unwrap();
}
}
Client.rs:
use std::io;
use std::io::Write;
use std::net::TcpStream;
use std::thread;
fn handle_client() {
let server_addr = String::from("localhost:3000");
if let Ok(mut stream) = TcpStream::connect(server_addr) {
println!("Connected to server {} ", c);
let msg = [0u8; 512];
loop {
stream.write(&msg).unwrap();
}
} else {
println!("could not connect");
}
}
fn main() {
for _i in 0..10000 {
thread::spawn(move || handle_client());
}
}
As the event-driven approach is better than the multi-threaded server, expected clients should more than 1020.
Related
I have to make a VPN connection with protocol IKEv2 using a certificate.
I already have the Network Extensions entitlements.
Starting with the certificate that i'm using:
I request that certificate with an ACS request that returns me the UserCertificate in Base64 that i need to send with the VPN connection.
Example User Certificate in Base64 received:
"MIIFqTCCA5GgAwIBAgIQdwPEs5oyDHNNp7e2ZyhwADANBgkqhkiG9w0 ... N972fR4pncgdIA=="
In order to pass that certificate in the "identityData" parameter i have to convert this to Data with the PKCS12 format as i understand.
This is my conversion from Base64 to Data:
certificate = SecCertificateCreateWithData(nil, Data(base64Encoded: "Base64_String_Here")! as CFData)!
certificateData = SecCertificateCopyData(certificate!) as Data
I'm able to print the Data as NSData and i get this example:
308205a9 30820391 a0030201 02021077 03c4b39a 320c734d a7b7b667 28700030 0d06092a 864886f7 0d01010b 05003060
311c301a 06035504 030c1354 45535420 5646435a 20455043 20537562 4341310c 300a0603 55040b0c 03455043 31253023
06035504 0a0c1c56 6f646166 6f6e6520 437a6563 68205265 7075626c 69632061 2e732e31 0b300906 03550406 1302435a
301e170d 31373131 31373131 32393531 5a170d31 39313131 38313132 3935315a 30818731 43304106 03550403 0c3a3467
66693233 30303339 39303030 30313433 31403467 66692e65 70632e6d 6e633030 332e6d63 63323330 2e336770 706e6574
776f726b 2e6f7267 310c300a 06035504 0b0c0345 50433125 30230603 55040a0c 1c566f64 61666f6e 6520437a 65636820
52657075 626c6963 20612e73 2e310b30 09060355 04061302 435a3082 0122300d 06092a86 4886f70d 01010105 00038201
0f003082 010a0282 010100cc 5effc8d7 fb357559 678fb8fc 48fd3558 8af4f236 a27295a6 b484bad0 921f8077 35ddbceb
5c064492 2f9df462 2a44c4fc 85b80cfa eec6b84b b13aefa1 69e4e6b3 db8ef30b 202229c7 421bfb8f 3f47c24f 6e3f7abc
ca7d216b 31d2a02c 94c3ef6c 9d50c2ab a590d99f 5d7afddb acbcc045 53949653 97034063 404cbe00 6aaec239 b37b2ce9
5c63d126 127247a5 75446c6b fa2e2217 6706a36c a0db0630 dac90d10 00ff24a8 2f189f18 37233151 f9825f60 aa8f66e7
89d656b9 5a52d9e3 b52cb3bc 5badb801 d9ea3685 8930cb3b 490ccba0 247f4bde 00b46d5e a63b1018 4ad8a581 978f5570
82f06f7f ba60db83 abcb4391 2f221e69 14bbdc46 ce19ae95 33773f02 03010001 a3820135 30820131 30370608 2b060105
05070101 042b3029 30270608 2b060105 05073001 861b6874 74703a2f 2f6f6373 702d6570 632e766f 6461666f 6e652e63
7a301d06 03551d0e 04160414 ea45a2e8 94fe37bc 08af3fab 60f9869b 5980d3cf 30090603 551d1304 02300030 1f060355
1d230418 30168014 f3c6b0f4 961565ab 232c6060 3e16f790 f63334b1 30440603 551d2004 3d303b30 39060960 86480186
f8420108 302c302a 06082b06 01050507 0201161e 68747470 3a2f2f70 6b692e76 6f646166 6f6e652e 637a2f70 6f6c6963
792f3040 0603551d 1f043930 373035a0 33a03186 2f687474 703a2f2f 706b692e 766f6461 666f6e65 2e637a2f 63726c2f
74657374 5f766663 7a5f6570 635f6361 2e63726c 300e0603 551d0f01 01ff0404 030203f8 30130603 551d2504 0c300a06
082b0601 05050703 02300d06 092a8648 86f70d01 010b0500 03820201 004cacde 1d7b7843 43aa546c 8984e308 57a12a27
0b9a2873 2adb598d da29fe4c ee087c92 3f18bbaa 1f3ef0b9 39e18d50 db982f87 36689c03 07ab7d11 a1665bf0 f6057de5
e89fe2fe 3f9c476b 8a28749c f9d7636c 8ece6012 d0a53902 ec437171 0af85024 0963bf85 518fa150 43dc7fe8 30c42f09
62cd2a31 c38e54ba de2b8a01 6e6804b2 b66c36c1 4626fc37 65064a65 d5087412 8e541e12 51525c4c e1f4a7e2 94a31397
2806d327 7a3c65c7 50e7db59 99460ae9 a446391e 5bbce05e 1f205e1c 118cd7aa 02e9881f 75b5b40d f2aa09bc bb7abf22
b11b3e31 2f0e78ae ee497d26 ef314b1d f16008af c433a653 983db0a9 97cbbca1 5a9e8d4d 3d34ee87 858e4f3d d2c5c111
05ccdcfa b1a41afe 2624ba31 decd561a 791e97e0 197b8bfe 97ebb3ad 7b511c1d e820c83f 81cdb9ed 21d59b49 7fb0bb39
69e735e7 350af763 ab26ee3d 7e954ef7 e5c994f6 c8f104ac 4f19ed25 9855f14c 619935d6 0d9f1206 910ffe03 74c312d5
b39da039 650d9004 e2186914 cfa19052 a2e47f1c 5af90903 959ab99f 2c52a9d8 f737b59f 00cababa 0b4f7236 e9102ee4
a76ed0a0 6d827860 653d6e62 f52fcc77 d9afdc90 53d942fe 434c8901 ab2a93e8 cc7c8555 eb6a94e5 0e1cc18c 9d6c8b48
ed9204eb 623f74d2 219a5ca9 032caa7c 900ed208 b88f5fb2 5c53bcd3 e82c48f8 80450f40 f05706c1 072427cb 815e8590
4fbf5137 def67d1e 299dc81d 20
So, basically the var certificateData is the one that i'm using in the parameter "identityData".
Here is what i do in order to connect the VPN:
First i set all the parameters that are required:
vpnManager = NEVPNManager.shared()
vpnManager!.loadFromPreferences { error in
var hasProtocolConfig = false;
if #available(iOS 9, *) {
hasProtocolConfig = self.vpnManager!.protocolConfiguration != nil
} else {
hasProtocolConfig = self.vpnManager!.`protocol` != nil
}
if hasProtocolConfig == true {
let p = NEVPNProtocolIKEv2()
p.serverAddress = "X.X.X.X"
p.localIdentifier = "My_Local_Identifier"
p.remoteIdentifier = "My_Remote_Identifier"
p.identityData = certificateData
p.ikeSecurityAssociationParameters.integrityAlgorithm = NEVPNIKEv2IntegrityAlgorithm.SHA256
p.ikeSecurityAssociationParameters.encryptionAlgorithm = NEVPNIKEv2EncryptionAlgorithm.algorithmAES128
p.ikeSecurityAssociationParameters.diffieHellmanGroup = NEVPNIKEv2DiffieHellmanGroup.group14
p.serverCertificateIssuerCommonName = "My_IssuerCommonName"
p.serverCertificateCommonName = "My_CommonName"
p.authenticationMethod = NEVPNIKEAuthenticationMethod.certificate
p.useExtendedAuthentication = true
p.disconnectOnSleep = false
if #available(iOS 9, *) {
self.vpnManager.protocolConfiguration = p
} else {
self.vpnManager.`protocol` = p
}
self.vpnManager.isEnabled = true
self.vpnManager.saveToPreferences { error in
if let e = error{
print("[VPN] error saving: " + e.localizedDescription)
} else {
print("[VPN] vpn saved")
Timer.scheduledTimer(timeInterval: 5, target: self, selector: #selector(self.connectVPN), userInfo: nil, repeats: false)
}
return
}
Then i establish the VPN connection:
do {
try vpnManager!.connection.startVPNTunnel()
} catch let error {
print("Error starting VPN Connection \(error.localizedDescription)");
}
If i understand this well, Since i am not able to force the "Notify: EAP_ONLY_AUTHENTICATION", i have to use "NEVPNIKEAuthentication.certificate", otherwise i could set it to ".none".
The ikeAssociationParameters are the right ones as i check with the server side.
When i try to connect to the VPN i get the status:
Connecting -> Disconnected
Any ideia of what i am doing wrong with this connection?
What are the intentions of this program:
I want to send some commands from a client to a server using sockets, the server then send these command to an Arduino using serial. And another thing that I want the server to do in the future is that periodically sends other commands to the Arduino without getting any input from the client, so the sockets needs to be non-blocking or there needs to be another way to run the code separately from the sockets code.
The problem is that the part that is supposed to send the command to the Arduino only runs once.
What I have come up with after playing with the debugger in Pycharm, is that the problem is that the following line blocks after a connection has been established, and thus not allowing the rest of the code to be run.
conn, addr = s.accept()
Is this correct, or is there something else wrong?
I have tried to set the socket to non-blocking but when I do this I get an error.
"BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately"
I have some basic knowledge of C/C++ and C# and am new to Python.
server.py
import socket
import serial
import sys
from _thread import *
import threading
import queue
# command that the client sends are "ON" and "OFF"
class serialConnect:
comPort =' '
baudrate = 115200
myserial = serial.Serial('COM5', baudrate)
def serialstart(self):
# self.comPort = input('Comport: ')
try:
self.myserial.open()
except IOError:
print('Port is already open!')
def serialRead(self):
data = self.myserial.read(16)
data.decode('UTF-8')
return data
def serialWrite(self, data):
data += '\n' #the arduino needs a \n after each command.
databytes = data.encode('UTF-8')
self.myserial.write(databytes)
print('send data: ', databytes)
def threaded_client(conn, dataqueue):
data = {bytes}
conn.send(str.encode('welcome, type your info \n'))
while True:
data = conn.recv(2048)
if not data:
break
reply = 'server output: ' + data.decode('UTF-8')
dataqueue.put(data.decode('UTF-8'))
print("Items in queue: ",dataqueue.qsize())
#conn.sendall(str.encode(reply))
print("Recieved data in threaded_client: ", data.decode('UTF-8') + '\n')
conn.close()
def Main():
ser = serialConnect()
host = ''
port = 5555
dataRecieved = 'hello'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
s.setblocking(1) #when set to non-blocking error occurs : "BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately"
workQueue = queue.Queue(10)
try:
s.bind((host,port))
except socket.error as e:
print(str(e))
s.listen(5)
print('waiting for a connection')
while True:
try:
conn, addr = s.accept() #once connection is established it blocks?
print('connected to: ' + addr[0] + ':' + str())
t = threading.Thread(target=threaded_client, args=(conn, workQueue))
t.daemon = True
t.start()
except:
e = sys.exc_info()
print('Error:', e)
# This section of code is only run once, doesn't matter if put inside try block or not. :(
dataRecieved = workQueue.get()
print('The recieved data: ', dataRecieved)
ser.serialstart()
ser.serialWrite(dataRecieved)
if __name__ == '__main__':
Main()
client.py
import socket
def Main():
host = '127.0.0.1'
port = 5555
message = "<,R,G,B,>"
mySocket = socket.socket()
mySocket.connect((host, port))
while message != 'q':
message = input(" -> ")
mySocket.send(message.encode())
mySocket.close()
if __name__ == '__main__':
Main()
Arduino Code
String inputString = ""; // a string to hold incoming data
boolean stringComplete = false; // whether the string is complete
int LEDpin = 10;
// the setup function runs once when you press reset or power the board
void setup() {
// initialize digital pin 13 as an output.
pinMode(10, OUTPUT);
Serial.begin(19200);
}
// the loop function runs over and over again forever
void loop() {
serialEvent();
if(stringComplete){
Serial.println(inputString);
if(inputString == "ON\n"){
digitalWrite(LEDpin, HIGH); // turn the LED on (HIGH is the voltage level)
}
if(inputString == "OFF\n"){
digitalWrite(LEDpin, LOW); // turn the LED off by making the voltage LOW
}
inputString = "";
stringComplete = false;
}
}
void serialEvent()
{
while (Serial.available()) {
// get the new byte:
char inChar = (char)Serial.read();
// add it to the inputString:
inputString += inChar;
// if the incoming character is a newline, set a flag
// so the main loop can do something about it:
if (inChar == '\n') {
stringComplete = true;
}
}
}
Refactored server code for anyone that is interested in it.
I am not sure if this is up to standard, but it is working.
import serial
import socket
import queue
import sys
import threading
class serialConnect:
comPort = 'COM5'
baudrate = 115200
myserial = serial.Serial(comPort, baudrate)
def serial_run(self):
# self.comPort = input('Comport: ')
try:
if not self.myserial.isOpen():
self.myserial.open()
else:
print('Port is already open!')
except IOError as e:
print('Error: ', e)
def serial_read(self):
data = self.myserial.read(16)
data.decode('UTF-8')
return data
def serial_write(self, data):
data += '\n' #the arduino needs a \n after each command.
databytes = data.encode('UTF-8')
self.myserial.write(databytes)
print('send data: ', databytes)
class socketServer:
host = ''
port = 5555
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.setblocking(1)
data_queue = queue.Queue(1)
def __init__(self):
try:
self.soc.bind((self.host, self.port))
except:
print('Bind error: ', sys.exc_info())
self.soc.listen(5)
def socket_accept_thread(self):
while True:
try:
print('Waiting for a new connection')
conn, addr = self.soc.accept()
client_thread = threading.Thread(target=self.threaded_client, args=(conn, self.data_queue))
client_thread.daemon = True
client_thread.start()
except:
print('Accept thread Error: ', sys.exc_info())
def threaded_client(self, conn, data_queue):
# conn.send(str.encode('welcome, type your info \n'))
try:
while True:
data = conn.recv(2048)
if not data:
break
# reply = 'server output: ' + data.decode('UTF-8')
data_queue.put(data.decode('UTF-8'))
print("Items in queue: ", data_queue.qsize())
# conn.sendall(str.encode(reply))
print("Received data in threaded_client: ", data.decode('UTF-8'))
except:
print("Error: ", sys.exc_info())
conn.close()
def get_data(self):
data = self.data_queue.get()
return data
def Main():
server = socketServer()
arduino_conn = serialConnect()
accept_thread = threading.Thread(target=server.socket_accept_thread)
data_received = 'Nothing received'
while True:
if not accept_thread.is_alive():
accept_thread.daemon = True
accept_thread.start()
arduino_conn.serial_run()
data_received = server.get_data()
arduino_conn.serial_write(data_received)
if __name__ == '__main__':
Main()
I trying to connect to Google with OAuth2. I have got access_token and user email. Here is not problem.
let singleton: Singleton = Singleton.sharedInstance
singleton.session.authType = MCOAuthType.XOAuth2 // session:IMAPSession
singleton.session.OAuth2Token = singleton.accessToken
singleton.session.username = singleton.email
singleton.session.hostname = "imap.gmail.ru"
singleton.session.port = 993
singleton.session.connectionType = MCOConnectionType.StartTLS
Next step i create fetch operation and get an error
let requestKind = MCOIMAPMessagesRequestKind.Headers | MCOIMAPMessagesRequestKind.Flags | MCOIMAPMessagesRequestKind.Structure
let uids = MCOIndexSet(range: MCORangeMake(1, UINT64_MAX))
let folder = "INBOX"
let fetchOperation = singleton.session.fetchMessagesOperationWithFolder(folder, requestKind: requestKind, uids: uids)
fetchOperation.start { (error, fetchedMessages, vanishedMessages) -> Void in
if (error != nil)
{
println("Error: \(error)") // this line return me error
}
Error: Error Domain=MCOErrorDomain Code=1 "A stable connection to the server could not be established." UserInfo=0x7f9e260036b0 {NSLocalizedDescription=A stable connection to the server could not be established.}
Error: Error Domain=MCOErrorDomain Code=1 "A stable connection to the server could not be established." UserInfo=0x7f9e23c46c00 {NSLocalizedDescription=A stable connection to the server could not be established.}
what could be the problem?
Here's the working code guys:
var smtpSession = MCOSMTPSession()
smtpSession.hostname = "xxxxxxxx"
smtpSession.username = "xxxxxxx"
smtpSession.password = "xxxxxxxx"
smtpSession.port = 8465
smtpSession.authType = MCOAuthType.SASLPlain
smtpSession.connectionType = MCOConnectionType.TLS
smtpSession.connectionLogger = {(connectionID, type, data) in
if data != nil {
if let string = NSString(data: data, encoding: NSUTF8StringEncoding){
NSLog("Connectionlogger: \(string)")
}
}
}
var builder = MCOMessageBuilder()
builder.header.to = [MCOAddress(displayName: "Rool", mailbox: "xxxxxxx#hxxxx.com")]
builder.header.from = MCOAddress(displayName: "Matt R", mailbox: "xxxxxx#sxxxxxx.com")
builder.header.subject = "My message"
builder.htmlBody = "<h3>This is a test message!</h3>"
let rfc822Data = builder.data()
let sendOperation = smtpSession.sendOperationWithData(rfc822Data)
sendOperation.start { (error) -> Void in
if (error != nil) {
NSLog("Error sending email: \(error)")
} else {
NSLog("Successfully sent email!")
}
}
PS: I used this code for SMTP2GO and It's working flawlessly, just make sure you're using the right port.
How can I close a WebSocket connection? The example on the documentation works if you want to close it immediately.
But how about the following case: I want to close the connection when some condition occurs in the future. For instance, when I receive a certain message from the client.
def indexWS = WebSocket.using[String] {
request => {
var channel: Option[Concurrent.Channel[String]] = None
var outEnumerator: Enumerator[String] = Concurrent.unicast(c => channel = Some(c))
val myIteratee: Iteratee[String, Unit] = Iteratee.foreach[String] {gotString => {
// received a string from the client
if (gotString == "close_me") {
// outEnumerator = Enumerator.eof // doesn't work
// outEnumerator >>> Enumerator.eof // doesn't work
}
}}
(myIteratee, outEnumerator)
}
}
Thank you for your help!
I got it: I had to go through the channel that I opened at
var outEnumerator: Enumerator[String] = Concurrent.unicast(c => channel = Some(c))
and the commented out block would become
if (gotString == "close_me") {
channel.foreach(_.eofAndEnd())
}
which will push an EOF through the enumerator and close the connection.
I am writing a program in blackberry curve 9300 using socketConnection. I get stream closed/bad socket id error when I try to read the inputstream.
Can you suggest me what could be the issue.
String url = Constants.SOCKET + serverIp + Constants.COLON + serverPort + UserInfo.getConnectionType();
//connection open
socketConne`enter code here`ction = (SocketConnection) Connector.open(url, Connector.READ_WRITE,true);
socketConnection.setSocketOption(SocketConnection.DELAY, 0);
socketConnection.setSocketOption(SocketConnection.KEEPALIVE, 1);
socketConnection.setSocketOption(SocketConnection.RCVBUF, 16834);
socketConnection.setSocketOption(SocketConnection.LINGER, 10);
outputStream = socketConnection.openDataOutputStream();
//Output stream deligated to Sender Object to send the command
sender = new Sender(outputStream);
sender.send(command);
inputStream = socketConnection.openDataInputStream();
//read inputstream
while (tobeRead < toRead && retryCount < 5) {
try {
if ((toRead - tobeRead) > 8192) {
readBufferLength = 8192;
} else {
readBufferLength = (toRead - tobeRead);
}
buffer = new byte[readBufferLength + 1];
bytesRead = inputStream.read(buffer, 0, readBufferLength);
sb.append(new String(buffer, 0, bytesRead));
tobeRead += bytesRead;
totalBytesRead += bytesRead;
} catch (Exception cce) {
}
}
say if number of bytes to read is 5192. I get say 2123 bytes then it iterates the loop, next time while reading it fails. I run this in seperate thread. I use WIFI network for connectivity