data exchange using socket between Julia and Groovy - sockets

Recently I'm trying to use a socket to exchange data between a solver in Julia as a server and a client in Groovy. Im new to the socket concept and both the language. The thing is that it seems that once julia server done 'readLine()', the socket close and the 'write()' cannot proceed. I cannot send messages to the Groovy client nor exchange data.
Server code(Julia):
using Sockets
server = listen(2000)
while true
conn = accept(server)
println("connection accepted")
#async begin
try
line = readlines(conn)
println(line)
str = "server"
write(conn,str)
println("write done")
sleep(2)
catch err
print("connection ended with error $err")
sleep(2)
end
end
end
Client code(Groovy):
class groovy_client {
static void main(String[] args) {
while (true) {
try {
dataTransfer()
println 'dataTransfer success'
}catch (e) {
println(e)
sleep(2000)
}
}
}
private static void dataTransfer() {
Socket s = new Socket('localhost', 2000)
println 'connect to the server'
s.withStreams { input, output ->
// BufferedReader reader = new BufferedReader(new InputStreamReader(input))
// str = reader.readLine()
// println(str)
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(output))
writer.write('Hi server')
writer.flush()
input.withReader{reader ->
str = reader.readLine()
println 'haha'
println(str)
// def reader = input.newReader()
// def buffer = reader.readLine()
// println "server: $buffer"
sleep(2000)
}
println 'disconnect'
}
}
}
And I also have a doubt about how the socket works. Do server and client send and receive messages at the same time? Like server tells the client to add 1 to int_a while client send int_a back to the sender. Does int_a remain the original value or with 1 added and sent to the server?

Related

How to have multiple readers and writers using TCP streams (in the manner of Pub/Sub)?

A TCP Stream represents a connection between one local and one remote socket, but I want to create an API that gives the impression that the user can write to one local socket, and have it read by multiple remote sockets, and also create multiple local sockets that are all reading from one remote socket. À la Pub/Sub (See the diagram here)
The tests
The tests show how I want my API to be used, and the expected behaviour. The readers and writers should also work in the same way if created in separate processes.
#[test]
fn single_writer_single_reader() {
//reader 1 should print "hello world"
create_reader(|s| {println!("reader 1 {}", s)});
let mut writer = create_writer();
writer.write_all(b"hello world").unwrap();
thread::sleep(Duration::from_millis(1000));
}
#[test]
fn multi_writer_single_reader() {
//reader 1 should print "hello" twice
create_reader(|s| {println!("reader 1 {}", s)});
let mut writer1 = create_writer();
let mut writer2 = create_writer();
writer1.write_all(b"hello").unwrap();
writer2.write_all(b"hello").unwrap();
thread::sleep(Duration::from_millis(1000));
}
#[test]
fn single_writer_multi_reader() {
//each reader should print "hello world"
create_reader(|s| {println!("reader 1 {}", s)});
create_reader(|s| {println!("reader 2 {}", s)});
let mut writer = create_writer();
writer.write_all(b"hello world").unwrap();
thread::sleep(Duration::from_millis(1000));
}
My implementation
create_reader starts a new thread running a server (referred to as the "writer listener") on WRITER_LISTENER_ADDR that continuously accepts all connections and creates a new reader thread for each connection. The reader threads will wait for the "reader listener" to accept the reader, and then will continuously read from the writer, and call the callback function with each received message.
pub fn create_reader(callback: fn(String)) {
std::thread::spawn(move || {listen_for_writers_and_make_readers(callback)});
}
fn listen_for_writers_and_make_readers(callback: fn(String)) {
//listen for remote sockets that are connecting to WRITER_LISTENER_ADDR.
let writer_listener = TcpListener::bind(WRITER_LISTENER_ADDR).expect("writer_listener: failed to bind");
println!("writer_listener: started on {}", WRITER_LISTENER_ADDR);
loop {
let (stream, _) = writer_listener.accept().unwrap();
println!("writer_listener: New writer connected");
let _reader_thread = std::thread::spawn( move || {read_loop(stream, callback)});
}
}
fn read_loop(stream: TcpStream, callback: fn(String)) {
let mut incoming_stream = stream; //stream from the writer
//wait for the reader listener to accept this reader (this is to signal the reader listener that a new reader is created)
TcpStream::connect(READER_LISTENER_ADDR).expect("reader: failed to connect to reader listener");
println!("reader: connected to reader listener");
println!("reader: starting read loop");
loop {
let mut buf = [0; 1024];
match incoming_stream.read(&mut buf) {
Ok(0) => {
println!("reader: writer disconnected");
break;
},
Ok(_) => {
let msg = String::from_utf8(buf.to_vec()).unwrap();
println!("reader: calling callback");
callback(msg);
}
Err(e) => {
println!("{}", e);
break;
}
}
}
}
create_writer creates a TcpStream (known as writer) and waits for the "writer listener" to accept this connection. It then starts a new thread running a server (known as the "reader listener") on READER_LISTENER_ADDR that continuously accepts all connections and clones the writer for each one. Calling write() on writer will send the message to all readers because each cloned writer is connected to a reader, and writes to the original are propagated to the clones.
fn create_writer() -> TcpStream {
//wait for the writer listener to accept this writer
let writer = TcpStream::connect(WRITER_LISTENER_ADDR).expect("writer: failed to connect to writer listener");
println!("writer: connected to the writer listener");
//create a thread that listens for readers and clones the writer
let writer_clone = writer.try_clone().unwrap();
std::thread::spawn( move || {listen_for_readers_and_clone_writer(&writer_clone)});
return writer
}
fn listen_for_readers_and_clone_writer(writer: &TcpStream) {
//listen for remote sockets that are connecting to READER_LISTENER_ADDR
let reader_listener = TcpListener::bind(READER_LISTENER_ADDR).expect("reader_listener: failed to bind");
println!("reader_listener: started on {}", READER_LISTENER_ADDR);
let mut writers = vec![];
loop {
let (_stream, _) = reader_listener.accept().unwrap();
println!("reader_listener: New reader connected");
writers.push(writer.try_clone().unwrap());
}
}
The problem
The single_writer_single_reader test passes, but for the other tests, I get the errors:
For multi_writer_single_reader:
reader_listener: failed to bind: Os { code: 10048, kind: AddrInUse, message: "Only one usage of each socket address (protocol/network address/port) is normally permitted." }
For single_writer_multi_reader:
writer_listener: failed to bind: Os { code: 10048, kind: AddrInUse, message: "Only one usage of each socket address (protocol/network address/port) is normally permitted." }
Is there any way to allow multiple TcpListeners to bind to the same address? And if not, what would be the solution to this?
edit
I found a partial solution by giving the listeners an address range to pick from:
#[derive(Clone, Copy)]
struct AddrRange([&'static str; 10]);
static WRITER_LISTENER_ADDR_RANGE : AddrRange = AddrRange([
"127.0.0.1:80",
"127.0.0.1:81",
"127.0.0.1:82",
"127.0.0.1:83",
"127.0.0.1:84",
"127.0.0.1:85",
"127.0.0.1:86",
"127.0.0.1:87",
"127.0.0.1:88",
"127.0.0.1:89"
]);
static READER_LISTENER_ADDR_RANGE : AddrRange = AddrRange([
"127.0.0.1:90",
"127.0.0.1:91",
"127.0.0.1:92",
"127.0.0.1:93",
"127.0.0.1:94",
"127.0.0.1:95",
"127.0.0.1:96",
"127.0.0.1:97",
"127.0.0.1:98",
"127.0.0.1:99"
]);
Test 2 now passes, but test 3 does not create a new connection for the cloned writer as I thought it would. How can I get the writer_listener to detect that the writer has been cloned, and start a new read_loop thread in response to this?

Jmeter - Force close a socket/wait until message recieved

I am opening a socket in jmeter (using groovy in JSR223 Sampler), and storing the message in a jmeter variable. This is the below code:
SocketAddress inetSocketAddress = new InetSocketAddress(InetAddress.getByName("localhost"),4801);
def server = new ServerSocket()
server.bind(inetSocketAddress)
while(!vars.get("caseId"))) {
server.accept { socket ->
log.info('Someone is connected')
socket.withStreams { input, output ->
InputStreamReader isReader = new InputStreamReader(input);
BufferedReader reader = new BufferedReader(isReader);
StringBuffer sb = new StringBuffer();
String str;
while((str = reader.readLine())!= null){
sb.append(str);
}
String finalStr = sb.toString()
String caseId = finalStr.split("<caseId>")[1].split("</caseId>")[0]
vars.put("caseId", caseId)
}
log.info("Connection processed")
}
}
if(vars.get("caseId"))
{
try
{
server.close();
vars.put("socketClose",true);
}
catch(Exception e)
{
log.info("Error in closing the socket: " + e.getMessage());
}
}
Now, there is some time delay between the first loop is executed and the message being recieved from the port. It doesnt receive the message immediately, and hence while loop is executed again. And then message is received and it sets caseId. It goes on to close the socket, because caseId is set. And that is throwing the error, because socket is still waiting for the message. So is there a way, to wait until socket has recieved all the messages, so i could properly close it?
Or just force close the socket, and Jmeter wont throw any exception?
Or when i execute next component, say IF controller in Jmeter, it waits until variable socketClose is set true? In that way, instead of while loops inside JSR223 sampler, i could use multiple If Controllers in Jmeter thread.
This is how ServerSocket.close() function works
public void close()
throws IOException
Closes this socket. Any thread currently blocked in accept() will throw a SocketException.
I don't think there is a way "to wait until socket has recieved all the messages" because Socket is dump as a rock and it can either listen for connections or shut down.
Maybe you might be interested in setSoTimeout() function?
Also this line:
vars.put("socketClose",true)
is very suspicious, I think you need to change it either to:
vars.put("socketClose", "true")
or to
vars.putObject("socketClose",true)
as JMeterVariables.put() function can accept only a String, see Top 8 JMeter Java Classes You Should Be Using with Groovy article for more details.

Matching a String in a text file, and return back the meaning of the word in the client part (Scala)

I would like to solve a problem where the client will send a word to the server, and server will run through the text file and return the meaning of the word that is typed by the user. My problem is when a matching word is typed, i will return the match word meaning, at the same time return back other "Word not found" (which is my control statement). So i would like to just return the meaning of the word only if the word is matched and found
Heres my code:
Server part
Future {
//store local socket references for processing
val client = socket
try {
// Get a communication stream associated with the socket
val is = new BufferedReader(new InputStreamReader(client.getInputStream))
// Get a communication stream associated with the socket
val os = new PrintStream(client.getOutputStream)
val inFile = new File(filename)
val readFile = new Scanner(inFile)
var input : String = null;
while (true) {
// Read from input stream
var line: String = is.readLine() // read the word from client
println("The client send " + line)
while (readFile.hasNext()) {
input = readFile.nextLine()
println("From the file " + input)
if (input.contains(line)) {
os.println(input)
os.flush()
}
else{
os.println("Word not found")
}
}
}
} catch {
case e: Exception => e.printStackTrace
} finally {
// Close the connection, but not the server socket
client.close()
}
}
Client Part
val is = new BufferedReader(new InputStreamReader(client.get.getInputStream))
val os = new PrintStream(client.get.getOutputStream) // write to server a strin
println("Please Input Ur word ")
val user = readLine
os.println(user)
while (true) {
var line: String = is.readLine() //receive string from server
println(line)
}
My text file is formatted this way:
super-very good or pleasant; excellent.
Use the following approach:
var found = false;
while (readFile.hasNext() && !found) {
input = readFile.nextLine()
println("From the file " + input)
if (input.contains(line)) {
os.println(input)
os.flush()
found = true;
}
}
if(!found) {
os.println("Word not found")
os.flush()
}
There are much more beautiful ways to do this, but as you seem to be a beginner, this is the simplest solution possible.
If you are interested in a functional approach, I will gladly help.
Hope this helps.

Creating a Chat With Server Socket Channel

I'm creating a chat with ServerSocketChannel and maintain communication between Clients - Server.
The server receives a message from the client and broadcasts to every client.
I tried to send a message to the server and everything was fine, but when I try to send a message to the client from the server the message doesn't get there. It only delivers when I close the socket. (It's like it was buffered)
Here's my code for the server:
static public void main( String args[] ) throws Exception {
// Parse port from command line
int port = Integer.parseInt( args[0] );
try {
// Instead of creating a ServerSocket, create a ServerSocketChannel
ServerSocketChannel ssc = ServerSocketChannel.open();
// Set it to non-blocking, so we can use select
ssc.configureBlocking( false );
// Get the Socket connected to this channel, and bind it to the
// listening port
ServerSocket ss = ssc.socket();
InetSocketAddress isa = new InetSocketAddress( port );
ss.bind( isa );
// Create a new Selector for selecting
Selector selector = Selector.open();
// Register the ServerSocketChannel, so we can listen for incoming
// connections
ssc.register( selector, SelectionKey.OP_ACCEPT );
System.out.println( "Listening on port "+port );
while (true) {
// See if we've had any activity -- either an incoming connection,
// or incoming data on an existing connection
int num = selector.select();
// If we don't have any activity, loop around and wait again
if (num == 0) {
continue;
}
// Get the keys corresponding to the activity that has been
// detected, and process them one by one
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> it = keys.iterator();
while (it.hasNext()) {
// Get a key representing one of bits of I/O activity
SelectionKey key = it.next();
// What kind of activity is it?
if ((key.readyOps() & SelectionKey.OP_ACCEPT) ==
SelectionKey.OP_ACCEPT) {
// It's an incoming connection. Register this socket with
// the Selector so we can listen for input on it
Socket s = ss.accept();
clientList.add(new Client(s));
System.out.println( "Got connection from "+s );
// Make sure to make it non-blocking, so we can use a selector
// on it.
SocketChannel sc = s.getChannel();
sc.configureBlocking( false );
// Register it with the selector, for reading
sc.register( selector, SelectionKey.OP_READ );
} else if ((key.readyOps() & SelectionKey.OP_READ) ==
SelectionKey.OP_READ) {
SocketChannel sc = null;
try {
// It's incoming data on a connection -- process it
sc = (SocketChannel)key.channel();
boolean ok = processInput( sc );
/* HERE, TRYING TO SEND A TEST MESSAGE BACK */
ByteBuffer bf = ByteBuffer.allocate(48);
bf.clear();
bf.put("testmessage".getBytes());
sc.write(bf);
// If the connection is dead, remove it from the selector
// and close it
if (!ok) {
key.cancel();
Socket s = null;
try {
s = sc.socket();
System.out.println( "Closing connection to "+s );
s.close();
} catch( IOException ie ) {
System.err.println( "Error closing socket "+s+": "+ie );
}
}
} catch( IOException ie ) {
// On exception, remove this channel from the selector
key.cancel();
try {
sc.close();
} catch( IOException ie2 ) { System.out.println( ie2 ); }
System.out.println( "Closed "+sc );
ie.printStackTrace();
}
}
}
// We remove the selected keys, because we've dealt with them.
keys.clear();
}
} catch( IOException ie ) {
System.err.println( ie );
}
}
On the server, please note the lines:
ByteBuffer bf = ByteBuffer.allocate(48);
bf.clear();
bf.put("testmessage".getBytes());
sc.write(bf);
This is where I try to answer back.
Client receives the messages in a method:
// Main method for incoming
public void run() throws IOException {
while(true) {
InputStream is = con.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String answer = br.readLine();
printMessage(answer);
}
}
Regards,
Pedro
Several problems here.
You're reading lines but you're not writing lines. Add a line terminator when sending.
You need to close the channel immediately you get -1 from read(). You almost certainly can't send on it after that.
You don't need to cancel the key or close the socket of the channel. Closing the channel does all that.
Your client read loop needs to break when you get null from readLine().

Cloud Service for incoming TCP connections hangs

I'm developing a cloud service (worker role) for collecting data from a number of instruments. These instruments reports data randomly every minute or so. The service itself is not performance critical and doesn't need to be asynchronous. The instruments are able to resend their data up to an hour on failed connection attempt.
I have tried several implementations for my cloud service including this one:
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.stop(v=vs.110).aspx
But all of them hang my cloud server sooner or later (sometimes within an hour).
I suspect something is wrong with my code. I have a lot of logging in my code but I get no errors. The service just stops to receive incoming connections.
In Azure portal it seems like the service is running fine. No error logs and no suspicious cpu usage etc.
If I restart the service it will run fine again until it hangs next time.
Would be most grateful if someone could help me with this.
public class WorkerRole : RoleEntryPoint
{
private LoggingService _loggingService;
public override void Run()
{
_loggingService = new LoggingService();
StartListeningForIncommingTCPConnections();
}
private void StartListeningForIncommingTCPConnections()
{
TcpListener listener = null;
try
{
listener = new TcpListener(RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["WatchMeEndpoint"].IPEndpoint);
listener.Start();
while (true)
{
_loggingService.Log(SeverityLevel.Info, "Waiting for connection...");
var client = listener.AcceptTcpClient();
var remoteEndPoint = client.Client != null ? client.Client.RemoteEndPoint.ToString() : "Unknown";
_loggingService.Log(SeverityLevel.Info, String.Format("Connected to {0}", remoteEndPoint));
var netStream = client.GetStream();
var data = String.Empty;
using (var reader = new StreamReader(netStream, Encoding.ASCII))
{
data = reader.ReadToEnd();
}
_loggingService.Log(SeverityLevel.Info, "Received data: " + data);
ProcessData(data); //data is processed and stored in database (all resources are released when done)
client.Close();
_loggingService.Log(SeverityLevel.Info, String.Format("Connection closed for {0}", remoteEndPoint));
}
}
catch (Exception exception)
{
_loggingService.Log(SeverityLevel.Error, exception.Message);
}
finally
{
if (listener != null)
listener.Stop();
}
}
private void ProcessData(String data)
{
try
{
var processor = new Processor();
var lines = data.Split('\n');
foreach (var line in lines)
processor.ProcessLine(line);
processor.ProcessMessage();
}
catch (Exception ex)
{
_loggingService.Log(SeverityLevel.Error, ex.Message);
throw new Exception(ex.InnerException.Message);
}
}
}
One strange observation i just did:
I checked the log recently and no instrument has connected for the last 30 minutes (which indicates that the service is down).
I connected to the service myself via a TCP client i've written myself and uploaded some test data.
This worked fine.
When I checked the log again my test data had been stored.
The strange thing is, that 4 other instruments had connected about the same time and send their data successfully.
Why couldn't they connect by themself before I connected with my test client?
Also, what does this setting in .csdef do for an InputEndpoint, idleTimeoutInMinutes?
===============================================
Edit:
Since a cuple of days back my cloud service has been running successfully.
Unfortunately this morning last log entry was from this line:
_loggingService.Log(SeverityLevel.Info, String.Format("Connected to {0}", remoteEndPoint));
No other connections could be made after this. Not even from my own test TCP client (didn't get any error though, but no data was stored and no new logs).
This makes me think that following code causes the service to hang:
var netStream = client.GetStream();
var data = String.Empty;
using (var reader = new StreamReader(netStream, Encoding.ASCII))
{
data = reader.ReadToEnd();
}
I've read somewhere that StremReader's ReadToEnd() could hang. Is this possible?
I have now changed this piece of code to this:
int i;
var bytes = new Byte[256];
var data = new StringBuilder();
const int dataLimit = 10;
var dataCount = 0;
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
{
data.Append(Encoding.ASCII.GetString(bytes, 0, i));
if (dataCount >= dataLimit)
{
_loggingService.Log(SeverityLevel.Error, "Reached data limit");
break;
}
dataCount++;
}
Another explanation could be something hanging in the database. I use the SqlConnection and SqlCommand classes to read and write to my database. I always close my connection afterwards (finally block).
SqlConnection and SqlCommand should have default timeouts, right?
===============================================
Edit:
After some more debugging I found out that when the service wasn't responding it "hanged" on this line of code:
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
After some digging I found out that the NetStream class and its read methods could actually hang. Even though MS declares otherwise.
NetworkStream read hangs
I've now changed my code into this:
Thread thread = null;
var task = Task.Factory.StartNew(() =>
{
thread = Thread.CurrentThread;
while ((i = netStream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data.Append(Encoding.ASCII.GetString(bytes, 0, i));
}
streamReadSucceeded = true;
});
task.Wait(5000);
if (streamReadSucceeded)
{
//Process data
}
else
{
thread.Abort();
}
Hopefully this will stop the hanging.
I'd say that part of your problem is you are processing your data on the thread that listens for connections from clients. This would prevent new clients from connecting if another client has started a long running operation of some type. I'd suggest you defer your processing to worker threads thus freeing the "listener" thread to accept new connections.
Another problem you could be experiencing, if your service throws an error, then the service will stop accepting connections as well.
private static void ListenForClients()
{
tcpListener.Start();
while (true)
{
TcpClient client = tcpListener.AcceptTcpClient();
Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);
}
}
private static void HandleClientComm(object obj)
{
try
{
using(TcpClient tcpClient = (TcpClient)obj)
{
Console.WriteLine("Got Client...");
using (NetworkStream clientStream = tcpClient.GetStream())
using (StreamWriter writer = new StreamWriter(clientStream))
using(StreamReader reader = new StreamReader(clientStream))
{
//do stuff
}
}
}
catch(Exception ex)
{
}
}