I want to realize a simple client-server connection using Nodejs.
But I've encountered with the following problem.
Consider the code
server.js:
var net = require('net'),
sys = require('sys');
net.createServer(onConnection).listen(8124);
function onConnection(socket) {
socket.setNoDelay(true);
socket.addListener("connect", function () {
sys.puts('client connected: ' + this.remoteAddress);
});
socket.addListener("data", function (data) {
sys.puts("message: \n" + data + "\n - end of msg.");
});
socket.addListener("end", function () {
sys.puts('end of connection');
this.end();
});
}
sys.puts('Server running at 127.0.0.1:8124');
client.js:
var net = require('net'),
sys = require('sys');
var stream = net.createConnection(8124);
stream.addListener("connect", function(){
sys.puts('connected');
stream.write('a');
stream.flush();
stream.write('b');
stream.flush();
});
stream.addListener("data", function(data){
sys.puts("Message: \n" + data + "\n - end of msg.");
});
When I run client.js I sometimes get only one message 'ab' instead of two messages 'a' and 'b'.
Is there some 'right method' to deal with that?
TCP is a stream protocol. Single write on one end of the pipe can result in multiple "reads" on the other end, and the other way around. You have to either explicitly tell the other side how many bytes you are sending by including the length in the message; or provide easily recognizable message delimiters. In any case you need to read in a loop.
use socket.write return value and callback as documented here https://nodejs.org/api/net.html#net_socket_write_data_encoding_callback to know when the data is completly flushed to the kernel . Wait for that to happen and after that call the second write. that way you make sure of the ordering. Regarding the problem of logic separartion of "a" and "b" you might want to design/implement that "protocol yourself, it's not a responsiblity of the (low-level) socket API.
FYI, if you are able to structure to be a line-based text protocol, you can use readline. See:
https://nodejs.org/api/readline.html#example-tiny-cli
https://nodejs.org/api/readline.html#example-read-file-stream-line-by-line
Related
Can anyone please help me with this code , what is the use of "boost::asio::async_write" function here
Does it sends acknowledgment back to the client ?
void handle_read(const boost::system::error_code& error,
size_t bytes_transferred)
{
if (!error)
{
boost::asio::async_write(socket_,
boost::asio::buffer(data_, bytes_transferred),
boost::bind(&session::handle_write, this,
boost::asio::placeholders::error));
}
else
{
delete this;
}
}
It looks like this is from an "echo server" example. async_write writes the contents of boost::asio::buffer(data_, bytes_transferred) to the socket.
Since we're inside handle_read we can guess that this function itself is the completion handler for a likely async_read call that filled that data_ buffer. Since we use the exact number of bytes reported back by async_read (bytes_transferred) and there's no visible manipulation on data_, we can assume that this simply sends the exact message (or data in general) received to socket_. If socket_ was also the endpoint in the async_read this is the definition of an echo server.
I'm writing a server application in D, who should be able to manage n connections simultaneously.
To achieve this i am using std.socket.Socket.select. This works fine. But I can't bind session specific data to the socket and i don't see any way to do this, cause Socket does not allow to save a handle to user specific data. After
Socket.select(socketSet, null, null);
I'm able to get all affected sockets, but I can't assign this sockets to my user specific session data. What's my mistake? Is it possible to reach my goal in this way? Or should I choose another way for my requirements?
My relevant code:
ushort port = 5010;
stoprequest = false;
auto listener = new TcpSocket();
assert(listener.isAlive);
listener.blocking = false;
listener.bind(new InternetAddress(port));
listener.listen(10);
enum MAX_CONNECTIONS = 100;
auto socketSet = new SocketSet(MAX_CONNECTIONS + 1);
Socket[] reads;
Session[] sessions;
while (true)
{
socketSet.add(listener);
foreach (session; sessions)
socketSet.add(session.socket);
Socket.select(socketSet, null, null);
for (size_t i = 0; i < reads.length; i++)
{
if (socketSet.isSet(reads[i]))
{
// Now i should acces to session related data, but how?
char[1024] buf;
auto datLength = reads[i].receive(buf[]);
if (datLength == Socket.ERROR)
writeln("Connection error.");
else if (datLength != 0)
{
writefln("Received %d bytes from %s: \"%s\"", datLength, reads[i].remoteAddress().toString(), buf[0..datLength]);
continue;
}
else { // Error Handling. Shortened, since unimportant for the example}
reads[i].close();
reads = reads.remove(i);
i--;
}
}
if (socketSet.isSet(listener))
{
Socket sn = null;
sn = listener.accept();
if (reads.length < MAX_CONNECTIONS)
{
Session session = new Session();
session.socket = sn;
sessions ~= session;
}
else { // Error Handling for too many connection. Shortened, since unimportant for the example}}
}
socketSet.reset();
}
The hint to use poll() was helpful. After reading https://daniel.haxx.se/docs/poll-vs-select.html I think that both variants work and neither of them are the real thing. For an efficient way, I should better deal with libev. Fortunately, efficiency is not my problem in this particular project. For this reason I will use select(), because i found out, that accessing handle gives me a unique number which can be passed to a own lookup table. This allows me to assign session data to a socket. So I prefer to stick with the encapsulated functionality of std.socket.Socket and don't work around it.
My concrete question can therefore be answered with :
Use Socket.handle to identify the socket and manage session related
data
A few other alternatives you can consider:
1) use a subclass of Socket. You can make your own class that inherits from it and adds more stuff.
2) The poll function is found in import core.sys.posix.poll;, and you can pass socket.handle to that as well. But note it will not work on Windows without modification.
or indeed 3) do your own lookup table, that works too.
Note that the std.socket.Socket is a very thin wrapper around the bsd socket api, just internally it does conveniently handle the slight differences between Windows and posix. Still it is pretty easy to adapt code to use the other apis with it (or tutorials on C language stuff to D) since it is all basically the same thing - and literally the same functions if you import core.sys stuff.
At a Vert.x verticle I'm implementing I have a Buffer that was previously loaded into memory and now I want to dump it into disk.
As far as I understood we should use a Pump to make sure not to overload the WriteStream.
But I'm not finding a way to get a ReadStream child instance from a Buffer. Shouldn't there be an easy / standard way to do this?
Regards
Generally, vert.x does not warn on any issues writing directly into AsyncFiles. Furthermore, they provide the corresponding example of using AsyncFile.write directly here and state that you can use those to write directly: http://vertx.io/docs/vertx-core/java/#_asynchronous_files
However, if you want the pump with Buffer you need an instance of ReadStream<Buffer> along with an AsyncFile to pump into. You can make use of the implementation by PitchPoint Solutions (Copyright 2016 The Simple File Server Authors):
https://github.com/pitchpoint-solutions/sfs/blob/master/sfs-server/src/main/java/org/sfs/io/BufferReadStream.java
Putting it all together:
CompletableFuture<Void> done = new CompletableFuture<>();
Buffer buffer = Buffer.buffer(new byte[100]);
Vertx.vertx().fileSystem.open("myfile.txt", new OpenOptions(), res -> {
if (res.succeeded()) {
AsyncFile outputFile = res.result();
BufferReadStream reader = new BufferReadStream(buffer)
Pump pump = Pump.pump(reader, outputFile);
pump.start();
reader.endHandler((r) -> {
pump.stop(); // not sure this is required
done.complete(null);
});
} else {
// Something went wrong!
}
});
// wait elsewhere
done.get();
In our project we are using ReactiveMongo with Play 2.2.1.
The problem is, that to stream of data in a form of Enumerator[A], returned by ReactiveMongo is actually a stream of value objects, which are not separated by commas, and do not have stream beginning and end annotations, which can be treated as array opening and close statements.
This creates a problem for JSON consumer JS client, since the expected format is
[A1,A2, ...]
So we jumped in hoops, and transformed our Enumeratee[A] to Enumerator[String], with checking, if it's the first element, or not:
var first:Boolean = true
val aToStrs = (as.map(a => {
if(first) {
first = false;
Json.stringify(Json.toJson(a))
} else {
"," + Json.stringify(Json.toJson(a))
}
}))
Ok.chunked(
Enumerator.enumInput(Input.El("[")) andThen
aToStrs andThen
Enumerator.enumInput(Input.El("]")) andThen
Enumerator.enumInput(Input.EOF)
)
This works, but feels like inventing the wheel.
Is there a better solution, for this common problem?
If you use comet or EventSource you wont have to hand craft a way to generate output and you will also actually be able to parse the response item for item in the client. Responding with an array would force you to either write your own parsing code or wait until everything has arrived on the client side before you can use the build int JSON parser in your JavaScript.
Streaming with the EventSource protocol is pretty easy with play, you should be able to do something like:
implicit val cometEncoder = new Comet.CometMessage[JsValue](_.toString)
Ok.chunked(yourEnumerator &> EventSource()).as(EVENT_STREAM)
And then in the client html:
<script type="text/javascript">
var es = new EventSource(jsRouter.controllers.Application.streamIt().url)
es.addEventListener("message", function (event) {
var item = JSON.parse(event.data)
// ... do something with the json value ...
})
</script>
There is an example of this in the play sample projects that you might want to look at as well $YOUR_PLAY_DIR/samples/scala/eventsource-clock/
I need to send several commands over telnet to a server. If I try to send them without a time delay between every command, the server freaks out:
var net = require('net');
var conn = net.createConnection(8888, 'localhost');
conn.on('connect', function() {
conn.write(command_1);
conn.write(command_2);
conn.write(command_3);
//...
conn.write(command_n);
})
I guess the server needs some time to respond to command n before I send it command n+1. One way is to write something to the log and fake a "wait":
var net = require('net');
var conn = net.createConnection(8888, 'localhost');
conn.on('connect', function() {
console.log('connected to server');
console.log('I'm about to send command #1');
conn.write(command_1);
console.log('I'm about to send command #2');
conn.write(command_2);
console.log('I'm about to send command #3');
conn.write(command_3);
//...
console.log('I'm about to send command #n');
conn.write(command_n);
})
It might also be the fact that conn.write() is asynchronous, and putting one command after another doesn't guranty the correct order??
Anyway, what is the correct pattern to assure correct order and enough time between two consecutive commands, for the server to respond?
First things first: if this is truly a telnet server, then you should do something with the telnet handshaking (where terminal options are negotiated between the peers, this is the binary data you can see when opening the socket).
If you don't want to get into that (it will depend on your needs), you can ignore the negotiation and go straight to business, but you will have to read this data and ignore it yourself.
Now, in your code, you're sending the data as soon as the server accepts the connection. This may be the cause of your troubles. You're not supposed to "wait" for the response, the response will get to you asynchronously thanks to nodejs :) So you just need to send the commands as soon as you get the "right" response from the server (this is actually useful, because you can see if there were any errors, etc).
I've tried this code (based on yours) against a device I've got at hand that has a telnet server. It will do a login and then a logout. See how the events are dispatched according to the sever's response:
var net = require('net');
var conn = net.createConnection(23, '1.1.1.1');
var commands = [ "logout\n" ];
var i = 0;
conn.setEncoding('ascii');
conn.on('connect', function() {
conn.on('login', function () {
conn.write('myUsername\n');
});
conn.on('password', function () {
conn.write('myPassword\n');
});
conn.on('prompt', function () {
conn.write(commands[i]);
i++;
});
conn.on('data', function(data) {
console.log("got: " + data + "\n");
if (data.indexOf("login") != -1) {
conn.emit('login');
}
if (data.indexOf("password") != -1) {
conn.emit('password');
}
if (data.indexOf(">#") != -1) {
conn.emit('prompt');
}
});
});
See how the commands are in an array, where you can iteratively send them (the prompt event will trigger the next command). So the right response from the server is the next prompt. When the server sends (in this case) the string ># another command is sent.
Hope it helps :)
The order of writes is guaranteed. However:
You must subscribe to data event. conn.on('data', function(data)
{}) will do.
You must check return values of writes - if a write
fails, you must wait for 'drain' event. So you should check if any
write really fails and if it does then fix the problem. If it
doesn't - then you can leave current dirty solution as is.
You
must check if your server supports request piplining (sending
multiple requests without waiting for responses). If it doesn't -
you must not send next request before receiving a data event after
the previous one.
You must ensure that the commands you send are real telnet commands - telnet expects a \0 byte after \r\n (see the RFC), so certain servers may freak if \0 is not present.
So:
var net = require('net');
var conn = net.createConnection(8888, 'localhost');
conn.on('connect', function() {
console.log(conn.write(command_1) &&
conn.write(command_2) &&
conn.write(command_3) &&
//...
conn.write(command_n))
})
conn.on('data', function () {})
If it writes false - then you must wait for 'drain'. If it writes true - you must implement waiting. I discourage the event-based solution and suggest to look at using async or Step NPM modules instead.