I having the following code to create a client socket to send/receive data:
val socket:Socket = new Socket(InetAddress.getByName("127.0.0.1"), 7777)
val inputStream = socket.getInputStream()
val bufferSource = new BufferedSource(inputStream)
val out = new PrintStream(socket.getOutputStream())
var data = "Hello Everyone"
out.println(data)
out.flush()
***socket.shutdownOutput()***
val in = bufferSource.getLines()
if (in.hasNext) {
println(in.next())
}
If I don't run socket.shutdownOutput(), I won't get the data from server,
because Server side is still waiting the input. Therefore I have to shutdown the outputStream.
But if shutdown the output, it can not be reopen. So I have to create a new socket for sending new data.
That caused sending one record needs to create a new socket. This is really awkward.
Is there any other way to tell the server that the output already finished without shutting down the output please.
Thanks in advance!
The problem is that the server doesn't know when to stop reading and process and reply.
What you need here is an application-level protocol that would dictate how server and clients are to communicate - what is a command, how a response to be formatted, etc.
This could be a line-oriented protocol - each new line represents a message (in general the message delimiter could be any other character sequence not appearing in the messages).
Or it could be fixed length messages; or messages pre-pended with message length (or type) to let the other side know how much data yo expect.
Related
I'm working with a Camel flow that uses a Netty TCP socket consumer to receive messages from a client program (which is outside of my control). The client should be opening a socket, sending us one message, then closing the socket, but we've been seeing cases where instead of one message Camel is "splitting" the text stream into two parts and trying to process them separately.
So I'm trying to figure out, since you can re-use the same socket for multiple Camel messages, but TCP sockets don't have a built-in concept of "frames" or a standard for message delimiters, how does Camel decide that a complete message has been received and is ready to process? I haven't been able to find a documented answer to this in the Netty component docs (https://camel.apache.org/components/3.15.x/netty-component.html), although maybe I'm missing something.
From playing around with a test script, it seems like one answer is "Camel assumes a message is complete and should be processed if it goes more than 1ms without receiving any input on the socket". Is this a correct statement, and if so is this behavior documented anywhere? Is there any way to change or configure this behavior? Really what I would prefer is for Camel to wait for an ETX character (or a much longer timeout) before processing a message, is that possible to set up?
Here's my test setup:
Camel flow:
from("netty:tcp://localhost:3003")
.log("Received: ${body}");
Python snippet:
DELAY_MS = 3
def send_msg(sock, msg):
print("Sending message: <{}>".format(msg))
if not sock.sendall(msg.encode()) is None:
print("Message failed to send")
time.sleep(DELAY_MS / 1000.0)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
print("Using DELAY_MS: {}".format(str(DELAY_MS)))
s.connect((args.hostname, args.port))
cutoff = int(math.floor(len(args.msg) / 2))
msg1 = args.msg[:cutoff]
send_msg(s, msg1)
msg2 = args.msg[cutoff:]
send_msg(s, msg2)
response = s.recv(1024)
except Exception as e:
print(e)
finally:
s.close()
I can see that with DELAY_MS=1 Camel logs one single message:
2022-02-21 16:54:40.689 INFO 19429 --- [erExecutorGroup] route1 : Received: a long string sent over the socket
But with DELAY_MS=2 it logs two separate messages:
2022-02-21 16:56:12.899 INFO 19429 --- [erExecutorGroup] route1 : Received: a long string sen
2022-02-21 16:56:12.899 INFO 19429 --- [erExecutorGroup] route1 : Received: t over the socket
After doing some more research, it seems like what I need to do is add a delimiter-based FrameDecoder to the decoders list.
Setting it up like this:
from("netty:tcp://localhost:3003?sync=true"
+ "&decoders=#frameDecoder,#stringDecoder"
+ "&encoders=#stringEncoder")
where frameDecoder is provided by
#Bean
ChannelHandlerFactory frameDecoder() {
ByteBuf[] ETX_DELIM = new ByteBuf[] { Unpooled.wrappedBuffer(new byte[] { (char)3 }) };
return ChannelHandlerFactories.newDelimiterBasedFrameDecoder(1024, ETX_DELIM,
false, "tcp");
}
seems to do the trick.
On the flip side though, it seems like this will hang indefinitely (or until lower-level TCP timeouts kick in?) if an ETX frame is not received, and I can't figure out any way to set a timeout on the decoder, so would still be eager for input if anyone knows how to do that.
I think the default "timeout" behavior I was seeing might've just been an artifact of Netty's read loop speed -- How does netty determine when a read is complete?
Trying to figure out how to send/read data through socket. On remote server I create new netcat -l 4444 and from local send text data echo "test" | netcat remote.host 4444. This is always works fine.
Trying to reproduce:
require "socket"
HOST = "remote.host"
PORT = 4444
ch_request = Channel(String).new
ch_response = Channel(String).new
spawn do
socket = TCPSocket.new(HOST, PORT)
loop do
select
when request = ch_request.receive
socket << request
socket.flush
end
if response = socket.gets
ch_response.send response
end
end
end
sleep 0.1
ch_request.send "hello"
loop do
select
when response = ch_response.receive
pp response
end
end
In my dream I send data to channel, read it from first loop then send to socket. The same way but reverse order need for read it from second loop.
On practice this is not happens. On local after connect I got "test" and can't send back anything. On remote I can send to local but on local got only empty string once and nothing more after.
What mean this behavior and how to achieve planned?
You didn't show this, but I suppose you have a second implementation using TCPServer for the netcat -l equivalent.
You need to use separate fibers for reading/writing to the socket and the channel. It's a bit hard to judge what exactly happens without seeing the server, but I suppose you end up in a deadlock where both sides are waiting on input of the other side or user, but cannot proceed to actually send or read anything. In other words you interlocked the sending/receiving part, requiring the other side to carefully react and interplay so to not lock up the client. This is obviously a brittle approach.
Instead you should make sure any fiber does not do more than one operation in a loop. One receives from the socket and forwards that to a channel, the second one receives from a channel and forwards that to the socket, the third one receives from the reader side channel and prints or does whatever you want to do the data and the last one fills the sender channel. This way no operation can block one of the others. Of course one of those fibers should simply be the main program one.
In the server you additionally need one fiber that accepts the client connections and spawns the sender and receiver loops for each.
Finally note that a select statement with a single when branch has no effect, you can make the call directly. select is useful if you need to read or write to multiple channels concurrently in the same fiber, so for example if you would have multiple channels providing data to be send out to a socket, you would use select to not have the messages be corrupted by two fibers writing to the same socket at the same time. An additional usecase for select is to send or receive from a channel with a timeout.
Who is looking for an answer to similar questions. The final result what I wanted looks like this:
# Open socket for simulate remote server: `netcat -v -4 -l 4444`
require "socket"
HOST = "remote.host"
PORT = 4444
# JFYI: In real life this packed into class and I use class variable instead consts.
TUBE_REQUEST = Channel(String).new
TUBE_RESPONSE = Channel(String).new
SOCKET = TCPSocket.new(HOST, PORT)
spawn do
until SOCKET.closed?
if request = TUBE_REQUEST.receive
SOCKET << request
SOCKET.flush
end
end
end
spawn do
until SOCKET.closed?
if response = SOCKET.gets
TUBE_RESPONSE.send response
end
end
end
sleep 0.1
def receive_response
TUBE_RESPONSE.receive
end
def send(message, wait_for_response = true)
TUBE_REQUEST.send message
receive_response if wait_for_response
end
send("command with response")
send("command with new line and response\n")
send("command without new line and response", false)
It will send each command and wait for answer (except the last) from remote and then call the next command.
As I couldn't find any way to peek for data (read data without consuming the buffer) as asked at How to peek StreamSocket for data in UWP apps I'm now trying to make my own "peek" but still no luck.
I don't see how I can read data from StreamSocket in the manner which will let me use timeouts and leave the connection usable in case if timeout elapses.
In the end, the problem is as follows. In my, let's say, IMAP client, I get response from a server and if this response is negative, I need to wait a bit to see if the server immediately sends yet another response (sometimes, the server can do it, with extra details on the error or even a zero packet to close the connection). if the server didn't send another response, I'm fine, just leaving the method and returning to the caller. The caller can then send more data to the stream, receive more responses, etc.
So, after sending a request and getting initial response I need in some cases to read socket once again with a very small timeout interval and if no data arrives, just do nothing.
You can use a CancelationTokenSource to generate a timeout and stop an async operation.
The DataReader consumes the data from the input stream of the StreamSocket. Its LoadAsync() method will return when there is at least one byte of data. Here, we are adding a cancellation source that will cancel the asynchronous task after 1 second to stop the DataReader.LoadAsync() if no data has been consumed.
var stream = new StreamSocket();
var inputStream = stream.InputStream;
var reader = new DataReader(inputStream);
reader.InputStreamOptions = InputStreamOptions.Partial;
while(true)
{
try
{
var timeoutSource = new CancellationTokenSource(TimeSpan.FromSeconds(1));
var data = await reader.LoadAsync(1).AsTask(timeoutSource.Token);
while(reader.UnconsumedBufferLength > 0)
{
var read = reader.ReadUInt32();
}
}
catch(TaskCanceledException)
{
// timeout
}
}
Do no forget that disposing the DataReader will close the stream and the connection.
I am trying to connect to Apple Push Notification Service which uses a simple binary protocol over TCP protected with TLS (or SSL). The protocol indicates that when an error is encountered (there are about 10 well defined error conditions) APNS will send back an error response and then close the connection. This results in a half closed socket because the remote peer closed the socket. I can see its a graceful shutdown because APNS sends a FIN and RST using tcpdump.
Out of all the error conditions, I can deal with most before sending with validation. The situation in which this fails is when a notification is sent to an invalid device token which cannot be dealt with that easily because the tokens could be malformed. Tokens are opaque 32 byte values that are provided by APNS to a device and then registered with me. I have no way of knowing if it is valid when submitted to my service. Presumably APNS checksums the tokens in some way that they can do quick validation on the token fast.
Anyway,
I did what I thought was the right thing:-
a. open socket
b. try writing
c. if write failed, read the error response
Unfortunately, this doesn't seem to work. I figure APNS is sending an error response and I am not reading it back right or I am not setting the socket up right. I have tried the following techniques:-
Use a separate thread per socket to try-read the response if any every 5ms or so.
Use a blocking read after write failure.
Use a final read after remote disconnect.
I have tried this with C# + .NET 4.5 on Windows and Java 1.7 on Linux. In either case, I never seem to get the error response and the socket indicates that no data is available to read.
Are half-closed sockets supported on these operating systems and/or frameworks? There isn't anything that seems to indicate either way.
I know that the way I am setting things up works correctly because if I use a valid token with a valid notification, those do get delivered.
In response to one of the comments, I am using the enhanced notification format so a response should arrive from APNS.
Here is the code I have for C#:-
X509Certificate certificate =
new X509Certificate(#"Foo.cer", "password");
X509CertificateCollection collection = new X509CertificateCollection();
collection.Add(certificate);
Socket socket =
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.Connect("gateway.sandbox.push.apple.com", 2195);
NetworkStream stream =
new NetworkStream(socket, System.IO.FileAccess.ReadWrite, false);
stream.ReadTimeout = 1000;
stream.WriteTimeout = 1000;
sslStream =
new SslStream(stream, true,
new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
sslStream.AuthenticateAsClient("gateway.sandbox.push.apple.com", collection,
SslProtocols.Default, false);
sslStream.ReadTimeout = 10000;
sslStream.WriteTimeout = 1000;
// Task rdr = Task.Factory.StartNew(this.reader);
// rdr is used for parallel read of socket sleeping 5ms between each read.
// Not used now but another alternative that was tried.
Random r = new Random(DateTime.Now.Second);
byte[] buffer = new byte[32];
r.NextBytes(buffer);
byte[] resp = new byte[6];
String erroneousToken = toHex(buffer);
TimeSpan t = (DateTime.UtcNow - new DateTime(1970, 1, 1));
int timestamp = (int) t.TotalSeconds;
try
{
for (int i = 0; i < 1000; ++i)
{
// build the notification; format is published in APNS docs.
var not = new ApplicationNotificationBuilder().withToken(buffer).withPayload(
#'{"aps": {"alert":"foo","sound":"default","badge":1}}').withExpiration(
timestamp).withIdentifier(i+1).build();
sslStream.Write(buffer);
sslStream.Flush();
Console.Out.WriteLine("Sent message # " + i);
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd);
break;
}
// doesn't really matter how fast or how slow we send
Thread.Sleep(500);
}
}
catch (Exception ex)
{
Console.Out.WriteLine("Failed to write ...");
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd); ;
}
}
// rdr.Wait(); change to non-infinite timeout to allow error reader to terminate
I implemented server side for APNS in Java and have problems reading the error responses reliably (meaning - never miss any error response), but I do manage to get error responses.
You can see this related question, though it has no adequate answer.
If you never manage to read the error response, there must be something wrong with your code.
Using a separate thread for reading worked for me, though not 100% reliable.
Use a blocking read after write fail - that's what Apple suggest to do, but it doesn't always work. It's possible that you send 100 messages, and the first has an invalid token, and only after the 100th message you get a write failure. At this point it is sometimes too late to read the error response from the socket.
I'm not sure what you mean there.
If you want to guarantee that the reading of the error responses will work, you should try to read after each write, with a sufficient timeout. This, of course, is not practical for using in production (since it's incredibly slow), but you can use it to verify that your code of reading and parsing the error response is correct. You can also use it to iterate over all the device tokens you have, and find all the invalid ones, in order to clean your DB.
You didn't post any code, so I don't know what binary format you are using to send messages to APNS. If you are using the simple format (that starts with a 0 byte and has no message ID), you won't get any responses from Apple.
I am attempting to write a client/server program where the client can:
*Write a string to server
*Receive an Object from server
and the server can:
*Receive a string from a client
*Send an object to the client.
However, my attempts so far have proved fruitless and I am quite frustrated. I am sure I am making a fundamental flaw, but being so new to Socket programming I just cannot see where I am going wrong.
Is there anything wrong with my choice of streams, or is there a better way of using streams which I have not picked up on?
The object I am sending is serialised, although the object is never written/read as the server hangs on the readUTF()
Thank you in advance for any help.
//Used to receive a string input from client side
DataInputStream input = new DataInputStream(clientSocket.getInputStream());
// Used to output an object to client side
ObjectOutputStream oo = new ObjectOutputStream(clientSocket.getOutputStream());
// This is never executed..
clientInput = input.readUTF();
oo.writeObject(book);
Client Side code:
// Used to read object from server side
input = new ObjectInputStream(clientSocket.getInputStream());
// Used to read input from command line
keyboard = new BufferedReader(new InputStreamReader(System.in));
// Used to send string to server
output = new DataOutputStream(clientSocket.getOutputStream());
cmdInput = keyboard.readLine();
// Write to server, this executes at runtime.
output.writeUTF(cmdInput);
// Object is never sent, this line is not executed.
book = (Book)input.readObject();
In general, you would be better off using a web service kit (e.g. Apache CXF) than doing this yourself.
In specific, try calling flush() on your output stream in the client.