Read from Half Open Socket - sockets

I am trying to connect to Apple Push Notification Service which uses a simple binary protocol over TCP protected with TLS (or SSL). The protocol indicates that when an error is encountered (there are about 10 well defined error conditions) APNS will send back an error response and then close the connection. This results in a half closed socket because the remote peer closed the socket. I can see its a graceful shutdown because APNS sends a FIN and RST using tcpdump.
Out of all the error conditions, I can deal with most before sending with validation. The situation in which this fails is when a notification is sent to an invalid device token which cannot be dealt with that easily because the tokens could be malformed. Tokens are opaque 32 byte values that are provided by APNS to a device and then registered with me. I have no way of knowing if it is valid when submitted to my service. Presumably APNS checksums the tokens in some way that they can do quick validation on the token fast.
Anyway,
I did what I thought was the right thing:-
a. open socket
b. try writing
c. if write failed, read the error response
Unfortunately, this doesn't seem to work. I figure APNS is sending an error response and I am not reading it back right or I am not setting the socket up right. I have tried the following techniques:-
Use a separate thread per socket to try-read the response if any every 5ms or so.
Use a blocking read after write failure.
Use a final read after remote disconnect.
I have tried this with C# + .NET 4.5 on Windows and Java 1.7 on Linux. In either case, I never seem to get the error response and the socket indicates that no data is available to read.
Are half-closed sockets supported on these operating systems and/or frameworks? There isn't anything that seems to indicate either way.
I know that the way I am setting things up works correctly because if I use a valid token with a valid notification, those do get delivered.
In response to one of the comments, I am using the enhanced notification format so a response should arrive from APNS.
Here is the code I have for C#:-
X509Certificate certificate =
new X509Certificate(#"Foo.cer", "password");
X509CertificateCollection collection = new X509CertificateCollection();
collection.Add(certificate);
Socket socket =
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.Connect("gateway.sandbox.push.apple.com", 2195);
NetworkStream stream =
new NetworkStream(socket, System.IO.FileAccess.ReadWrite, false);
stream.ReadTimeout = 1000;
stream.WriteTimeout = 1000;
sslStream =
new SslStream(stream, true,
new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
sslStream.AuthenticateAsClient("gateway.sandbox.push.apple.com", collection,
SslProtocols.Default, false);
sslStream.ReadTimeout = 10000;
sslStream.WriteTimeout = 1000;
// Task rdr = Task.Factory.StartNew(this.reader);
// rdr is used for parallel read of socket sleeping 5ms between each read.
// Not used now but another alternative that was tried.
Random r = new Random(DateTime.Now.Second);
byte[] buffer = new byte[32];
r.NextBytes(buffer);
byte[] resp = new byte[6];
String erroneousToken = toHex(buffer);
TimeSpan t = (DateTime.UtcNow - new DateTime(1970, 1, 1));
int timestamp = (int) t.TotalSeconds;
try
{
for (int i = 0; i < 1000; ++i)
{
// build the notification; format is published in APNS docs.
var not = new ApplicationNotificationBuilder().withToken(buffer).withPayload(
#'{"aps": {"alert":"foo","sound":"default","badge":1}}').withExpiration(
timestamp).withIdentifier(i+1).build();
sslStream.Write(buffer);
sslStream.Flush();
Console.Out.WriteLine("Sent message # " + i);
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd);
break;
}
// doesn't really matter how fast or how slow we send
Thread.Sleep(500);
}
}
catch (Exception ex)
{
Console.Out.WriteLine("Failed to write ...");
int rd = sslStream.Read(resp, 0, 6);
if (rd > 0)
{
Console.Out.WriteLine("Found response: " + rd); ;
}
}
// rdr.Wait(); change to non-infinite timeout to allow error reader to terminate

I implemented server side for APNS in Java and have problems reading the error responses reliably (meaning - never miss any error response), but I do manage to get error responses.
You can see this related question, though it has no adequate answer.
If you never manage to read the error response, there must be something wrong with your code.
Using a separate thread for reading worked for me, though not 100% reliable.
Use a blocking read after write fail - that's what Apple suggest to do, but it doesn't always work. It's possible that you send 100 messages, and the first has an invalid token, and only after the 100th message you get a write failure. At this point it is sometimes too late to read the error response from the socket.
I'm not sure what you mean there.
If you want to guarantee that the reading of the error responses will work, you should try to read after each write, with a sufficient timeout. This, of course, is not practical for using in production (since it's incredibly slow), but you can use it to verify that your code of reading and parsing the error response is correct. You can also use it to iterate over all the device tokens you have, and find all the invalid ones, in order to clean your DB.
You didn't post any code, so I don't know what binary format you are using to send messages to APNS. If you are using the simple format (that starts with a 0 byte and has no message ID), you won't get any responses from Apple.

Related

In UWP StreamSocket, can I read data with timeout and leave the connection open if timeout elapses

As I couldn't find any way to peek for data (read data without consuming the buffer) as asked at How to peek StreamSocket for data in UWP apps I'm now trying to make my own "peek" but still no luck.
I don't see how I can read data from StreamSocket in the manner which will let me use timeouts and leave the connection usable in case if timeout elapses.
In the end, the problem is as follows. In my, let's say, IMAP client, I get response from a server and if this response is negative, I need to wait a bit to see if the server immediately sends yet another response (sometimes, the server can do it, with extra details on the error or even a zero packet to close the connection). if the server didn't send another response, I'm fine, just leaving the method and returning to the caller. The caller can then send more data to the stream, receive more responses, etc.
So, after sending a request and getting initial response I need in some cases to read socket once again with a very small timeout interval and if no data arrives, just do nothing.
You can use a CancelationTokenSource to generate a timeout and stop an async operation.
The DataReader consumes the data from the input stream of the StreamSocket. Its LoadAsync() method will return when there is at least one byte of data. Here, we are adding a cancellation source that will cancel the asynchronous task after 1 second to stop the DataReader.LoadAsync() if no data has been consumed.
var stream = new StreamSocket();
var inputStream = stream.InputStream;
var reader = new DataReader(inputStream);
reader.InputStreamOptions = InputStreamOptions.Partial;
while(true)
{
try
{
var timeoutSource = new CancellationTokenSource(TimeSpan.FromSeconds(1));
var data = await reader.LoadAsync(1).AsTask(timeoutSource.Token);
while(reader.UnconsumedBufferLength > 0)
{
var read = reader.ReadUInt32();
}
}
catch(TaskCanceledException)
{
// timeout
}
}
Do no forget that disposing the DataReader will close the stream and the connection.

how to tell server that client already finished the output without shutting down the outputstream

I having the following code to create a client socket to send/receive data:
val socket:Socket = new Socket(InetAddress.getByName("127.0.0.1"), 7777)
val inputStream = socket.getInputStream()
val bufferSource = new BufferedSource(inputStream)
val out = new PrintStream(socket.getOutputStream())
var data = "Hello Everyone"
out.println(data)
out.flush()
***socket.shutdownOutput()***
val in = bufferSource.getLines()
if (in.hasNext) {
println(in.next())
}
If I don't run socket.shutdownOutput(), I won't get the data from server,
because Server side is still waiting the input. Therefore I have to shutdown the outputStream.
But if shutdown the output, it can not be reopen. So I have to create a new socket for sending new data.
That caused sending one record needs to create a new socket. This is really awkward.
Is there any other way to tell the server that the output already finished without shutting down the output please.
Thanks in advance!
The problem is that the server doesn't know when to stop reading and process and reply.
What you need here is an application-level protocol that would dictate how server and clients are to communicate - what is a command, how a response to be formatted, etc.
This could be a line-oriented protocol - each new line represents a message (in general the message delimiter could be any other character sequence not appearing in the messages).
Or it could be fixed length messages; or messages pre-pended with message length (or type) to let the other side know how much data yo expect.

OpenSSL Nonblocking Socket Accept And Connect Failed

Here is my question:
Is it bad to set socket to nonblocking before I call accept or connect? or it should be using blocking accept and connect, then change the socket to nonblocking?
I'm new to OpenSSL and not very experienced with network programming. My problem here is I'm trying to use OpenSSL with a nonblocking socket network to add security. When I call SSL_accept on server side and SSL_connect on client side, and check return error code using
SSL_get_error(m_ssl, n);
char error[65535];
ERR_error_string_n(ERR_get_error(), error, 65535);
the return code from SSL_get_error indicates SSL_ERROR_WANT_READ, while ERR_error_string_n prints out "error:00000000:lib(0):func(0):reason(0)", which I think it means no error. SSL_ERROR_WANT_READ means I need to retry both SSL_accept and SSL_connect.
Then I use a loop to retry those function, but this just leads to a infinite loop :(
I believe I have initialized SSL properly, here is the code
//CRYPTO_malloc_init();
SSL_library_init();
const SSL_METHOD *method;
// load & register all cryptos, etc.
OpenSSL_add_all_algorithms();
// load all error messages
SSL_load_error_strings();
if (server) {
// create new server-method instance
method = SSLv23_server_method();
}
else {
// create new client-method instance
method = SSLv23_client_method();
}
// create new context from method
m_ctx = SSL_CTX_new(method);
if (m_ctx == NULL) {
throwError(-1);
}
If there is any part I haven't mentioned but you think it could be the problem, please let me know.
SSL_ERROR_WANT_READ means I need to retry both SSL_accept and SSL_connect.
Yes, but this is not the full story.
You should retry the call only after the socket gets readable, e.g. you need to use select or poll or similar functions to wait, until the socket gets readable. Same applies to SSL_ERROR_WANT_WRITE, but here you have to wait for the socket to get writable.
If you just retry without waiting it will probably eventually succeed, but only after having 100s of failed calls. While doing select does not guarantee that it will succeed immediately at the next call it will take only a few calls of SSL_connect/SSL_accept until it succeeds and it will not busy loop and eat CPU in the mean time.

How to read the whole message with Chilkat socket?

I need to get the whole message(response), but socket.ReceiveBytes(); returns just part of the message. I tried to loop it but it fails on timeout when no bytes to receive.
List<byte> lb = new List<byte>();
byte[] receivedMsg = socket.ReceiveBytes();
while (receivedMsg.Length > 0)
{
lb.AddRange(receivedMsg);
receivedMsg = socket.ReceiveBytes();
}
So, how I can check if there are byte to read? How I can read the whole message?
Since its a Chilkat implementation, you should probably contact the developer. But I found this that could help: http://www.cknotes.com/?p=302
Ultimately, you need to know how much to read from the socket to constitute a whole message. For example, if the overlying protocol is a portmapper, then you know that you are expecting messsages in the format that the RFC specifies (http://tools.ietf.org/html/rfc1833.)
If you are rolling your own protocol over a socket connection, then use the method in the Chilkat blog post about putting the size of the total message in the first 4 bytes.

Python APNs does not process request

I'm trying to implement a server side script for sending push notifications to apple push notification server. I create the ssl connection, I send the payload - but am unable to get a response from the APNs. Here is my code:
import socket, ssl, pprint, struct, time, binascii
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# require a certificate from the server
ssl_sock = ssl.wrap_socket( s,
keyfile="/Users/Jeff/Desktop/pickmeup-key2-noenc.pem",
certfile="/Users/Jeff/Desktop/pickmeup-cert2.pem",
server_side=False,
do_handshake_on_connect=True,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs="/Users/Jeff/Desktop/entrustrootcert.pem",)
#ciphers="ALL")
ssl_sock.connect(('gateway.sandbox.push.apple.com', 2195))
print repr(ssl_sock.getpeername())
print ssl_sock.cipher()
print pprint.pformat(ssl_sock.getpeercert())
command = '\x00'
identifier = 1987
expiry = time.time()
deviceToken = "9858d81caa236a86cc67d01e1a07ba1df0982178dd7c95aae115d033b93cb3f5"
alert = "This is a test message"
sound = "UILocalNotificationDefaultSoundName"
payload = "{\"aps\":{\"alert\":\"%s\",\"sound\":\"%s\"}}" %(alert, sound)
packetFormat = "!cIIH%dsH%ds" %(32, len(payload))
packet = struct.pack(packetFormat,
command,
identifier,
int(expiry),
32,
binascii.unhexlify(deviceToken),
len(payload),
payload)
nBytesWritten = ssl_sock.write(packet)
print "nBytesWritten = %d" %(nBytesWritten)
data = ssl_sock.read(1024)
print len(data)
ssl_sock.close()
Running this script, I generate the following output:
('17.149.34.132', 2195)
('AES256-SHA', 'TLSv1/SSLv3', 256)
{'notAfter': 'May 31 00:04:27 2012 GMT',
'subject': ((('countryName', u'US'),),
(('stateOrProvinceName', u'California'),),
(('localityName', u'Cupertino'),),
(('organizationName', u'Apple Inc'),),
(('organizationalUnitName', u'Internet Services'),),
(('commonName', u'gateway.sandbox.push.apple.com'),))}
nBytesWritten = 133
0
Any ideas on what might be going wrong? (I am sending enhanced push notifications so I am expecting a response from apple push notification server)
The key thing to note is that read() is returning no data. In Python, read() is supposed to block until data is available or the connection closes. Apple is closing your connection.
Why? Well, probably because you sent a malformed request. command=0 is a normal push notification; command=1 is enhanced. The big-endian 1987 will be interpreted as a 0-byte device token and a 1987-byte payload, neither of which are valid.
(And FWIW, I'd use B instead of c for the command ID; it seems to make more sense.)
you may consider https://github.com/djacobs/PyAPNs that wrapped lot of useful features, including:
error handling
support enhanced message format and auto resend messages which are sent before error response
non-blocking ssl socket connection with great performance
Apple Push notification server doesn't give a response, it's a one-way binary socket.
Rather than rolling your own solution you could try apns-python-wrapper or apns