I have implemented a Perl script that sends push notifications through Apples apns services. I am having some issues with the error handling. As per APNS documentation:
If the stream isn't ready for writing, see if the stream is available for reading. If it is, read everything available from the stream. If you get zero bytes back, the connection was closed because of an error such as an invalid command byte or other parsing error. If you get six bytes back, that's an error response that you can check for the response code and the ID of the notification that caused the error. You'll need to send every notification following that one again.
I am doing the same. Whenever I get a write error due to connection drop; I read the socket. Everytime I get 6 bytes return from the socket. Meaning APNS is sending me back an error_response. The format of error-response packet as per APNS documentation is like below
The packet has a command value of 8 followed by a one-byte status code and the notification identifier of the malformed notification.
I am using the below code to unpack the data I read from the socket:
my $hex = unpack( 'H*', $data );
print $hex;
Everytime, I get the same value 080800000000. As per APNS documentation the first byte will always be 8, the next byte will represent the error status code. 8 means "Invalid Token". Up to this part it is OK. However, the remaining 4 bytes which is the identifier, always gives me 00000000. What does it mean ?
APNS has two different push notification format, Simple Notification Format and Enhanced Notification Format. The Simple Notification Format does not have a field for specifying the message ID (notification identifier). I was using a Perl module (Net::APNS::Persistent) for communicating with APNS; that only supports Simple Notification Format. Thats why I was always getting 00000000 for the notification identifier part. I updated the code of the module to use the enhanced format which is :
pack(
'cNNnH*na*', # format
1, # command
$id, # Notification identifier
0, # expiry timestamp
32, # token length
$devicetoken, # token
length $json, # payload length
$json # payload
);
Then for reading the response I used the below message format:
my($c,$status,$identifier) = unpack('ccN',$error);
Where $error is the response from APNS. Now the whole thing is working fine.
Related
I've just started working with sockets.
I've connected to a socket (a bitcoin node), and when I send data with socket_send(), the socket will reply with data, but I don't know what the length of that data will be.
I would like to receive the full data response with socket_recv() and move on to issuing the next socket_send(), but I don't know what to put for the len or flags in this function to get the full message (and not wait for anything more).
How do you get a full message of unknown length with socket_recv()?
ASIDE: This combination seems to work, but it was just trial and error and I
don't know why:
socket_recv($socket, $buf, 10000000, MSG_WAITALL&MSG_DONTWAIT)
I want some level of real-time speech to text conversion. I am using the web-sockets interface with interim_results=true. However, I am receiving results for the first audio chunk only. The second,third... audio chunks that I am sending are not getting transcribed. I do know that my receiver is not blocked since I do receive the inactivity message.
json {"error": "Session timed out due to inactivity after 30 seconds."}
Please let me know if I am missing something if I need to provide more contextual information.
Just for reference this is my init json.
{
"action": "start",
"content-type":"audio/wav",
"interim_results": true,
"continuous": true,
"inactivity_timeout": 10
}
In the result that I get for the first audio chunk, the final json field is always received as false.
Also, I am using golang but that should not really matter.
EDIT:
Consider the following pseudo log
localhost-server receives first 4 seconds of binary data #lets say Binary 1
Binary 1 is sent to Watson
{interim_result_1 for first chunk}
{interim_result_2 for first chunk}
localhost-server receives last 4 seconds of binary data #lets say Binary 2
Binary 2 is sent to Watson
Send {"action": "stop"} to Watson
{interim_result_3 for first chunk}
final result for the first chunk
I am not receiving any transcription for the second chunk
Link to code
You are getting the time-out message because the service waits for you to either send more audio or send a message signalling the end of an audio submission. Are you sending that message? It's very easy:
By sending a JSON text message with the action key set to the value stop: {"action": "stop"}
By sending an empty binary message
https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/doc/speech-to-text/websockets.shtml
Please let me know if this does not resolve your problem
This is a bit late, but I've open-sourced a Go SDK for Watson services here:
https://github.com/liviosoares/go-watson-sdk
There is some documentation about speech-to-text binding here:
https://godoc.org/github.com/liviosoares/go-watson-sdk/watson/speech_to_text
There is also an example of streaming data to the API in the _test.go file:
https://github.com/liviosoares/go-watson-sdk/blob/master/watson/speech_to_text/speech_to_text_test.go
Perhaps this can help you.
The solution to this question was to set the size header of the wav file to 0.
I am using a perl module called Net::APNS::Persistent. It helps me to open up a persistent connection with apple's apns server and send push notifications through APNS. This module uses Net::SSLeay for ssl communication with APNS server.
Now, I want to read from my socket periodically to check if APNS sends back any response. Net::APNS::Persistent already has a function called _read() which looks like below:
sub _read {
my $self = shift;
my ($socket, $ctx, $ssl) = #{$self->_connection};
my $data = Net::SSLeay::ssl_read_all( $ssl );
die_if_ssl_error("error reading from ssl connection: $!");
return $data;
}
However, this function works only after APNS drops the connection and I get error while trying to write. On other times my script gets stuck at,
my $data = Net::SSLeay::ssl_read_all( $ssl );
I checked Net::SSLeay doc and found it has a method called peek
Copies $max bytes from the specified $ssl into the returned value. In contrast to the Net::SSLeay::read() function, the data in the SSL buffer is unmodified after the SSL_peek() operation.
I though it might be useful, so I added another function within the Net::APNS::Persistent module:
sub ssl_peek {
my $self = shift;
my ($socket, $ctx, $ssl) = #{$self->_connection};
print "Peeking \n";
my $data = Net::SSLeay::peek( $ssl, $pending );
print "Done peeking \n";
return $data;
}
Unfortunately this also gave me the same problem. It only prints Peeking and never reaches the line where it would print Done peeking. Had same problem using Net::SSLeay::read. Is there a way to check if the socket can be read or maybe set a read timeout so that my script doesnt get stuck while trying to read from socket?
The APNS documentation says the following:
If you send a notification that is accepted by APNs, nothing is returned.
If you send a notification that is malformed or otherwise unintelligible, APNs returns an error-response packet and closes the connection. Any notifications that you sent after the malformed notification using the same connection are discarded, and must be resent
As long as your notifications as accepted, there won't be any data to read and thus a read operation on the socket will block. The only time there's data available is when there's an error, and then the connection is immediately closed. That should explain the behaviour you're observing.
To check if the underlying socket can be read use select, i.e.
IO::Select->new(fileno($socket))->can_read(timeout);
timeout can be 0 to just check and not wait, can be a number of seconds or can be undef to wait forever. But before you do the select check if data are still available in the SSL buffer:
if (Net::SSLeay::pending($ssl)) { ... use SSL_peek or SSL_read ... }
Apart from that it does look like that the module you use does not even attempt to validate the servers certificate :(
In our Windows C++ application I am using InitializeSecurityContext() client side to open an schannel connection to a server which is running stunnel SSL proxy. My code now works, but only with a hack I would like to eliminate.
I started with this sample code:http://msdn.microsoft.com/en-us/library/aa380536%28v=VS.85%29.aspx
In the sample code, look at SendMsg and ReceiveMsg. The first 4 bytes of any message sent or received indicates the message length. This is fine for the sample, where the server portion of the sample conforms to the same convention.
stunnel does not seem to use this convention. When the client is receiving data during the handshake, how does it know when to stop receiving and make another call to InitializeSecurityContext()?
This is how I structured my code, based on what I could glean from the documentation:
1. call InitializeSecurityContext which returns an output buffer
2. Send output buffer to server
3. Receive response from server
4. call InitializeSecurityContext(server_response) which returns an output buffer
5. if SEC_E_INCOMPLETE_MESSAGE, go back to step 3,
if SEC_I_CONTINUE_NEEDED go back to step 2
I expected InitializeSecurityContext in step 4 to return SEC_E_INCOMPLETE_MESSAGE if not enough data was read from the server in step 3. Instead, I get SEC_I_CONTINUE_NEEDED but an empty output buffer. I have experimented with a few ways to handle this case (e.g. go back to step 3), but none seemed to work and more importantly, I do not see this behavior documented.
In step 3 if I add a loop that receives data until a timeout expires, everything works fine in my test environment. But there must be a more reliable way.
What is the right way to know how much data to receive in step 3?
SChannel is different than the Negotiate security package. You need to receive at least 5 bytes, which is the SSL/TLS record header size:
struct {
ContentType type;
ProtocolVersion version;
uint16 length;
opaque fragment[TLSPlaintext.length];
} TLSPlaintext;
ContentType is 1 byte, ProtocolVersion is 2 bytes, and you have 2 byte record length. Once you read those 5 bytes, SChannel will return SEC_E_INCOMPLETE_MESSAGE and will tell you exactly how many more bytes to expect:
SEC_E_INCOMPLETE_MESSAGE
Data for the whole message was not read from the wire.
When this value is returned, the pInput buffer contains a SecBuffer structure with a BufferType member of SECBUFFER_MISSING. The cbBuffer member of SecBuffer contains a value that indicates the number of additional bytes that the function must read from the client before this function succeeds.
Once you get this output, you know exactly how much to read from the network.
I found the problem.
I found this sample:
http://www.codeproject.com/KB/IP/sslsocket.aspx
I was missing the handling of SECBUFFER_EXTRA (line 987 SslSocket.cpp)
The SChannel SSP returns SEC_E_INCOMPLETE_MESSAGE from both InitializeSecurityContext and DecryptMessage when not enough data is read.
A SECBUFFER_MISSING message type is returned from DecryptMessage with a cbBuffer value of the amount of desired bytes.
But in practice, I did not use the "missing data" value. The documentation indicates the value is not guaranteed to be correct, and is only a hint for developers can use to reduce calls.
InitalizeSecurityContext MSDN doc:
While this number is not always accurate, using it can help improve performance by avoiding multiple calls to this function.
So I unconditionally read more data into the same buffer whenever SEC_E_INCOMPLETE_MESSAGE was returned. Reading multiple bytes at a time from a socket.
Some extra input buffer management was required to append more read data and keep the lengths right. DecryptMessage will modify the input buffers' cbBuffer properties when it fails, which surprised me.
Printing out the buffers and return result after calling InitializeSecurityContext shows the following:
read socket:bytes(5).
InitializeSecurityContext:result(80090318). // SEC_E_INCOMPLETE_MESSAGE
inBuffers[0]:type(2),bytes(5).
inBuffers[1]:type(0),bytes(0). // no indication of missing data
outBuffer[0]:type(2),bytes(0).
read socket:bytes(74).
InitializeSecurityContext:result(00090312). // SEC_I_CONTINUE_NEEDED
inBuffers[0]:type(2),bytes(79). // notice 74 + 5 from before
inBuffers[1]:type(0),bytes(0).
outBuffer[0]:type(2),bytes(0).
And for the DecryptMessage Function, input is always in dataBuf[0], with the rest zeroed.
read socket:bytes(5).
DecryptMessage:len 5, bytes(17030201). // SEC_E_INCOMPLETE_MESSAGE
DecryptMessage:dataBuf[0].BufferType 4, 8 // notice input buffer modified
DecryptMessage:dataBuf[1].BufferType 4, 8
DecryptMessage:dataBuf[2].BufferType 0, 0
DecryptMessage:dataBuf[3].BufferType 0, 0
read socket:bytes(8).
DecryptMessage:len 13, bytes(17030201). // SEC_E_INCOMPLETE_MESSAGE
DecryptMessage:dataBuf[0].BufferType 4, 256
DecryptMessage:dataBuf[1].BufferType 4, 256
DecryptMessage:dataBuf[2].BufferType 0, 0
DecryptMessage:dataBuf[3].BufferType 0, 0
read socket:bytes(256).
DecryptMessage:len 269, bytes(17030201). // SEC_E_OK
We can see my TLS Server peer is sending TLS headers (5 bytes) in one packet, and then the TLS message (8 for Application Data), then the Application Data payload in a third.
You must read some arbitrary amount the first time, and when you receive SEC_E_INCOMPLETE_MESSAGE, you must look in the pInput SecBufferDesc for a SECBUFFER_MISSING and read its cbBuffer to find out how many bytes you are missing.
This problem was doing my head in today, as I was attempting to modify my handshake myself, and having the same problem the other commenters were having, i.e. not finding a SECBUFFER_MISSING. I do not want to interpret the tls packet myself, and I do not want to unconditionally read some unspecified number of bytes. I found the solution to that, so I'm going to address their comments, too.
The confusion here is because the API is confusing. Ordinarily, to read the output of InitializeSecurityContext, you look at the content of the pOutput parameter (as defined in the signature). It's that SecBufferDesc that contains the SECBUFFER_TOKEN etc to pass to AcceptSecurityContext.
However, in the case where InitializeSecurityContext returns SEC_E_INCOMPLETE_MESSAGE, the SECBUFFER_MISSING is returned in the pInput SecBufferDesc, in place of the SECBUFFER_ALERT SecBuffer that was passed in.
The documentation does say this, but not in a way that clearly contrasts this case against the SEC_I_CONTINUE_NEEDED and SEC_E_OK cases.
This answer also applies to AcceptSecurityContext.
From MSDN, I'd presume SEC_E_INCOMPLETE_MESSAGE is returned when not enough data is received from server at the moment. Instead, SEC_I_CONTINUE_NEEDED returned with InBuffers[1] indicating amount of unread data (note that some data is processed and must be skipped) and OutBuffers containing nothing.
So the algorithm is:
If SEC_I_CONTINUE_NEEDED returned, check type of InBuffers[1]
If it is SECBUFFER_EXTRA, handle it (move InBuffers[1].cbBuffer bytes to the beginning of input buffer) and jump to next recv & InitializeSecurityContext iteration
If OutBuffers is not empty, send its contents to server
I'm trying to implement a server side script for sending push notifications to apple push notification server. I create the ssl connection, I send the payload - but am unable to get a response from the APNs. Here is my code:
import socket, ssl, pprint, struct, time, binascii
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# require a certificate from the server
ssl_sock = ssl.wrap_socket( s,
keyfile="/Users/Jeff/Desktop/pickmeup-key2-noenc.pem",
certfile="/Users/Jeff/Desktop/pickmeup-cert2.pem",
server_side=False,
do_handshake_on_connect=True,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs="/Users/Jeff/Desktop/entrustrootcert.pem",)
#ciphers="ALL")
ssl_sock.connect(('gateway.sandbox.push.apple.com', 2195))
print repr(ssl_sock.getpeername())
print ssl_sock.cipher()
print pprint.pformat(ssl_sock.getpeercert())
command = '\x00'
identifier = 1987
expiry = time.time()
deviceToken = "9858d81caa236a86cc67d01e1a07ba1df0982178dd7c95aae115d033b93cb3f5"
alert = "This is a test message"
sound = "UILocalNotificationDefaultSoundName"
payload = "{\"aps\":{\"alert\":\"%s\",\"sound\":\"%s\"}}" %(alert, sound)
packetFormat = "!cIIH%dsH%ds" %(32, len(payload))
packet = struct.pack(packetFormat,
command,
identifier,
int(expiry),
32,
binascii.unhexlify(deviceToken),
len(payload),
payload)
nBytesWritten = ssl_sock.write(packet)
print "nBytesWritten = %d" %(nBytesWritten)
data = ssl_sock.read(1024)
print len(data)
ssl_sock.close()
Running this script, I generate the following output:
('17.149.34.132', 2195)
('AES256-SHA', 'TLSv1/SSLv3', 256)
{'notAfter': 'May 31 00:04:27 2012 GMT',
'subject': ((('countryName', u'US'),),
(('stateOrProvinceName', u'California'),),
(('localityName', u'Cupertino'),),
(('organizationName', u'Apple Inc'),),
(('organizationalUnitName', u'Internet Services'),),
(('commonName', u'gateway.sandbox.push.apple.com'),))}
nBytesWritten = 133
0
Any ideas on what might be going wrong? (I am sending enhanced push notifications so I am expecting a response from apple push notification server)
The key thing to note is that read() is returning no data. In Python, read() is supposed to block until data is available or the connection closes. Apple is closing your connection.
Why? Well, probably because you sent a malformed request. command=0 is a normal push notification; command=1 is enhanced. The big-endian 1987 will be interpreted as a 0-byte device token and a 1987-byte payload, neither of which are valid.
(And FWIW, I'd use B instead of c for the command ID; it seems to make more sense.)
you may consider https://github.com/djacobs/PyAPNs that wrapped lot of useful features, including:
error handling
support enhanced message format and auto resend messages which are sent before error response
non-blocking ssl socket connection with great performance
Apple Push notification server doesn't give a response, it's a one-way binary socket.
Rather than rolling your own solution you could try apns-python-wrapper or apns