TCP/.NET: Reading length prefixed buffer or fixed buffer size - sockets

I'm currently in the progress of writing my own TCP network library in C# and I want to make sure I'm doing things right.
Currently, I have every packet that gets sent through a socket prefixed with the length. The receiving end will read 4 bytes, convert that to an int, and read that many bytes afterward.
I'm not sure if that's the best approach. Would it be better to just read a fixed amount of bytes and process from there?
In the end, I would just extract 4 bytes from the fixed length buffer and read ahead however much I need.
Here is the code that I currently have to demonstrate my thought process.
internal void BeginReceive()
{
ReceiveImpl();
}
protected virtual void ReceiveImpl()
{
// Should we rather receive a bigger buffer (e.g. 8KB) immediately and then grab the length from that?
// It would require an internal buffer to store whatever we've already read ahead.
var recvState = new ReceiveState(new byte[4]);
Socket.BeginReceive(recvState.Buffer, 0, 4, SocketFlags.None, OnReceiveLength, recvState);
}
private void OnReceiveLength(IAsyncResult ar)
{
var recvState = (ar.AsyncState as ReceiveState)!;
var bytesRead = Socket.EndReceive(ar, out var errorCode);
if (errorCode != SocketError.Success)
{
// we ain't good fam!
return;
}
if (bytesRead > 0) // should we rather check if we read the 4 bytes we wanted?
{
var length = LengthFromBuffer(recvState.Buffer);
recvState.Buffer = new byte[length];
#if DEBUG
Console.WriteLine($"Receiving a packet with length of {length}.");
#endif
// what if the packet is absolutely massive? should we limit the buffer size?
Socket.BeginReceive(recvState.Buffer, 0, length, SocketFlags.None, OnReceive, recvState);
}
}
private void OnReceive(IAsyncResult ar)
{
var recvState = (ar.AsyncState as ReceiveState)!;
var bytesRead = Socket.EndReceive(ar, out var errorCode);
if (errorCode != SocketError.Success)
{
// we ain't good fam!
return;
}
if (bytesRead > 0)
{
recvState.BytesReceived += bytesRead;
if (recvState.BytesReceived < recvState.Buffer.Length)
{
Socket.BeginReceive(recvState.Buffer, recvState.BytesReceived,
recvState.Buffer.Length - recvState.BytesReceived, SocketFlags.None, OnReceive, recvState);
return;
}
OnDataReceived(recvState.Buffer); // this will call BeginReceive again.
}
}

Related

Problem receiving data from bluetooth at very high speeds with Bluetooth Serial

I'm using the flutter_bluetooth_serial 0.4.0 package, it has a listen function that receives a function that returns the reading of the sending of a string (List Uint8), but for my case I need to carry out the communication at a very high speed, and when this happens it does not understand where the end of a string is and joins overflow with 230 bytes and shows as if it was a single string, I tried to solve this in several ways, but I only managed to receive a complete string (18 bytes) when I slow down of transmission. I would need to read 40 bytes at a time regardless of '\n' or anything, I would have for this case maybe read byte by byte to validate the message and not a Uint8. In this case the input is a Stream and I would have to transform it into a Stream or something that makes me receive 40 bytes at a time regardless of the speed. Most of the time, a message comes together with another, and this could not happen in this transmission.
Here is the code snippet I get the data:
_dataSubscription = connection.input!.listen(_onDataReceived);
void _onDataReceived(Uint8List data) {
print('Data incoming: ${ascii.decode(data)}');
// Allocate buffer for parsed data
var backspacesCounter = 0;
for (var byte in data) {
if (byte == 8 || byte == 127) backspacesCounter++;
}
var buffer = Uint8List(data.length - backspacesCounter);
var bufferIndex = buffer.length;
// Apply backspace control character
backspacesCounter = 0;
for (int i = data.length - 1; i >= 0; i--) {
if (data[i] == 8 || data[i] == 127) {
backspacesCounter++;
} else {
if (backspacesCounter > 0) {
backspacesCounter--;
} else {
buffer[--bufferIndex] = data[i];
}
}
// print(backspacesCounter);
// print(buffer);
// print(bufferIndex);
}

Problem receiving data from bluetooth at very high speeds

I'm using the flutter_bluetooth_serial 0.4.0 package, it has a listen function that receives a function that returns the reading of the sending of a string (List Uint8), but for my case I need to carry out the communication at a very high speed, and when that it happens it does not understand where the end of a string is and it joins it until it overflows with 230 bytes and shows it as if it were a single string, I tried to solve this in several ways, but I can only receive the complete string (18 bytes) when I reduce the transmission speed, I've tried using some characters like '\n', to see if it understands where a string ends and another begins, but I wasn't successful either. If I could read character by character for me it would also solve it, because the messages have a sending pattern.
Do you have an idea how I could be solving this? Some package that works better than this one for this purpose or some way to determine where the end of the string is. I thank!
Here is the code snippet I get the data:
_dataSubscription = connection.input!.listen(_onDataReceived);
void _onDataReceived(Uint8List data) {
print('Data incoming: ${ascii.decode(data)}');
// Allocate buffer for parsed data
var backspacesCounter = 0;
for (var byte in data) {
if (byte == 8 || byte == 127) backspacesCounter++;
}
var buffer = Uint8List(data.length - backspacesCounter);
var bufferIndex = buffer.length;
// Apply backspace control character
backspacesCounter = 0;
for (int i = data.length - 1; i >= 0; i--) {
if (data[i] == 8 || data[i] == 127) {
backspacesCounter++;
} else {
if (backspacesCounter > 0) {
backspacesCounter--;
} else {
buffer[--bufferIndex] = data[i];
}
}
// print(backspacesCounter);
// print(buffer);
// print(bufferIndex);
}
I've tried using some characters like '\n', to see if it understands where a string ends and another begins, read character per character, but doesn't have function to do this.

Dispatch source is only called when I do a NSLog() first

I am trying to use grand central dispatch in conjunction with bsd sockets to send an icmp ping. I add DISPATCH_SOURCE_TYPE_WRITE and DISPATCH_SOURCE_TYPE_READ as dispatch sources to read and write async.
So this is the method were I create the bsd socket and install the dispatch sources:
- (void)start
{
int err;
const struct sockaddr * addrPtr;
assert(self.hostAddress != nil);
// Open the socket.
addrPtr = (const struct sockaddr *) [self.hostAddress bytes];
fd = -1;
err = 0;
switch (addrPtr->sa_family) {
case AF_INET: {
fd = socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP);
if (fd < 0) {
err = errno;
}
} break;
case AF_INET6:
assert(NO);
// fall through
default: {
err = EPROTONOSUPPORT;
} break;
}
if (err != 0) {
[self didFailWithError:[NSError errorWithDomain:NSPOSIXErrorDomain code:err userInfo:nil]];
} else {
dispatch_source_t writeSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_WRITE, fd, 0, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0));
dispatch_source_set_event_handler(writeSource, ^{
abort(); // testing
// call call method here to send a ping
});
dispatch_resume(writeSource);
//NSLog(#"testout");
dispatch_source_t readSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, fd, 0, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0));
dispatch_source_set_event_handler(readSource, ^{
unsigned long bytesAvail = dispatch_source_get_data(readSource);
NSLog(#"bytes available: %lu", bytesAvail);
});
dispatch_resume(readSource);
}
}
You see the //NSLog(#"testout");? The funny thing is that the write block is only called when the //NSLog(#"testout"); is NOT commented out. This is very odd. I didn't test the read callback. The sending needs to be working first.
So what is going on here?
There are kind of a bunch of things missing here. I'm not sure exactly which one is causing the weird behavior, but when I do all of the missing things, it seems to work "as expected" and my write event handler is called reliably and repeatedly. In general, there are a bunch of things you need to do when setting up a socket like this before passing it off to GCD. They are:
Create the socket
Bind it to a local address (missing in your code)
Set it to non-blocking (missing in your code)
Here is a little example I was able to put together in which the write handler gets called repeatedly, as expected:
int DoStuff()
{
int fd = -1;
// Create
if ((fd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) {
perror("cannot create socket");
return 0;
}
// Bind
struct sockaddr_in *localAddressPtr = (struct sockaddr_in *)malloc(sizeof(struct sockaddr_in));
memset((char *)localAddressPtr, 0, sizeof(*localAddressPtr));
localAddressPtr->sin_family = AF_INET;
localAddressPtr->sin_addr.s_addr = htonl(INADDR_ANY);
localAddressPtr->sin_port = htons(0);
if (bind(fd, (struct sockaddr *)localAddressPtr, sizeof(*localAddressPtr)) < 0) {
perror("bind failed");
return 0;
}
// Set non-blocking
int flags;
if (-1 == (flags = fcntl(fd, F_GETFL, 0)))
flags = 0;
if (-1 == fcntl(fd, F_SETFL, flags | O_NONBLOCK))
{
perror("Couldnt set non-blocking");
return 0;
}
// Do a DNS lookup...
struct hostent *hp;
struct sockaddr_in *remoteAddressPtr = malloc(sizeof(struct sockaddr_in));
// Fill in the server's address and data
memset((char*)remoteAddressPtr, 0, sizeof(*remoteAddressPtr));
remoteAddressPtr->sin_family = AF_INET;
remoteAddressPtr->sin_port = htons(12345);
// Look up the address of the server by name
const char* host = "www.google.com";
hp = gethostbyname(host);
if (!hp) {
fprintf(stderr, "could not obtain address of %s\n", host);
return 0;
}
// Copy the host's address into the remote address structure
memcpy((void *)&remoteAddressPtr->sin_addr, hp->h_addr_list[0], hp->h_length);
dispatch_source_t writeSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_WRITE, fd, 0, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0));
dispatch_source_set_event_handler(writeSource, ^{
// Send message
const char* my_message = "the only thing we have to fear is fear itself.";
unsigned long len = strlen(my_message);
if (sendto(fd, my_message, len, 0, (struct sockaddr *)remoteAddressPtr, sizeof(*remoteAddressPtr)) != len) {
perror("sendto failed");
dispatch_source_cancel(writeSource);
}
});
dispatch_source_set_cancel_handler(writeSource, ^{
close(fd);
free(localAddressPtr);
free(remoteAddressPtr);
});
dispatch_resume(writeSource);
return 1;
}
NB: There's no way to dispose of the writeSource in my example without there being an error in a send operation. It's a trivial example...
My general theory on why NSLog triggers the handler to fire in your case, is that it keeps execution at or below that stack frame long enough for the background thread to come around and call the handler, but without that NSLog, your function returns, and something has a chance to die before the handler can get called. In fact, if you're using ARC it's probably the writeSource itself that is getting deallocated, since I don't see you making a strong reference to it anywhere outside the scope of this function. (My example captures a strong reference to it in the block, thus keeping it alive.) You could test this in your code by stashing a strong reference to writeSource.
I found the error:
In newer SDKs dispatch sources are subject to automatic reference counting despite the fact that they are no Objective-C objects.
So when the start method is over ARC disposes the dispatch source and they never get called.
NSLog delays the end of the start method in a way that the dispatch source triggers before the source gets disposed.

Struct and Thread DWORD WINAPI

What's up guys, hope you are ok !
well, the problem is that I'm doing a chat client/server aplication but doing some tests with the server, I found out that I have a problem sending messages. I'm using a struct, sockets and DWORD WINAPI threads...
So the code in the struct is:
DWORD WINAPI threadSendMessages(LPVOID vpParam); //THREAD
typedef struct messagesServerChat{ //STRUCT
const char *messageServEnv;
}MESSAGE, *SMESSAGES;
then in the main method I call the struct to use the const char messageServEnv, a HeapAlloc to give some memory to the thread that is going to send the message and a char variable that I use to store the message
char mServer[1024] = ""; //variable to pre-store the message
SMESSAGES messages; //call the struct
messages = (SMESSAGES) HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, sizeof(MESSAGE));
in the main method, I ask the user to insert the message that he wants to send and I use the struct to store the message and send it to the thread as a parameter:
cout<<"Dear user, please insert your message: ";
setbuf(stdin, NULL);
fgets(mServer, 1024, stdin);
messages->messageServEnv = mServer;
DWORD hSend; //send the parameters to the thread function
HANDLE sendThread = CreateThread(0, 0, threadSendMessages, mServer, 0, &hSend);
and finally the thread code function
DWORD WINAPI threadSendMessages(LPVOID lpParam){
SMESSAGES messages;
messages = (SMESSAGES)lpParam;
int mesa;
mesa = send(sConnect, (char *)messages->messageServEnv, sizeof messages->messageServEnv, 0);
//sConnect is the socket
//messages = to use the struct, and messageServEnv is the struct data that should contain the message
return 0;
}
--Edit-- I fix a lot of problems using Remy's solution but maybe I'm missing something... in the Thread threadSendMessages(SMESSAGES lpMessage)
char *ptr = messages->messageServEnv;
int len = strlen(messages->messageServEnv);
I get and error that says messages is undifined, then, I changed to:
SMESSAGES messages;
char *ptr = messages->messageServEnv;
int len = strlen(messages->messageServEnv);
now I can use messages and struct value messageServEnv but if I start debugging visual studio and I try to send a message, I get an error that says messages is used without being initialized, then I change that part to
SMESSAGES messages = new MESSAGE;
and now I can send messages to client but only characters and garbage code
You need to dynamically allocate the memory for each message's string data and then have the thread free the memory when finished sending it.
You are also passing the wrong pointer to the lpParameter parameter of CreateThread(), you are passing your char[] buffer instead of your allocated MESSAGE struct.
You are also using sizeof() when calling send(). Since your messageServEnv is a char* pointer, sizeof() will return 4 (32-bit) or 8 (64-bit) instead of the actual size of the string that is being pointed at.
I would suggest moving the char[] buffer directly into the struct instead of using a pointer to an external buffer, eg:
typedef struct messagesServerChat
{
char messageServEnv[1024];
}
MESSAGE, *SMESSAGES;
DWORD WINAPI threadSendMessages(SMESSAGES lpMessage);
.
cout << "Dear user, please insert your message: ";
setbuf(stdin, NULL);
SMESSAGES message = new MESSAGE;
fgets(message->messageServEnv, sizeof(message->messageServEnv), stdin);
DWORD hSend;
HANDLE sendThread = CreateThread(0, 0, (LPTHREAD_START_ROUTINE)&threadSendMessages, message, 0, &hSend);
if (!sendThread)
delete message;
.
DWORD WINAPI threadSendMessages(SMESSAGES lpMessage)
{
// send() is not guaranteed to send the entire message
// in one go, so call it in a loop...
char *ptr = lpMessage->messageServEnv;
int len = strlen(lpMessage->messageServEnv); // or sizeof() if you really want to send all 1024 bytes instead
while (len > 0)
{
int mesa = send(sConnect, ptr, len, 0);
if (mesa > 0)
{
ptr += mesa;
len -= mesa;
continue;
}
// this is only needed if you are using a non-blocking socket...
if ((mesa == SOCKET_ERROR) && (WSAGetLastError() == WSAEWOULDBLOCK))
{
fd_set fd;
FD_ZERO(&fd);
FD_SET(sConnect, &fd);
timeval tv;
tv.tv_sec = 5;
tv.tv_usec = 0;
if (select(0, NULL, &fd, NULL, &tv) > 0)
continue;
}
... error handling ...
break;
}
delete message;
return 0;
}
If you want to pass a dynamically-lengthed string instead, you are better off using a std::string instead of a char[]:
typedef struct messagesServerChat
{
std::string messageServEnv;
}
MESSAGE, *SMESSAGES;
DWORD WINAPI threadSendMessages(SMESSAGES lpMessage);
.
cout << "Dear user, please insert your message: ";
setbuf(stdin, NULL);
SMESSAGES message = new MESSAGE;
getline(stdin, message->messageServEnv);
DWORD hSend;
HANDLE sendThread = CreateThread(0, 0, (LPTHREAD_START_ROUTINE)&threadSendMessages, message, 0, &hSend);
if (!sendThread)
delete message;
.
DWORD WINAPI threadSendMessages(SMESSAGES lpMessage)
{
// send() is not guaranteed to send the entire message
// in one go, so call it in a loop...
char *ptr = lpMessage->messageServEnv.c_str();
int len = lpMessage->messageServEnv.length(); // or sizeof() if you really want to send all 1024 bytes instead
while (len > 0)
{
int mesa = send(sConnect, ptr, len, 0);
if (mesa > 0)
{
ptr += mesa;
len -= mesa;
continue;
}
// this is only needed if you are using a non-blocking socket...
if ((mesa == SOCKET_ERROR) && (WSAGetLastError() == WSAEWOULDBLOCK))
{
fd_set fd;
FD_ZERO(&fd);
FD_SET(sConnect, &fd);
timeval tv;
tv.tv_sec = 5;
tv.tv_usec = 0;
if (select(0, NULL, &fd, NULL, &tv) > 0)
continue;
}
... error handling ...
break;
}
delete message;
return 0;
}

Unable to Write On CFWriteStreamWrite

i am having trouble in writing data to CFStream.
// i am getting the CFSocketRef and then from it getting native Handle.
CFSocketNativeHandle sock = CFSocketGetNative( [appDelegate getSocketRef]);
Does above Code return me the same handler of the created socket?what ever i write onto stream will be written on the created socket?
// and then wrote
CFStreamCreatePairWithSocket(kCFAllocatorDefault, sock,
&readStream, &writeStream);
if (!readStream || !writeStream) {
// close([appDelegate TCPClient]);
// close(sock);
fprintf(stderr, "CFStreamCreatePairWithSocket() failed\n");
return;
}
above works fine,it does not give me failed message
// does not give error ,else portion is executed
if (!CFWriteStreamOpen(writeStream)) {
CFStreamError myErr = CFWriteStreamGetError(writeStream);
// An error has occurred.
if (myErr.domain == kCFStreamErrorDomainPOSIX) {
// Interpret myErr.error as a UNIX errno.
NSLog(#"kCFStreamErrorDomainPOSIX");
} else if (myErr.domain == kCFStreamErrorDomainMacOSStatus) {
// Interpret myErr.error as a MacOS error code.
OSStatus macError = (OSStatus)myErr.error;
// Check other error domains.
NSLog(#"kCFStreamErrorDomainMacOSStatus");
}
}else
/* Send the connect call to stream */
// while (send_len < (originalLength + 1))
{
// if (CFWriteStreamCanAcceptBytes(writeStream))
{
//UInt8 buf[] = "Hello, world";//(unsigned char *) "connectStream"
//CFIndex bufLen = (CFIndex)strlen(buf);
bytes = CFWriteStreamWrite(writeStream,
(unsigned char *) connectStream,
originalLength );
NSLog(#"%#",[[NSString alloc] initWithData:connectStream encoding:NSASCIIStringEncoding] );
if (bytes < 0) {
fprintf(stderr, "CFWriteStreamWrite() failed\n");
// close(sock);
return;
}
send_len += bytes;
}
// close(sock);
CFReadStreamClose(readStream);
CFWriteStreamClose(writeStream);
return;
}
CFWriteStreamCanAcceptBytes always return false so i have commented it and directly wrote bytes,and it blocks the call and does not return any thing neither any byte is written on to the stream,
Can any one please guide me in this rergard?
is there any other way of doing this?
Regards,
Aamir