using curllib to send data to server - sockets

Question

I am very new to all this http thing and I am trying to make communication with a server.
I need to write the client that post to server - I know what server returns but I am not sure how to send it the data:
I (the client) Post XML (binary format) to server at url: 192.168.2.111/senddata (i.e post /senddata) using HTTP.
Server response is:
TCP: [ACK] Seq=1274 Ack=504 Win=525088 Len=0
I.E here is the problem - the server does not reply with HTTP reply.
The protocol server uses after I send it the XML is TCP and this makes curl_easy_perform
to not return.
After server TCP reply in step 2, I-the client should send binary data in a loop to server using HTTP Continuation or non-HTTP traffic.
Server asynchronicity answers with
TCP: [ACK] Seq=1274 Ack=3424 Win=522176 Len=0
for each binary data sending .
So to send the XML in step 1, I use
curl_easy_setopt(mCurlHndl, CURLOPT_URL, "192.168.2.111");
curl_easy_setopt(mCurlHndl, CURLOPT_CUSTOMREQUEST, "POST /senddata");
curl_easy_setopt(mCurlHndl, CURLOPT_POST, 1L);
curl_easy_setopt(mCurlHndl, CURLOPT_POSTFIELDS, sXML.c_str());
curl_easy_setopt(mCurlHndl, CURLOPT_POSTFIELDSIZE, sXML.length());
headers = curl_slist_append(headers, "Content-Type: application/octet-stream");
headers = curl_slist_append(headers, "Accept:");
headers = curl_slist_append(headers, (string("Content-Length: ") + string(to_string(sXML.length()))).c_str());
curl_easy_setopt(mCurlHndl, CURLOPT_HTTPHEADER, headers);
res = curl_easy_perform(mCurlHndl);
Server does send in reply the:
TCP: [ACK] Seq=1274 Ack=504 Win=525088 Len=0
(just with other seq/ack/win numbers of course).
OK, here starts the problems.
a. curl_easy_perform does not return since (I think) server reply was not HTTP, but TCP.
b. OK, lets say that i can bypass it giving Timeout
curl_easy_setopt(mCurlHndl, CURLOPT_TIMEOUT_MS, 1000);
But i just know that i am doing something wrong.
c. Lets say I gave the timeout, now in step 3 I need to send in a loop stream of data using HTTP Continuation or non-HTTP traffic.
I do not know how to do that since it should not be regular post again like in step 1.
It should be HTTP Continuation or non-HTTP traffic.
Sorry for my bad explanation but again it is from my very little experience in HTTP.
Seems that after step 1, server breaks that normal http connection rules and what I need to send it is the binary data in a loop but again, i do not know how to do step 2 and 3. I.E how to get the TCP ACK from server using curllib that I will know to move to step 3 and how to (step 3) send to server over the same connection (I think) the binary data in loop.
I want to add that servers returns what is supposed to return. in step 2 server should return the TCP: [ACK] Seq=1274 Ack=504 Win=525088 Len=0 since this is what server should do - this is that server protocol. It is not that i do not send it right (in step 1) and that is why it send TCP reply and not http.
Here is Wireshark log:
192.168.2.7 is the client (that works).
192.168.2.111 is the server.
in the first line (seq 9) the clinet post the binary xml (POST /senddata) and then the loop of binary data stream sending starts as you can see:
9 0.173574000 192.168.2.7 192.168.2.111 HTTP 380 POST /senddata HTTP/1.1 (application/octet-stream)
10 0.176611000 192.168.2.111 192.168.2.7 TCP 60 4089 > 53419 [ACK] Seq=1274 Ack=504 Win=525088 Len=0
11 0.182581000 192.168.2.111 192.168.2.7 UDP 90 Source port: 4020 Destination port: 4005
12 0.553367000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
13 0.553403000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
14 0.555539000 192.168.2.111 192.168.2.7 TCP 60 4089 > 53419 [ACK] Seq=1274 Ack=3424 Win=522176 Len=0
15 0.555698000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
16 0.555724000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
17 0.555724000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
18 0.555724000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
19 0.558870000 192.168.2.111 192.168.2.7 TCP 60 4089 > 53419 [ACK] Seq=1274 Ack=6344 Win=519248 Len=0
20 0.559033000 192.168.2.7 192.168.2.111 HTTP 1354 Continuation or non-HTTP traffic
..... and many other sends like seq 12 - seq 20 ........
How do i do something like this with curllib - this is my question.
Thanks again

Answer 1

You are not using curl correctly for sending an HTTP POST request. Try this instead:
curl_easy_setopt(mCurlHndl, CURLOPT_URL, "http://192.168.2.111/senddata");
curl_easy_setopt(mCurlHndl, CURLOPT_POST, 1L);
curl_easy_setopt(mCurlHndl, CURLOPT_POSTFIELDS, sXML.c_str());
curl_easy_setopt(mCurlHndl, CURLOPT_POSTFIELDSIZE, sXML.length());
struct curl_slist *headers = curl_slist_append(NULL, "Content-Type: application/octet-stream");
headers = curl_slist_append(headers, "Accept:");
curl_easy_setopt(mCurlHndl, CURLOPT_HTTPHEADER, headers);
res = curl_easy_perform(mCurlHndl);
curl_slist_free_all(headers);
// process reply using curl_easy_getinfo() and curl_easy_recv() as needed...
With that said, are you sure that you should be sending the Content-Type as application/octet-stream? Typically XML is posted using text/xml, application/xml, or other XML-related Content-Type so the server knows it is dealing with XML data. application/octet-stream refers to arbitrary binary data when its actual type is not known.
Also, setting the Accept header to a blank value basically means you do not accept any data in reply. Unless you want to accept only specific Content-Type reply types, or want to accept any data the server wants to send, then you should omit the Accept header.
Read RFC 2616 for the formal definition of the HTTTP protocol and how it works.
Update: since you are obviously dealing with a custom protocol and not standard HTTP, you cannot use curl_easy_perform() to send the HTTP request. You will have to enable the CURLOPT_CONNECT_ONLY option so curl_easy_perform() will only open the socket connection, and then use curl_easy_send() to send the actual request and subsequent binary data, eg:
CURLcode curl_send(CURL *curl, const void * buffer, size_t buflen)
{
size_t sent;
uint8_t *pbuf = (uint8_t*) buffer;
while (buflen > 0)
{
do
{
res = curl_easy_send(curl, pbuf, buflen, &sent);
}
while (res == CURLE_AGAIN);
if (res != CURLE_OK)
return res;
pbuf += sent;
buflen -= sent;
}
while (buflen > 0);
return CURLE_OK;
}
mCurlHndl = curl_easy_init();
...
curl_easy_setopt(mCurlHndl, CURLOPT_URL, "http://192.168.2.111");
curl_easy_setopt(mCurlHndl, CURLOPT_CONNECT_ONLY, 1);
res = curl_easy_perform(mCurlHndl);
if (res == CURLE_OK)
{
string req =
"POST /senddata HTTP/1.1\r\n"
"Content-Type: application/octet-stream\r\n"
"Content-Length: " + to_string(sXML.length()) + "\r\n"
"\r\n"
+ sXML;
res = curl_send(mCurlHndl, req.c_str(), req.length());
if (res !- CURLE_OK) ...
res = curl_send(mCurlHndl, ...binary data...);
if (res !- CURLE_OK) ...
res = curl_send(mCurlHndl, ...binary data...);
if (res !- CURLE_OK) ...
res = curl_send(mCurlHndl, ...binary data...);
if (res !- CURLE_OK) ...
...
}
...
curl_easy_cleanup(mCurlHndl);