I am using the following code to perform a tcp socket connection and send a string to an IP. But sometimes in the response, I not receiving the entire file
Socket m_socClient;
IPSelected ="1.1.2.3"
Port = "80"
string query ="My Query"
m_socClient = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
System.Net.IPAddress remoteIPAddress = System.Net.IPAddress.Parse(IPSelected);
System.Net.IPEndPoint remoteEndPoint = new System.Net.IPEndPoint(remoteIPAddress, Port);
m_socClient.Connect(remoteEndPoint);
try
{
if (m_socClient.Connected)
{
var reQuestToSend = "";
reQuestToSend = string.Format("POST /TMHP/Request HTTP/1.1\r\nHost:{0}\r\nContent-Length:{1}\r\n\r\n{2}", "edi-webtest.tmhp.com", Query270.Length, Query270);
byte[] bytesToSend = Encoding.ASCII.GetBytes(reQuestToSend);
byteCount = m_socClient.Send(bytesToSend, SocketFlags.None);
byte[] bytesReceived = new byte[3000];
byteCount = m_socClient.Receive(bytesReceived, SocketFlags.None);
Response271 = Encoding.ASCII.GetString(bytesReceived);
}
}
catch (Exception ex)
{
EVCommon.Log(ex.Message);
}
finally
{
m_socClient.Disconnect(false);
m_socClient.Close(5000);
}
I think the problem is with byte[] bytesReceived = new byte[3000];
Is there a way to not hardcode this number 3000. It works most of the time but for longer strings it gets only half of it.
I want it be handling variable sized messages instead of setting the byte size to 30000
Thank you
Read RFC 2616 Section 4.4. It tells you how to determine the end of the server's response so you know how many bytes to read. You have to read and process the server's response headers first in order to know how the remaining data, if any, is being transmitted. Then you can keep reading from the socket accordingly, potentially parsing what you have read, until the end of the response has actually been reached. Your current reading code is not even close to satisfying that requirement.
For example (pseudo code):
line = read a CRLF-delimited line;
responseNum = extract from line;
httpVer = extract from line;
do
{
line = read a CRLF-delimited line;
if (line == "") break;
add line to headers list;
}
while (true);
if (((responseNum / 100) != 1) &&
(responseNum != 204) &&
(responseNum != 304) &&
(request was not "HEAD"))
{
if ((headers has "Transfer-Encoding") &&
(headers["Transfer-Encoding"] != "identity"))
{
do
{
line = read a CRLF-delimited line;
chunkLen = extract from line, decode hex value;
if (chunkLen == 0) break;
read exactly chunkLen number of bytes;
read and discard a CRLF-delimited line;
}
while (true);
do
{
line = read a CRLF-delimited line;
if (line == "") break;
add line to headers list, overwrite if exists;
}
while (true);
decode/transform read data based on headers["Transfer-Encoding"] values if more than just "chunked"
}
else if (headers has "Content-Length")
{
read exactly headers["Content-Length"] number of bytes
}
else if (headers["Content-Type"] == multipart/byteranges)
{
boundary = headers["Content-Type"]["boundary"];
read and parse MIME encoded data until "--"+boundary+"--" line is reached;
}
else
{
read until disconnected;
}
}
if (((httpVer >= 1.1) && (headers["Connection"] == "close)) ||
((httpVer < 1.1) && (headers["Connection"] != "keep-alive")))
{
disconnect;
}
I leave it as an exercise for you to actually implement this in your code.
Related
This is rather ugly. We're getting weird hangups caused (read: triggered) by some security scanning software. I suspect it has a nonstandard TCP/IP stack, and the customer is asking why hangups.
Static analysis suggests only two possible locations for the hangup. The hangup either has to be in ReadFile() or WriteFile() on a socket; and WriteFile() cannot hang here unless the scanner is designed to make WriteFile() hang by setting the window size to zero. If WriteFile() were to return at all even if it didn't make progress I'd be able to knock the thing out of its wedged state. I also don't think the log state is consistent with WriteFile() returning.
So onto ReadFile(): this is the calling sequence:
SOCKET conn;
HANDLE unwedgeEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
HANDLE listenEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
if (listenEvent == NULL) return;
//...
conn = accept(lstn);
for (;;) {
HANDLE wakeup[2];
wakeup[0] = unwedgeEvent;
wakeup[1] = listenEvent;
if (WSAEventSelect(conn, socket, FD_READ | FD_CLOSE) == 0) {
// Log error
break;
}
which = WaitForMultipleObjects(2, wakeup, FALSE, INFINITE);
if (which < 0) {
// Log error
break;
}
if (which == 1) {
DWORD read;
r = ReadFile(conn, chunk, 4096, &read, NULL);
if (r == 0) {
// Handle error -- not stuck here
} else {
// Handle data -- not stuck here either
}
}
if (which == 0) break;
}
Where signalling unwedgeEvent doesn't manage to accomplish anything and the thread remains stuck forever.
So the real question is have I gone nuts or is this really a thing that can happen?
So this has gone somewhat off the deep end; I don't need non-blocking sockets at all. I need a select() that takes handle arguments to things that are sockets and things that are not sockets.
The following API sequences do not hang in ReadFile:
---------------------------------------------------------------
Sender B Receiver
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
................................................................
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 2)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
* WaitForMultipeObjects
send(buffer size = 1)
send(buffer size = 1)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 2)
* WaitForMultipeObjects
send(buffer size = 1)
ReadFile(size = 1)
................................................................
WSAEventSelect
send(buffer size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
send(buffer size = 1)
WaitForMultipeObjects
ReadFile(size = 1)
I see a number of issues with your code:
You should use WSACreateEvent() and WSAWaitForMultipleEvents() instead of CreateEvent() and WaitForMultipleObjects(). Although the current implementation is that the former APIs simply map to the latter APIs, Microsoft is free to change that implementation at any time without breaking code that uses the former APIs properly.
In your call to WSAEventSelect(), socket should be listenEvent instead.
WSAEventSelect() returns SOCKET_ERROR (-1) on failure, not 0 like you have coded.
You are not calling WSAEnumNetworkEvents() at all, which you need to do in order to determine if FD_READ was the actual type of event triggered, and to clear the socket's event state and reset the event object. So, you may be acting on a stale read state, which could explain why you end up calling ReadFile() when there is actually nothing available to read.
WSAEventSelect() puts the socket into non-blocking mode (per its documentation), so it is actually not possible for ReadFile() (or any other reading function) to block on the socket. However, it could fail immediately with a WSAEWOULDBLOCK error, so make sure you are not treating that condition as a fatal error.
The WSAEventSelect() documentation does not list ReadFile() as a supported function for re-enabling events when calling WSAEnumNetworkEvents(). Although the Socket Handles documentation does say that a Winsock SOCKET can be used with non-Winsock I/O functions like ReadFile(), it recommends that a SOCKET should only be used with Winsock functions. So, you should use send()/recv() or WSASend()/WSARecv() instead of WriteFile()/ReadFile().
With that said, try something more like the following:
HANDLE unwedgeEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (unwedgeEvent == NULL) {
// Log error
return;
}
...
SOCKET conn = accept(lstn, NULL, NULL);
if (conn == INVALID_SOCKET) {
// Log error
return;
}
...
WSAEVENT listenEvent = WSACreateEvent();
if (listenEvent == NULL) {
// Log error
}
else if (WSAEventSelect(conn, listenEvent, FD_READ | FD_CLOSE) == SOCKET_ERROR) {
// Log error
}
else {
WSAEVENT wakeup[2];
wakeup[0] = (WSAEVENT) unwedgeEvent;
wakeup[1] = listenEvent;
char chunk[4096];
int read;
do {
DWORD which = WSAWaitForMultipleEvents(2, wakeup, FALSE, WSA_INFINITE, FALSE);
if (which == WSA_WAIT_FAILED) {
// Log error
break;
}
if (which == WSA_WAIT_EVENT_0) {
break;
}
if (which == (WSA_WAIT_EVENT_0+1)) {
WSANETWORKEVENTS events = {};
if (WSAEnumNetworkEvents(conn, listenEvent, &events) == SOCKET_ERROR) {
// Log error
break;
}
if (events.lNetworkEvents & FD_READ) {
read = recv(conn, chunk, sizeof(chunk), 0);
if (read == SOCKET_ERROR) {
if (WSAGetLastError() != WSAEWOULDBLOCK) {
// Log error
break;
}
}
else if (read == 0) {
break;
}
else {
// Handle chunk up to read number of bytes
}
}
if (events.lNetworkEvents & FD_CLOSE) {
break;
}
}
}
while (true);
WSACloseEvent(listenEvent);
}
closesocket(conn);
I am new in this superb place. I got help several times from this site. I have seen many answers regarding my question that was previously discussed but i am facing problem to count the number of characters using FileReader. It's working using Scanner. This is what i tried:
class CountCharacter
{
public static void main(String args[]) throws IOException
{
File f = new File("hello.txt");
int charCount=0;
String c;
//int lineCount=0;
if(!f.exists())
{
f.createNewFile();
}
BufferedReader br = new BufferedReader(new FileReader(f));
while ( (c=br.readLine()) != null) {
String s = br.readLine();
charCount = s.length()-1;
charCount++;
}
System.out.println("NO OF LINE IN THE FILE, NAMED " +f.getName()+ " IS " +charCount);
}
}`
It looks to me that each time you go through the loop, you assign the charCount to be the length of the line that iteration of the loop is concerned with. i.e. instead of
charCount = s.Length() -1;
try
charCount = charCount + s.Length();
EDIT:
If you have say the document with the contents "onlyOneLine"
Then when you first hit the while check the br.readLine() will make the BufferredReader read the first line, during the while's code block however br.readLine() is called again which advances the BufferredReader to the second line of the document, which will return null. As null is assigned to s, and you call length(), then NPE is thrown.
try this for the while block
while ( (c=br.readLine()) != null) {
charCount = charCount + c.Length(); }
I am creating a client server app using a simple socket to transfer Protocol Buffer objects between C++ and Java. I have it created on the Java side both as the client and receiver. I even got the Java to send to C++ but I am having trouble with sending from C++ to Java. Not sure how to write it. First I am sending the size and then reading the proto object.
I am using Visual Studio C++ 2008. DataGram and Bookmartlet are my proto objects.
Here is my C++ Sender Client
int loopt = 99;
do {
DataGram dataGram;
dataGram.set_state("ACK");
time ( &rawtime );
dataGram.set_status(ctime(&rawtime));
Bookmarklet* bookmarklet = dataGram.mutable_bookmarklet();
bookmarklet->set_name("TEST");
bookmarklet->set_utl("TEST");
dataGram.SerializeToArray(recvbuf,recvbuflen);
dataGram.ByteSize();
//Trouble SPOT
send(ConnectSocket,size,1,0);
send(ConnectSocket,recvbuf,recvbuflen,0);
} while (loopt > 0);
This is what I am trying to replicate from Java.
OutputStream output = socket.getOutputStream();
try {
int testtimes = 99;
while (socket.isConnected() && testtimes > 0) {
Thread.sleep(1000); // do nothing for 1000 miliseconds (1
// second)
DataGram dataGram = null;
Bookmarklet bookmarklet = Bookmarklet.newBuilder()
.setName("TEST").setUtl("TEST").build();
if (testtimes % 2 == 0) {
dataGram = DataGram.newBuilder().setState("ACK")
.setBookmarklet(bookmarklet)
.setStatus(new Date().toGMTString()).build();
} else {
dataGram = DataGram.newBuilder().setState("ACK")
.setStatus(new Date().toGMTString()).build();
}
output.write(dataGram.getSerializedSize());
output.write(dataGram.toByteArray());
testtimes--;
}
} catch (Exception e) {
e.printStackTrace();
}
Any help would be great. Thanks.
Client in Java for Reference:
InputStream input = socket.getInputStream();
int count = input.read();
int counter = 0;
while (count > 0) {
byte[] buffer = new byte[count];
count = input.read(buffer);
DataGram dataGram = DataGram.parseFrom(buffer);
Bookmarklet bookmarklet = null;
if ((bookmarklet = dataGram.getBookmarklet()) != null) {
System.out.println(bookmarklet.toString());
}
System.out.println(dataGram.getState() + " "
+ dataGram.getStatus());
count = input.read();
System.out.println(++counter);
}
I am trying to read a xml file from the web and parse it out using XDocument. It normally works fine but sometimes it gives me this error for day:
**' ', hexadecimal value 0x1F, is an invalid character. Line 1, position 1**
I have tried some solutions from Google but they aren't working for VS 2010 Express Windows Phone 7.
There is a solution which replace the 0x1F character to string.empty but my code return a stream which doesn't have replace method.
s = s.Replace(Convert.ToString((byte)0x1F), string.Empty);
Here is my code:
void webClient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
using (var reader = new StreamReader(e.Result))
{
int[] counter = { 1 };
string s = reader.ReadToEnd();
Stream str = e.Result;
// s = s.Replace(Convert.ToString((byte)0x1F), string.Empty);
// byte[] str = Convert.FromBase64String(s);
// Stream memStream = new MemoryStream(str);
str.Position = 0;
XDocument xdoc = XDocument.Load(str);
var data = from query in xdoc.Descendants("user")
select new mobion
{
index = counter[0]++,
avlink = (string)query.Element("user_info").Element("avlink"),
nickname = (string)query.Element("user_info").Element("nickname"),
track = (string)query.Element("track"),
artist = (string)query.Element("artist"),
};
listBox.ItemsSource = data;
}
}
XML file:
http://music.mobion.vn/api/v1/music/userstop?devid=
0x1f is a Windows control character. It is not valid XML. Your best bet is to replace it.
Instead of using reader.ReadToEnd() (which by the way - for a large file - can use up a lot of memory.. though you can definitely use it) why not try something like:
string input;
while ((input = sr.ReadLine()) != null)
{
string = string + input.Replace((char)(0x1F), ' ');
}
you can re-convert into a stream if you'd like, to then use as you please.
byte[] byteArray = Encoding.ASCII.GetBytes( input );
MemoryStream stream = new MemoryStream( byteArray );
Or else you could keep doing readToEnd() and then clean that string of illegal characters, and convert back to a stream.
Here's a good resource for cleaning illegal characters in your xml - chances are, youll have others as well...
https://seattlesoftware.wordpress.com/tag/hexadecimal-value-0x-is-an-invalid-character/
What could be happening is that the content is compressed in which case you need to decompress it.
With HttpHandler you can do this the following way:
var client = new HttpClient(new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.GZip
| DecompressionMethods.Deflate
});
With the "old" WebClient you have to derive your own class to achieve the similar effect:
class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}
Above taken from here
To use the two you would do something like this:
HttpClient
using (var client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }))
{
using (var stream = client.GetStreamAsync(url))
{
using (var sr = new StreamReader(stream.Result))
{
using (var reader = XmlReader.Create(sr))
{
var feed = System.ServiceModel.Syndication.SyndicationFeed.Load(reader);
foreach (var item in feed.Items)
{
Console.WriteLine(item.Title.Text);
}
}
}
}
}
WebClient
using (var stream = new MyWebClient().OpenRead("http://myrss.url"))
{
using (var sr = new StreamReader(stream))
{
using (var reader = XmlReader.Create(sr))
{
var feed = System.ServiceModel.Syndication.SyndicationFeed.Load(reader);
foreach (var item in feed.Items)
{
Console.WriteLine(item.Title.Text);
}
}
}
}
This way you also recieve the benefit of not having to .ReadToEnd() since you are working with the stream instead.
Consider using System.Web.HttpUtility.HtmlDecode if you're decoding content read from the web.
If you are having issues replacing the character
For me there were some issues if you try to replace using the string instead of the char. I suggest trying some testing values using both to see what they turn up. Also how you reference it has some effect.
var a = x.IndexOf('\u001f'); // 513
var b = x.IndexOf(Convert.ToString((byte)0x1F)); // -1
x = x.Replace(Convert.ToChar((byte)0x1F), ' '); // Works
x = x.Replace(Convert.ToString((byte)0x1F), " "); // Fails
I blagged this
I had the same issue and found that the problem was a embedded in the xml.
The solution was:
s = s.Replace("", " ")
I'd guess it's probably an encoding issue but without seeing the XML I can't say for sure.
In terms of your plan to simply replace the character but not being able to, because you have a stream rather than a text, simply read the stream into a string and then remove the characters you don't want.
Works for me.........
string.Replace(Chr(31), "")
I used XmlSerializer to parse XML and faced the same exception.
The problem is that the XML string contains HTML codes of invalid characters
This method removes all invalid HTML codes from string (based on this thread - https://forums.asp.net/t/1483793.aspx?Need+a+method+that+removes+illegal+XML+characters+from+a+String):
public static string RemoveInvalidXmlSubstrs(string xmlStr)
{
string pattern = "&#((\\d+)|(x\\S+));";
Regex regex = new Regex(pattern, RegexOptions.IgnoreCase);
if (regex.IsMatch(xmlStr))
{
xmlStr = regex.Replace(xmlStr, new MatchEvaluator(m =>
{
string s = m.Value;
string unicodeNumStr = s.Substring(2, s.Length - 3);
int unicodeNum = unicodeNumStr.StartsWith("x") ?
Convert.ToInt32(unicodeNumStr.Substring(1), 16)
: Convert.ToInt32(unicodeNumStr);
//according to https://www.w3.org/TR/xml/#charsets
if ((unicodeNum == 0x9 || unicodeNum == 0xA || unicodeNum == 0xD) ||
((unicodeNum >= 0x20) && (unicodeNum <= 0xD7FF)) ||
((unicodeNum >= 0xE000) && (unicodeNum <= 0xFFFD)) ||
((unicodeNum >= 0x10000) && (unicodeNum <= 0x10FFFF)))
{
return s;
}
else
{
return String.Empty;
}
})
);
}
return xmlStr;
}
Nobody can answer if you don't show relevant info - I mean the Xml content.
As a general advice I would put a breakpoint after ReadToEnd() call. Now you can do a couple of things:
Reveal Xml content to this forum.
Test it using VS Xml visualizer.
Copy-paste the string into a txt file and investigate it offline.
I'm trying to send a list of 600 records over TCP/IP sockets using a java server and a Blackberry client. But every time it reaches the 63th record it stops, the odd thing about this is that if I only send 200 records they are sent ok.
I haven't been able to understand why it happens, only that 63 records equals aprox to 4kB, basically it sends:
an integer with the total number of records to be sent
And for every record
an integer with the length of the string
the string
a string terminator "$$$"
Since i need to send the whole 600 i have tried to close the InputStreamReader and reopen it, also reset it but without any result.
Does anybody else have experienced this behaviour? thanks in advanced.
EDIT
Here the code that receives:
private String readfromserver() throws IOException {
int len=_in.read(); // receives the string length
if (len==0) // if len=0 then the string was empty
return "";
else {
char[] input = new char[len+1];
for (int i = 0; i < len; ++i)
input[i] = (char)_in.read();
StringBuffer s = new StringBuffer();
s.append(input);
return s.toString();
}
}
private void startRec(String data) throws IOException
{
boolean mustcontinue=true;
int len=_in.read(); // read how many records is about to receive
if (len==0) {
scr.writelog("There is no data to receive");
}
else {
for(int i=0; i<len; i++)
if (mustcontinue) {
mustcontinue=mustcontinue && showdata(readfromserver());
}
else {
scr.writelog("Inconsistency error #19");
}
}
}
the function showdata only shows the received string in a LabelField.
The code in the server:
try {
_out.write(smultiple.size()); // send the number of records
_out.flush();
for (int x=0; x<smultiple.size(); x++)
{
int l=smultiple.elementAt(x).length();
_out.write(l); // send string length
if (l>0)
_out.write(smultiple.elementAt(x)); // send string
}
_out.flush();
} catch (Exception e) {
principal.dblog(e.toString());
}
smultiple is a vector containing the strings and everyone already have the terminator $$$.
Thanks.
I think 200 goes fine and 600 -not, because the latter number is bigger than 255 :-) Your code
int len=_in.read();
probably reads a byte and not an integer (4 bytes)