I'm using the MongoDB C# driver 1.8.1.20 with Mongo 2.4.3. I use the following infinite loop to poll new messages from a capped collection and process them as they come (with a tailable cursor and await data). It works for the most part, but in production, it seems that from time to time the call to enumerator.MoveNext() blocks and never returns. This causes the loop to stall, and my application no longer receives updates. It seems to be happening when the connection is closed unexpectedly.
while (true)
{
try
{
using (MongoCursorEnumerator<QueueMessage> enumerator = GetCursor())
{
while (!enumerator.IsDead)
{
while (enumerator.MoveNext()) // This is blocking forever when connection is temporarily lost
this.processMessage(enumerator.Current);
}
}
}
catch (Exception ex)
{
Trace.TraceError("Error in the ReceiveCore loop: " + ex.ToString());
}
}
The GetCursor function does this:
cursor = (MongoCursorEnumerator<QueueMessage>)collection
.FindAllAs<QueueMessage>()
.SetFlags(QueryFlags.AwaitData | QueryFlags.TailableCursor | QueryFlags.NoCursorTimeout)
.SetSortOrder(SortBy.Ascending("$natural"))
.GetEnumerator();
Why is that blocking forever, and what can I do to make sure it throws an exception when it can't complete (possibly by timing out)?
I think I would just remove the QueryFlags.NoCursorTimeout, then let it timeout occasionally and just restart.
Related
In the tokio-postgres documentation in the first example, there is an example showing that you should run the database connection in a separate thread:
// The connection object performs the actual communication with the database,
// so spawn it off to run on its own.
tokio::spawn(async move {
if let Err(e) = connection.await {
eprintln!("connection error: {}", e);
}
});
If you do so, how can you kill that connection afterwards?
If you're on tokio 1, tokio::task::JoinHandle has an abort() function that cancels the task, thus dropping the connection.
let handle = task::spawn(async move {
if let Err(e) = connection.await {
eprintln!("connection error: {}", e);
}
}
handle.abort(); // this kills the task and drops the connection
Using my snippet as-is will immediately kill the task, thus this is probably not what you want in the end, but if you keep the handle around and use it e.g. in combination with some kind of shutdown listener you should be able to control the connection as wanted.
I am opening a socket in jmeter (using groovy in JSR223 Sampler), and storing the message in a jmeter variable. This is the below code:
SocketAddress inetSocketAddress = new InetSocketAddress(InetAddress.getByName("localhost"),4801);
def server = new ServerSocket()
server.bind(inetSocketAddress)
while(!vars.get("caseId"))) {
server.accept { socket ->
log.info('Someone is connected')
socket.withStreams { input, output ->
InputStreamReader isReader = new InputStreamReader(input);
BufferedReader reader = new BufferedReader(isReader);
StringBuffer sb = new StringBuffer();
String str;
while((str = reader.readLine())!= null){
sb.append(str);
}
String finalStr = sb.toString()
String caseId = finalStr.split("<caseId>")[1].split("</caseId>")[0]
vars.put("caseId", caseId)
}
log.info("Connection processed")
}
}
if(vars.get("caseId"))
{
try
{
server.close();
vars.put("socketClose",true);
}
catch(Exception e)
{
log.info("Error in closing the socket: " + e.getMessage());
}
}
Now, there is some time delay between the first loop is executed and the message being recieved from the port. It doesnt receive the message immediately, and hence while loop is executed again. And then message is received and it sets caseId. It goes on to close the socket, because caseId is set. And that is throwing the error, because socket is still waiting for the message. So is there a way, to wait until socket has recieved all the messages, so i could properly close it?
Or just force close the socket, and Jmeter wont throw any exception?
Or when i execute next component, say IF controller in Jmeter, it waits until variable socketClose is set true? In that way, instead of while loops inside JSR223 sampler, i could use multiple If Controllers in Jmeter thread.
This is how ServerSocket.close() function works
public void close()
throws IOException
Closes this socket. Any thread currently blocked in accept() will throw a SocketException.
I don't think there is a way "to wait until socket has recieved all the messages" because Socket is dump as a rock and it can either listen for connections or shut down.
Maybe you might be interested in setSoTimeout() function?
Also this line:
vars.put("socketClose",true)
is very suspicious, I think you need to change it either to:
vars.put("socketClose", "true")
or to
vars.putObject("socketClose",true)
as JMeterVariables.put() function can accept only a String, see Top 8 JMeter Java Classes You Should Be Using with Groovy article for more details.
Suppose "doc" is some document I want to insert into a MongoDB collection and "collection" is the collection I am inserting the document into.
I have something like the following:
try
{
WriteConcern wc = new WriteConcern();
wc.W = 1;
wc.Journal = true;
WriteConcernResult wcResult = collection.Insert(doc, wc);
if (!string.IsNullOrWhiteSpace(wcResult.ErrorMessage) || !wcResult.Ok)
{
return ErrorHandler(...);
}
else
{
return SuccessFunction(...);
}
}
catch (Exception e)
{
return e.Message;
}
Basically, if the insertion fails for any reason (other than hardware no longer working properly) I want to handle it (through the ErrorHandler function or the catch clause), while if it succeeds I want to call SuccessFunction.
My question: Is the above code sufficient for error checking purposes? In other words, will all failed insertions be caught, so that SuccessFunction is never called in those situations?
You don't even need to do any checking. collection.Insert will throw an exception if the write was not successful when you are using any write concern other than unacknowledged.
If you want to know if an error occured, you need to catch a WriteConcernException.
I seem to be struggling with the std::io::TcpStream. I'm actually trying to open a TCP connection with another system but the below code emulates the problem exactly.
I have a Tcp server that simply writes "Hello World" to the TcpStream upon opening and then loops to keep the connection open.
fn main() {
let listener = io::TcpListener::bind("127.0.0.1", 8080);
let mut acceptor = listener.listen();
for stream in acceptor.incoming() {
match stream {
Err(_) => { /* connection failed */ }
Ok(stream) => spawn(proc() {
handle(stream);
})
}
}
drop(acceptor);
}
fn handle(mut stream: io::TcpStream) {
stream.write(b"Hello Connection");
loop {}
}
All the client does is attempt to read a single byte from the connection and print it.
fn main() {
let mut socket = io::TcpStream::connect("127.0.0.1", 8080).unwrap();
loop {
match socket.read_byte() {
Ok(i) => print!("{}", i),
Err(e) => {
println!("Error: {}", e);
break
}
}
}
}
Now the problem is my client remains blocked on the read until I kill the server or close the TCP connection. This is not what I want, I need to open a TCP connection for a very long time and send messages back and forth between client and server. What am I misunderstanding here? I have the exact same problem with the real system i'm communicating with - I only become unblocked once I kill the connection.
Unfortunately, Rust does not have any facility for asynchronous I/O now. There are some attempts to rectify the situation, but they are far from complete yet. That is, there is a desire to make truly asynchronous I/O possible (proposals include selecting over I/O sources and channels at the same time, which would allow waking tasks which are blocked inside an I/O operation via an event over a channel, though it is not clear how this should be implemented on all supported platforms), but there's still a lot to do and there's nothing really usable now, as far as I'm aware.
You can emulate this to some extent with timeouts, however. This is far from the best solution, but it works. It could look like this (simplified example from my code base):
let mut socket = UdpSocket::bind(address).unwrap();
let mut buf = [0u8, ..MAX_BUF_LEN];
loop {
socket.set_read_timeout(Some(5000));
match socket.recv_from(buf) {
Ok((amt, src)) => { /* handle successful read */ }
Err(ref e) if e.kind == TimedOut => {} // continue
Err(e) => fail!("error receiving data: {}", e) // bail out
}
// do other work, check exit flags, for example
}
Here recv_from will return IoError with kind set to TimedOut if there is no data available on the socket during 5 seconds inside recv_from call. You need to reset the timeout before inside each loop iteration since it is more like a "deadline" than a timeout - when it expires, all calls will start to fail with timeout error.
This is definitely not the way it should be done, but Rust currently does not provide anything better. At least it does its work.
Update
There is now an attempt to create an asynchronous event loop and network I/O based on it. It is called mio. It probably can be a good temporary (or even permanent, who knows) solution for asynchronous I/O.
The error above occurs when I try to do a dataReader.Read on the data recieved from the database. I know there are two rows in there so it isnt because no data actually exists.
Could it be the CommandBehavior.CloseConnection, causing the problem? I was told you had to do this right after a ExecuteReader? Is this correct?
try
{
_connection.Open();
using (_connection)
{
SqlCommand command = new SqlCommand("SELECT * FROM Structure", _connection);
SqlDataReader dataReader = command.ExecuteReader(CommandBehavior.CloseConnection);
if (dataReader == null) return null;
var newData = new List<Structure>();
while (dataReader.Read())
{
var entity = new Structure
{
Id = (int)dataReader["StructureID"],
Path = (string)dataReader["Path"],
PathLevel = (string)dataReader["PathLevel"],
Description = (string)dataReader["Description"]
};
newData.Add(entity);
}
dataReader.Close();
return newData;
}
}
catch (SqlException ex)
{
AddError(new ErrorModel("An SqlException error has occured whilst trying to return descendants", ErrorHelper.ErrorTypes.Critical, ex));
return null;
}
catch (Exception ex)
{
AddError(new ErrorModel("An error has occured whilst trying to return descendants", ErrorHelper.ErrorTypes.Critical, ex));
return null;
}
finally
{
_connection.Close();
}
}
Thanks in advance for any help.
Clare
When you use the Using in C#, after the last } from the using, the Connection automatically close, thats why you get the fieldcount to be closed when u try to read him, as that is impossible, because u want those datas, read then before close the using, or u can open and close manually the connection, by not using the (using)
Your code, as displayed is fine. I've taken it into a test project, and it works. It's not immediately clear why you get this message with the code shown above. Here are some debugging tips/suggestions. I hope they're valuable for you.
Create a breakpoint on the while (dataReader.Read()). Before it enters its codeblock, enter this in your Immediate or Watch Window: dataReader.HasRows. That should evaluate to true.
While stopped on that Read(), open your Locals window to inspect all the properties of dataReader. Ensure that the FieldCount is what you expect from your SELECT statement.
When stepping into this Read() iteration, does a student object get created at all? What's the value of dataReader["StructureID"] and all others in the Immediate Window?
It's not the CommandBehavior.CloseConnection causing the problem. That simply tells the connection to also close itself when you close the datareader.
When I got that error, it happened to be a command timeout problem (I was reading some large binary data). As a first attempt, I increased the command timeout (not the connection timeout!) and the problem was solved.
Note: while attempting to find out the problem, I tried to listen to the (Sql)connection's StateChanged event, but it turned out that the connection never fall in a "broken" state.
Same problem here. Tested all the above solutions
increase command timeout
close the connection after read
Here's the code
1 objCmd.Connection.Open()
2 objCmd.CommandTimeout = 3000
3 Dim objReader As OleDbDataReader = objCmd.ExecuteReader()
4 repeater.DataSource = objReader
5 CType(repeater, Control).DataBind()
6 objReader.Close()
7 objCmd.Connection.Dispose()
Moreover, at line 4 objReader has Closed = False
I got this exception while using the VS.NET debugger and trying to examine some IQueryable results. Bad decision because the IQueryable resulted in a large table scan. Stopping and restarting the debugger and NOT trying to preview this particular IQueryable was the workaround.