Boost Asio tcp::iostream construction raise an Access Violation Exception on every second use - sockets

I am trying to use the implementation of std::iostream provided by boost::asio on top of boost::asio::ip::tcp::socket. My code replicate almost line to line the example that is published in Boost Asio's documentation:
#include <iostream>
#include <stdexcept>
#include <boost/asio.hpp>
int main()
{
using boost::asio::ip::tcp;
try
{
boost::asio::io_service io_service;
tcp::endpoint endpoint(tcp::v4(), 8000);
tcp::acceptor acceptor(io_service, endpoint);
for (;;)
{
tcp::iostream stream; // <-- The exception is triggered on this line, on the second loop iteration.
boost::system::error_code error_code;
acceptor.accept(*stream.rdbuf(), error_code);
std::cout << stream.rdbuf() << std::flush;
}
}
catch (std::exception& exception)
{
std::cerr << exception.what() << std::endl;
}
return 0;
}
The only difference is the use I make of the resulting tcp::iostream: I forward everything I receive to the standard output.
When I compile this code with VisualStudio2019/toolset v142 and Boost from the NuGet boost-vc142, I get an Access Violation Exception only in the second iteration in the for loop, in the function
template <typename Service>
Service& service_registry::use_service(io_context& owner)
{
execution_context::service::key key;
init_key<Service>(key, 0);
factory_type factory = &service_registry::create<Service, io_context>;
return *static_cast<Service*>(do_use_service(key, factory, &owner));
} // <-- The debugger show the exception was raised on this line
in asio/detail/impl/service_registry.hpp. So the first iteration everything goes as planned, the connection is accepted, the data shows up on the standard output, and as soon as the stream is instanciated on the stack for the second time, the exception pops.
I don't have a high confidence in the accuracy of this location of the exception reported by the debugger. For some reason, the stack seams to be messed up and show only one frame.
If the declaration of stream is moved out of the loop, no exception is raised any more but then I need to stream.close() at the end of the loop, or nothing shows up on the standard output except the data from the first client's connection.
Basically, as soon as I try to instanciate more than one boost::asio::tcp::iostream (not necessarily at the same time), the exception is raised.
I tried the exact same code under linux (Arch linux, latest version of g++, same version of Boost) and everything works perfectly.
I could work around this issue by not using iostreams, but my idea is to feed the data received on the tcp socket to a parser which only accept implementations of std::iostream, hence I would still need to wrap asio's tcp socket in an homebrewed (and mediocre) implementation of std::iostream.
Does anybody have an idea on what's wrong with this setup, if I missed a crucial #define somewhere or anything?
Update:
Subsequent investigation show that the only situation where the access violation happens is when the executable is run from within Visual Studio (typ. from the menu Debug -> Start Debugging).
The build process seems to have no effect (calling directly cl.exe, using MSBuild, using devenv.exe).
Moreover, if the executable is run from a command prompt, and only then the debugger is attached, no access violation happens.
At this point, the issue is most likely not linked to the code itself.

Okay, it was exceedingly painful to test this on windows.
Of course I first tried on Linux (clang/gcc) and MingW 8.1 on windows.
Then I bit the bullet and jumped the hoops to get MSVC in command line with boost packages¹.
I cheated by manually copying the .lib/.dll for boost_{system,date_time,regex} into the working directory so the command line stayed "wieldy":
C:\work>C:\Users\sghee\Downloads\nuget.exe install boost_system-vc142
C:\work>C:\Users\sghee\Downloads\nuget.exe install boost_date_time-vc142
C:\work>C:\Users\sghee\Downloads\nuget.exe install boost_regex-vc142
(Be sure to get some coffee during those)
C:\work\> cl /EHsc test.cpp /I .\boost.1.72.0.0\lib\native\include /link
Now I can run test.exe
C:\work\> test.exe
And it listens fine, accepts connections (sequentially, not simultaneously). If you connect a second client while the first is still connected, it will be queued and be accepted only after the first disconnects. That's fine, because it's what you expect with the synchronous accept and loop.
I used Ncat.exe (from Nmap) to connect:
C:\Program Files (x86)\Nmap>.\ncat.exe localhost 8000
Quirk: The buffering was fine with the MSVC cl.exe build (linewise) as opposed to MingW behaviour, even though MingW also uses ws2_32.dll. #trivia
I know this doesn't "help", but maybe you can compare notes and see what is different with your system.
Video Of Test
¹ (that's a tough job without VS and also I - obviously - ran out of space, because 50GiB for a VM can't be enough right)

Related

Opencascade crash when calling calling Transfer()

I have tested two cases:
I use STEPCAFControl_Reader then STEPControl_Reader to read my step file but both methods crash when I call STEPCAFControl_Reader::Transfer, repsectively STEPControl_Reader:: TransferRoots.
By using STEPControl_Reader, I displayed a log on my console, then there is a message like this:
1 F:(BOUNDED_SURFACE,B_SPLINE_SURFACE,B_SPLINE_SURFACE_WITH_KNOTS,GEOMETRIC_REPRESENTATION_ITEM,RATIONAL_B_SPLINE_SURFACE,REPRESENTATION_ITEM,SURFACE): Count of Parameters is not 1 for representation_item
EDIT:
There is a null reference inside TransferRoots() method.
const Handle(Transfer_TransientProcess) &proc = thesession->TransferReader()->TransientProcess();
if (proc->GetProgress().IsNull())
{
//This condition does not exist from the source code
std::cout << "GetProgress is null" << std::endl;
return 0;
}
Message_ProgressSentry PS ( proc->GetProgress(), "Root", 0, nb, 1 );
My app and FreeCAD crash but if I use CAD Assitant which OCC official viewer, it loads.
It looks like comments already provide an answer to the question - or more precisely answers:
STEPCAFControl_Reader::ReadFile() returns reading status, which should be checked before calling STEPCAFControl_Reader::Transfer().
Normally, it is a good practice to put OCCT algorithm into try/catch block and check for OCCT exceptions (Standard_Failure).
Add OCC_CATCH_SIGNALS at the beginning of try statements (required only on Linux) and OSD::SetSignal(false) within working thread creation to redirect abnormal cases (access violation, NULL dereference and others) to C++ exceptions (OSD_Signal which is subclass of Standard_Failure). This may conflict other signal handlers in mixed environment - so check also documentation of other frameworks used by application.
If you catch failures like NULL dereference on calling OCCT algorithm with valid arguments - this is a bug in OCCT which is desirable to be fixed in one or another way, even if input STEP file contains syntax/logical errors triggering such kind of issues. Report the issue on OCCT Bugtracker with sufficient information for reproducing bug, including sample files - it is not helpful to developers just saying that OCCT crashes somewhere. Consider also contributing into this open source project by debugging OCCT code and suggesting patches.
Check STEP file reading log for possible errors in the file itself. Consider reporting an issue to system producing a broken file, even if main file content can be loaded by STEP readers.
It is a common practice to use OSD::SetSignal() within OCCT-based applications (like CAD Assistant) to improve their robustness on non-fatal errors in application/OCCT code. It is more user friendly reporting an internal error message instead of silently crashing.
But it should be noted, that OSD::SetSignal() doesn't guarantee application not being crashed nor that application can work properly after catching such failure - due to asynchronous nature of some signals, the memory can be already corrupted at the moment, when C++ exception has been raised leading to all kinds of undesired behavior. For that reason, it is better not ignoring such kind of exceptions, even if it looks like application works fine with them.
OSD::SetSignal(false); // should be called ones at application startup
STEPCAFControl_Reader aReader;
try
{
OCC_CATCH_SIGNALS // necessary for redirecting signals on Linux
if (aReader.ReadFile (theFilePath) != IFSelect_RetDone) { return false; }
if (!aReader.Transfer (myXdeDoc)) { return false; }
}
catch (Standard_Failure const& theFailure)
{
std::cerr << "STEP import failed: " << theFailure.GetMessageString() << "\n";
return false;
}
return true;

Crash inside http_client constructor (Casablanca SDK)

I'm trying to use Casablanca to consume a REST api.
I've been following the microsoft tutorial, how ever i'm getting a crash and I cannot figure it out.
I'm using visual studio 2017 with C++11
I've codded a function GetRequest() that do work when used in a new empty project, but when I try to use it on my Project (Very big project with millions of code lines).
I'm crashing in the constructor of http_client, in the file xmemory0 line 118.
const uintptr_t _Ptr_container = _Ptr_user[-1];
This is a link to the callstack : https://i.imgur.com/lBm0Hv7.png
void RestManager::GetRequest()
{
auto fileStream = std::make_shared<ostream>();
// Open stream to output file.
pplx::task<void> requestTask = fstream::open_ostream(U("results.html")).then([=](ostream outFile)
{
*fileStream = outFile;
// Create http_client to send the request.
http_client client(U("XXX/XXX.svc/"));
// Build request URI and start the request.
uri_builder builder(U("/IsLive"));
builder.append_query(U("q"), U("cpprestsdk github"));
return client.request(methods::GET, builder.to_string());
})
// Handle response headers arriving.
.then([=](http_response response)
{
printf("Received response status code:%u\n", response.status_code());
// Write response body into the file.
return response.body().read_to_end(fileStream->streambuf());
})
// Close the file stream.
.then([=](size_t)
{
return fileStream->close();
});
// Wait for all the outstanding I/O to complete and handle any exceptions
try
{
requestTask.wait();
}
catch (const std::exception &e)
{
printf("Error exception:%s\n", e.what());
}
}
EDIT : I just want to add that the http_client constructor is the issue. It always crash inside it no matter what I send as parameter.
The wierd thing is that it's not crashing when i just make a main() that call this function.
I guess it must be due to some memory issues, however I have no idea how could I debug that.
Does anyone would have an idea about it?
Thanks and have a great day!
I've experienced a similar issue on ubuntu. It works in an empty project, but crashes randomly when put into an existing large project, complaining memory corruptions.
Turns out that the existing project loaded a proprietary library, which is using cpprestsdk (casablanca) internally. Even cpprestsdk is static linked, its symbols are still exported as Weak Symbols. So either my code crashes, or the proprietary library crashes.
Ideally, my project can be divided into several libraries, and load them with RTLD_LOCAL to avoid symbol clashes. But the proprietary library in my project only accept RTLD_GLOBAL, otherwise it crashes... So the import order and flags become important:
dlopen("my-lib-uses-cpprest", RTLD_LOCAL); //To avoid polluting the global
dlopen("proprietary-lib-with-built-in-cpprest", RTLD_GLOBAL); //In my case, this lib must be global
dlopen("another-lib-uses-cpprest", RTLD_DEEPBIND); //To avoid being affected by global
"it will probably never concern anyone."
I agree with that.
I guess this issues was very specific, and it will probably never concern anyone, but still I'm going to update on everything I found out about it.
On this project, we are using custom allocator, if i'm not wrong, it's not possible to give our custom allocator to this lib, which result to many random crash.
A good option to fix it would be to use the static version to this lib, however, since we are using a lot of dynamic lib, this option wasn't possible for us.
If you are on my case, I would advice to use the libcurl and rapidjson, it's a bit harder to use, but you can achieve the same goal.

Can't process quickfix messages written in file

In my C++ QuickFix application, I am recording all MarketDataIncrementalRefresh messages i am getting into a file. This is being done using the following code:
void Application::onMessage(const FIX44::MarketDataIncrementalRefresh& message, const FIX::SessionID&)
{
ofstream myfile("tapedol.txt", std::ios::app);
myfile << message << endl << endl;
}
This part's working just fine. The problem occurs when I try to load the message later on.
FIX::Message msg
ifstream myfile("tapedol.txt");
getline(myfile,aux);
msg = aux;
msg.getField(55);
The program crashes every time it executes the last line. I suspect the problem is at the assignment to msg, but i'm not sure. If it is, what is the correct way to do such assignment? If not, how can I process the data within tapedol.txt, so that a message of type MarketDataIncremental refresh would be generated for each string in the file?
Your question is not complete enough to provide a full answer, but I do see one red flag:
msg.getField(55);
The Symbol field is not a top-level field of MarketDataIncrementalRefresh (it's inside the NoMDEntries repeating group), so this line will fail. I think it would raise a FieldNotFound exception.
My C++ is rusty, but you should be able to catch an exception or something that should tell you exactly what line is erroring out. Barring that, you need to open up a debugger. Just saying "it crashed" means you quit looking too soon.

gem5 cache statistics - reset and dump

I am trying to get familiar with gem5 simulator.
To start, I wrote a simple program with
int main()
{
m5_reset_stats(0, 0);
m5_dump_stats(0, 0);
return 0;
}
I compiled it with util/m5/m5op_x86.S and ran it using...
./build/X86/gem5.opt configs/example/se.py --caches -c ~/tmp/hello
The m5out/stats.txt shows (among other things)...
system.cpu.dcache.ReadReq_hits::total 881
system.cpu.dcache.WriteReq_hits::total 917
system.cpu.dcache.ReadReq_misses::total 54
system.cpu.dcache.WriteReq_misses::total 42
Why is an empty function showing so much hits and misses? Are the hits and misses caused by libc? If so, then what is the purpose of m5_reset_stats() and m5_dump_stats()?
I would check in the stats.txt file if there are two chunks of
---Begin---
---End-----
because as you explained it, the simulator is supposed to dump the stats at dump_stats(0,0) and at the end of the run. So, it seems like you either are looking at one of those intervals (and I would expect the other interval to have 0 for all stats); or there was a bug in the simulation and the dump_stats() (or reset_stats())didn't actually do anything. That actually happened to me plenty of times, but I am not really sure as to the source of this bug.
If you want to troubleshoot further, you could do the following:
Look at the disassembly of your code and find the reset_stats.w and dump_stats.w
Dump a trace from gem5 and see if it ends up executing the dump and reset instructions and also what instructions (and how many) are executed before/after.
Hope this helps!

Output to command-line if started from command line

I'm writing an application that can be started either as a standard WinForms app or in unattended mode from the command-line. The application was built using the VS 2k5 standard WinForms template.
When the application is executed from the command-line, I want it to output information that can be captured by the script executing the application. When I do this directly from Console.WriteLine(), the output does not appear, although it can be captured by piping to a file.
On the other hand, I can force the application to pop up a second console by doing a P/Invoke on AllocConsole() from kernel32. This is not what I want, though. I want the output to appear in the same window the application was called from.
This is the salient code that allows me to pop up a console from the command line:
<STAThread()> Public Shared Sub Main()
If My.Application.CommandLineArgs.Count = 0 Then
Dim frm As New ISECMMParamUtilForm()
frm.ShowDialog()
Else
Try
ConsoleControl.AllocConsole()
Dim exMan As New UnattendedExecutionManager(ConvertArgs())
IsInConsoleMode = True
OutputMessage("Application started.")
If Not exMan.SetSettings() Then
OutputMessage("Execution failed.")
End If
Catch ex As Exception
Console.WriteLine(ex.ToString())
Finally
ConsoleControl.FreeConsole()
End Try
End If
End Sub
Public Shared Sub OutputMessage(ByVal msg As String, Optional ByVal isError As Boolean = False)
Trace.WriteLine(msg)
If IsInConsoleMode Then
Console.WriteLine(msg)
End If
If isError Then
EventLog.WriteEntry("ISE CMM Param Util", msg, EventLogEntryType.Error)
Else
EventLog.WriteEntry("ISE CMM Param Util", msg, EventLogEntryType.Information)
End If
End Sub
Raymond Chen recently posted (a month after the question was posted here on SO) a short article about this:
How do I write a program that can be run either as a console or a GUI application?
You can't, but you can try to fake it.
Each PE application contains a field
in its header that specifies which
subsystem it was designed to run
under. You can say
IMAGE_SUBSYSTEM_WINDOWS_GUI to mark
yourself as a Windows GUI application,
or you can say
IMAGE_SUBSYSTEM_WINDOWS_CUI to say
that you are a console application. If
you are GUI application, then the
program will run without a console.
The subsystem determines how the
kernel prepares the execution
environment for the program. If the
program is marked as running in the
console subsystem, then the kernel
will connect the program's console to
the console of its parent, creating a
new console if the parent doesn't have
a console. (This is an incomplete
description, but the details aren't
relevant to the discussion.) On the
other hand, if the program is marked
as running as a GUI application, then
the kernel will run the program
without any console at all.
In that article he points to another by Junfeng Zhang that discusses how a couple of programs (Visual Studio and ildasm) implement this behavior:
How to make an application as both GUI and Console application?
In VisualStudio case, there are actually two binaries: devenv.com and devenv.exe. Devenv.com is a Console app. Devenv.exe is a GUI app. When you type devenv, because of the Win32 probing rule, devenv.com is executed. If there is no input, devenv.com launches devenv.exe, and exits itself. If there are inputs, devenv.com handles them as normal Console app.
In ildasm case, there is only one binary: ildasm.exe. It is first compiled as a GUI application. Later editbin.exe is used to mark it as console subsystem. In its main method it determines if it needs to be run as console mode or GUI mode. If need to run as GUI mode, it relaunches itself as a GUI app.
In the comments to Raymond Chen's article, laonianren has this to add to Junfeng Zhang's brief description of how Visual Studio works:
devenv.com is a general purpose console-mode stub application. When it runs it creates three pipes to redirect the console's stdin, stdout and stderr. It then finds its own name (usually devenv.com), replaces the ".com" with ".exe" and launches the new app (i.e. devenv.exe) using the read end of the stdin pipe and the write ends of the stdout and stderr pipes as the standard handles. Then it just sits and waits for devenv.exe to exit and copies data between the console and the pipes.
Thus even though devenv.exe is a gui app it can read and write the "parent" console using its standard handles.
And you could use devenv.com yourself for myapp.exe by renaming it to myapp.com. But you can't in practise because it belongs to MS.
Update 1:
As said in Michael Burr answer, Raymond Chen recently posted a short article about this. I am happy to see that my guess was not totally wrong.
Update 0:
Disclaimer: This "answer" is mostly speculation. I post it only because enough time has passed to establish that not many people have the answer to what look like a fundamental question.
I think that the "decision" if the application is gui or console is made at compile time and not at runtime. So if you compile your application as gui application, even if you don't display the gui, its still a gui application and doesn't have console. If you choose to compile it as console application then at minimum you will have a console windows flashing before moving to gui "mode". And I don't know if it is possible in managed code.
The problem is fundamental, I think, Because a console application has to take "control" of the calling console application. And it has to do so before the code of the child application is running.
If you want to check if your app is started from the command line in .NET, you can use Console.GetCursorPosition().
The reason that this works is that when you start it from the command line, the cursor moves away from the initial point ((0, 0)) because you typed something in the terminal (the name of the app).
You can do this with an equality check (code in C#):
class Program
{
public static void Main
{
if (Console.GetCursorPosition() == (0, 0))
{
//something GUI
}
else
{
//something not GUI
}
}
}
Note: You must set the output type to Console Application as other output types will make Console.GetCursorPosition() throw an exception.