Redirect stderr to file on network driver is very slow since VC++ 2015 / 14.0 - redirect

We experience a strange performance issue when using Visual C++ 2013 (also known as Visual C++ 12.0) compared to more recent versions like Visual C++ 2015 (also known as Visual C++ 14.0):
The same C code is much slower when writing to stderr and redirecting this stream to a file on a network drive (Server is Linux Debian 8 / Samba).
C:\tmp> myprog.exe 2>X:\devel\fgl\tests\zz
When redirecting to a file on the local disk, or when redirecting to nul, the execution is fast.
We have also tested with VC++ 2017 / 15.0, same problem.
The reason seams to be the /MD option (link with MSVCRT.LIB), I could reproduce with a simple program:
#include <stdio.h>
int main(int argc, char **argv)
{
int n;
system("echo %TIME%");
for (n=0; n<200; n++) {
fprintf(stderr,"aaaaaaaaaaaaaaaaaaaaaaaaaaaa %d\n", n);
}
system("echo %TIME%");
return 0;
}
Then compile with and without /MD option:
C:\tmp>cl /O1 myprog.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.11.25507.1 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
myprog.c
Microsoft (R) Incremental Linker Version 14.11.25507.1
Copyright (C) Microsoft Corporation. All rights reserved.
/out:myprog.exe
myprog.obj
C:\tmp>myprog.exe 2>X:\devel\fgl\tests\zz
9:32:27.96
9:32:27.97
C:\tmp>cl /O1 /MD myprog.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.11.25507.1 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
myprog.c
Microsoft (R) Incremental Linker Version 14.11.25507.1
Copyright (C) Microsoft Corporation. All rights reserved.
/out:myprog.exe
myprog.obj
C:\tmp>myprog.exe 2>X:\devel\fgl\tests\zz
9:32:37.15
9:32:40.12
C:\tmp>myprog.exe 2>c:\tmp\zz
9:32:47.15
9:32:47.20
Is there any known issue related to stderr / network drives with VC++?
Notes:
No problem when writing to stdout
No problem when using /MDd option (MSVCRTD.lib)
No problem with Visual C++ 2013 (Visual C++ 12.0, cl version 18.00.40629)
Thanks for reading!

Er, we have been experiencing the same issue with our C++ code so I added my comment using C++, initially. Sorry about that.
For C, the same "remedy" would look like this:
char buf[4096];
setvbuf(stderr, _IOLBF/*or _IOFBF*/, buf, sizeof(buf));
Initial response:
Try this at the beginning of your program:
static char buf[16384];
cerr.rdbuf()->pubsetbuf(buf, sizeof(buf));
cerr.unsetf(std::ios_base::unitbuf);
You can decrease the buffer size (to 4096 or less) and still see the speed boost -- the thing is that cerr is completely unbuffered by default, and having unitbuf set on it causes even more slowdown by implicit flush operations injected after each output to the stream.

Related

Facing issue while running my first C program in VS Code

I have written my first C program and while running facing problem,Below is the code.
#include<stdio.h>
void main()
{
printf("Hello World");
}
While running the program getting below warning in terminal.
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6
Warning: PowerShell detected that you might be using a screen reader and has disabled PSReadLine for compatibility purposes. If you want to re-enable it, run 'Import-Module PSReadLine'.
PS D:\C_Program> & 'c:\Users\azhar.vscode\extensions\ms-vscode.cpptools-1.11.5-win32-x64\debugAdapters\bin\WindowsDebugLauncher.exe' '--stdin=Microsoft-MIEngine-In-kultbrgv.bxw' '--stdout=Microsoft-MIEngine-Out-tvw2mt1p.wdr' '--stderr=Microsoft-MIEngine-Error-lhguvkjt.xrj' '--pid=Microsoft-MIEngine-Pid-vho11bhr.z5a' '--dbgExe=C:\MinGW\bin\gdb.exe' '--interpreter=mi'
PS D:\C_Program>
enter image description here

Analizing crash dump

I'm having an issue trying to find out to which problem is crash dump pointing. If someone could help me it would be nice.
This is what I get in windbg.
Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Users\MAJSTOR\Documents\Sports Interactive\Football Manager 2015\crash dumps\FM 2015 v15.3.2.627042 (2015.06.26 17.55.38).dmp]
User Mini Dump File: Only registers, stack and portions of memory are available
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path. *
* Use .symfix to have the debugger choose a symbol path. *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is:
Windows 7 Version 7601 (Service Pack 1) MP (2 procs) Free x86 compatible
Product: WinNt, suite: SingleUserTS
Machine Name:
Debug session time: Fri Jun 26 17:55:38.000 2015 (UTC + 2:00)
System Uptime: not available
Process Uptime: 0 days 0:00:32.000
................................................................
.......................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1bac.ea8): Access violation - code c0000005 (first/second chance not available)
eax=76a80781 ebx=00000000 ecx=0a7ff803 edx=777970f4 esi=000002c4 edi=00000000
eip=777970f4 esp=0a7ff794 ebp=0a7ff800 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
ntdll!KiFastSystemCallRet:
777970f4 c3 ret

UDK error, how to examine crash dump

I am working on a game in UDK, and sometimes the game crashes when restarting the level or trying to open a new one.
I cannot find the problem trough the log files, they are just displaying a critical error.
Now I am trying to fix it by examining the crash dump, but I do not have a clue how to do this. Does anyone have an idea how I can further investigate the problem? I tried some things that I found online and this is what I have so far.
*** ERROR: Symbol file could not be found. Defaulted to export symbols for UDK.exe -
eax=00000000 ebx=39280070 ecx=0cdc0f10 edx=ffffffff esi=2a193f40 edi=296f96a0
eip=01c2caf3 esp=007cdf84 ebp=2c0132b0 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
UDK!GetStackOwnerClass+0x10a73:
01c2caf3 8b10 mov edx,dword ptr [eax] ds:002b:00000000=????????
0:000> !sym noisy
noisy mode - symbol prompts on
0:000> lmvm ntdll
start end module name
77890000 77a10000 ntdll (export symbols) ntdll.dll
Loaded symbol image file: ntdll.dll
Mapped memory image file: C:\Windows\SysWOW64\ntdll.dll
Image path: C:\Windows\SysWOW64\ntdll.dll
Image name: ntdll.dll
Timestamp: Thu Aug 29 03:50:31 2013 (521EA8E7)
CheckSum: 00140982
ImageSize: 00180000
File version: 6.1.7601.18247
Product version: 6.1.7601.18247
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Windows® Operating System
InternalName: ntdll.dll
OriginalFilename: ntdll.dll
ProductVersion: 6.1.7601.18247
FileVersion: 6.1.7601.18247 (win7sp1_gdr.130828-1532)
FileDescription: NT Layer DLL
LegalCopyright: © Microsoft Corporation. All rights reserved.
The most basic analysis in WInDbg is done with
.symfix
.reload
!analyze -v
Since it crashes, there should be an exception somewhere. Try
.ecxr
.exr -1
Also the call stack is potentially interesting. Since this is about UDK (Unreal Engine Development Kit), I assume it's written in C++ and thus we need a native call stack
k

Inline::C build failure

I am using active perl 5.14.2. Here I am trying to use a windows C library in my perl code using Inline::C
use strict;
use Inline C => DATA =>
CCFLAGS => '-std:C99 -Tx86-coff -Zi -Ob1 -fp:precise -W1 -Gd -Ze -Go -D_USER_SPACE_' =>
INC => '"-ID:\test\incl"' =>
LIBS => '"-LD:\test\lib" -llib1 -llib2 -llib3 -llib4';
test();
__END__
__C__
#include <incl/hd1.h>
#include <stdio.h>
#define __STDC_FORMAT_MACROS
#include <inttypes.h>
void test(void) {
.........
While I am trying to run the script its failing with below error
Warning. No Inline C functions bound to Perl in .\test.pl
Check your C function definition(s) for Inline compatibility
Microsoft (R) Program Maintenance Utility Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
C:\Perl5.14\bin\perl.exe C:\Perl5.14\lib\ExtUtils\xsubpp -typemap "C:\Perl5.14\lib\ExtUtils\typemap" test_pl_7fbb.xs > test_pl_7fbb.xsc && C:\Perl5.14\bin\perl.exe -M
ExtUtils::Command -e "mv" -- test_pl_7fbb.xsc test_pl_7fbb.c
cl -c -I"D:/test" "-ID:\test\incl" -std:C99 -Tx86-coff -Zi -Ob1 -fp:precise -W1 -Gd -Ze -Go -D_USER_SPACE_ -MD -Zi -DNDEBUG -O1 -DVERSIO
N=\"0.00\" -DXS_VERSION=\"0.00\" "-IC:\Perl5.14\lib\CORE" test_pl_7fbb.c
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\cl.EXE"' : return code '0xc0000135'
Stop.
A problem was encountered while attempting to compile and install your Inline
C code. The command that failed was:
C:\PROGRA~2\MICROS~1.0\VC\bin\nmake.exe > out.make 2>&1
The build directory was:
D:\test\_Inline\build\test_pl_7fbb
To debug the problem, cd to the build directory, and inspect the output files.
at .\test.pl line 0
...propagated at C:/Perl5.14/site/lib/Inline/C.pm line 772.
INIT failed--call queue aborted.
PS D:\cmapl_test> perl -v
This is perl 5, version 14, subversion 2 (v5.14.2) built for MSWin32-x86-multi-thread
(with 1 registered patch, see perl -V for more detail)
Copyright 1987-2011, Larry Wall
Binary build 1402 [295342] provided by ActiveState http://www.ActiveState.com
Built Oct 7 2011 15:49:44
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
I tried to build the test_pl_7fbb.c file using the command:
cl -c "-ID:\test" "-ID:\test\include" -std:C99 -Tx86-coff -Zi -Ob1 -fp:precise -W1 -
Gd -Ze -Go -D_USER_SPACE_ -MD -Zi -DNDEBUG -O1 "-DVERSION=0.00"
"-DXS_VERSION=0.00" "-IC:\Perl5.14\lib\CORE" test_pl_7fbb.c
It got completed without error.
My guess the option -I"D:/test" used by Inline while compiling is causing the issue.
Thanks in advance for any help on this.
You seem to have installed Visual C VC-9 (VS 2008), but last time I used Activeperl's Inline C it required VC-6.
If that's the reason, you could try to install VC 6 or switch to Strawberry Perl which has the gcc 4.6 tool chain included.
You could try installing MinGW from ActivePerl's PPM servers. It would probably need a recompile of many of your installed modules to work, but i'm sure it'll make things like Inline::C work properly.

Host information from a crash dump file

Is it possible to get information about the host where a dump file was been created e.g., OS version, service pack, .NET framework version etc.
Also the vertarget command displays current version of the Microsoft Windows operating. system
OS version and so forth is listed automatically when you open the dump file.
You can get the .NET framework version using the !eeversion command.
0:000> !eeversion
2.0.50727.4952 free
Workstation mode
SOS Version: 2.0.50727.4952 retail build
To get additional information for loaded modules use the verbose option for lm. E.g.
0:000> lm vm mscoree
start end module name
000007fe`f0f10000 000007fe`f0f7f000 mscoree (deferred)
Image path: C:\Windows\System32\mscoree.dll
Image name: mscoree.dll
Timestamp: Fri Nov 06 06:09:26 2009 (4AF3AF86)
CheckSum: 00072187
ImageSize: 0006F000
File version: 4.0.31106.0
Product version: 4.0.31106.0
File flags: 8 (Mask 3F) Private
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® .NET Framework
InternalName: mscoree.dll
OriginalFilename: mscoree.dll
ProductVersion: 4.0.31106.0
FileVersion: 4.0.31106.0 (Main.031106-0000)
PrivateBuild: DDBLD278
FileDescription: Microsoft .NET Runtime Execution Engine
LegalCopyright: © Microsoft Corporation. All rights reserved.
Comments: Flavor=Retail