I was wondering if it's currently possible to have an 'external' (.so/.dylib) LLVM plugin (module) pass scheduled at LTO time? The reason for wanting this is a inter-modular optimization I want to add.
I also found this topic; How to write a custom intermodular pass in LLVM?
But a separate tool is not an option for me.
Thanks
I think the most helpful thing here might be to understand how passes are run and what the state of the code is during LTO.
First of all, when optimization passes are run by the compiler, they are done as a set that has been added to a PassManager. This means that LLVM/Clang, when passed something like -O3 will create a copy of a PassManager and subsequently provide it the set of passes expected to provide O3 level of optimization. This is very different from what you are doing with an external library which must be provided manually and cannot be fit into the pass pipeline normally.
Then we have the state of things when doing LTO. During Link Time Optimization, all of the individual translation units have been consolidated and are now a single Module. This means that an optimization which runs on each function will run on every function in the code base. Similarly, a per-module optimization will run on the full Module and therefor offer Inter-Procedural Analysis/Optimization.
If you're looking to use an Intra-Modular Pass then there is no reason to do this at LTO time and instead you can simply make a ModulePass and run that on each unit.
Related
Using Dymola, I'm looking for a way to automatically execute a script when loading a library. The intention is to define additional displayUnits using the defineUnitConversion() command, which are specific to the library that is loaded. Still I think there are quite some other cases where this could be helpful.
What I figured out in this regard:
I know that it is possible to add conversions to the file in DymolaInstallDir/insert/displayUnits.mos but this comes with the disadvantage that is has to be done again on every new computer or after an update of Dymola. I would like to avoid this.
Other than that I only found the libraryinfo.mos file, which seems to be read during the start-up of Dymola. Therefore I assume it is not the right place to put the conversions, as it contains general information about the library and should only contain the respective functions.
Dymola 2022 has a new (tool-specific) feature that covers exactly this use-case. It is mentioned in the Dymola 2022 release notes in the section "Library startup script" on page 24.
It basically introduces the a new annotation, which allows to specify a path to a .mos script, which is executed, when the respective library is loaded. Here is the example from the release notes:
package ThisPack
annotation(__Dymola_startup =
"modelica://ThisPack/Resources/Scripts/Dymola/startup.mos");
end ThisPack;
The annotation can also be set via the UI...
I want to compile my app using matlab-compiler it does so, but with issues...
It says there are some functions that are not licensed for compilation.
The problem is that I haven't used those functions (one of them is fimath.m) in my app.
I think these functions are used inside some of my functions which I don't know.
My question is how to find out which one of my functions are using those functions in order to remove them or replace them with other functions.
There are more than 50 functions in my app and it's not possible to check them one by one.
For every returned "unlicensed" function you can execute the following command,
dbstop in <function name> % without the <>
and afterwards run your code normally for several typical inputs/cases. If it stops at one of these breakpoints, look at the call stack (using either dbstack or the Editor tab of the MATLAB GUI), and identify the entry point from your own code.
If none of the breakpoints is ever hit, it could mean that these functions are referred-to inside the code, but some logic is preventing their execution (turning them, practically, to "unreachable code"). In this case, you will likely need to remove these references manually. To know where from, using information from the link posted by VTodorov you can list the dependencies of each file using
[fList,pList] = matlab.codetools.requiredFilesAndProducts('myFun.m');
which can be called on the output of dir (after some minor conversion). It could be useful to use the toponly flag.
I'm getting ready to try to deploy some code to multiple machines. As far as I know, using a Makefile.pm to track dependencies is the best way to ensure they are installed everywhere. The problem I have is I'm not sure our Makefile.pm has been updated as this application has passed through a few different developers.
Is there any way to automatically parse through either my source or a few full runs of my program to determine exactly what versions of what modules my application is depending on? On top of that, is there any way to filter it based on CPAN packages? (So that I only depend on Moose instead of every single module that comes with Moose.)
A third related question is, if you depend on a version of a module that is not the latest, what is the best way to have someone else install it? Should I start including entire localized Perl installations with my application?
Just to be clear - you can not generically get a list of modules that the app depends on by code analysis alone. E.g. if your apps does eval { require $module; $module->import() }, where $module is passed via command line, then this can ONLY be detected by actually running the specific command line version with ALL the module values.
If you do wish to do this, you can figure out every module used by a combination of runs via:
Devel::Cover. Coverage reports would list 100% of modules used. But you don't get version #s.
Print %INC at every single possible exit point in the code as slu's answer said. This should probably be done in END{} block as well as __DIE__ handler to cover all possible exit points, and even then may be not fully 100% covering in generic case if somewhere within the program your __DIE__ handler gets overwritten.
Devel::Modlist (also mentioned by slu's answer) - the downside compared to Devel::Cover is that it does NOT seem to be able to aggregate a database across multiple sample runs like Devel::Cover does. On the plus side, it's purpose-built, so has a lot of very useful options (CPAN paths, versions).
Please note that the other module (Module::ScanDeps) does NOT seem to allow you to do runtime analysis based on arbitrary command line arguments (e.g. it seems at first glance to only allow you to execute the program with no arguments) and if that's true, is inferior to all the above 3 methods for any code that may possibly load modules dynamically.
Module::ScanDeps - Recursively scan Perl code for dependencies
Does both static and runtime scanning. Just modules, I don't know of any exact way of verifying what versions from what distributions. You could get old packages from BackPan, or just package your entire chain of local dependencies up with PAR.
You could look at %INC, see http://www.perlmonks.org/?node_id=681911 which also mentions Devel::Modlist
I would definitely use Devel::TraceUse, which also shows a tree of the modules, so it's easy to guess where they are being loaded.
I have a workspace built using MS-Visual Studio 2005 with all C code.In that i see many functions which are not called but they are still compiled(they are not under any compile time macro to disable them from compiling).
I set following optimization settings for the MS-VS2005 project to remove that unused code:-
Optimization level - /Ox
Enable whole program optimization - /GL
I tried both Favor speed /Ot and Favor Size /Os
Inspite of all these options, when i see the linker generated map file, I see the symbols(unsed functions) names present in the map file.
Am I missing something? I want to completely remove the unused code.
How do I do this?
The compiler compiles C files one-at-a-time. Therefore, while compiling a C-file that does contains an unused function, the compiler cannot be sure that it will not be called from another file and hence it will compile that function too. However, if that function were declared as static (file-scope), then the compiler would know it is not used and hence remove it.
Even with whole program optimization, I think it would still not be done since the compilation could be for a library.
Linkers do something similar to what you are looking for. If your code links against a library containing multiple objects, then any objects that do not contain functions used by your code (directly or indirectly) would not be included in the final executable.
One option would be to separate your code into individual libraries and object files.
PS - This is just my guess. The behavior of the compiler (with whole program optimization) or linker essentially depends on the design choices of that particular compiler or linker
On our projects we have a flag set under the project properties\Linker\Refrences. We set it to Eliminate Unreferenced Data (/OPT:REF), according to the description this is supposed to remove function calls or data that are never used. I am just going by the description, I have never tested this or worked with it. But I just happened to see it within the last hour and figured it might be something you could try.
os i figured out how to use the -mthumb and -mno-thumb compiler flag and more or less understand what it's doing.
But what is the -mthumb-interlinking flag doing? when is it needed, and is it set for the whole project if i set 'compile for thumb' in my project settings?
thanks for the info!
Open a terminal and type man gcc
Do you mean -mthumb-interwork ?
-mthumb-interwork
Generate code which supports calling between the ARM and Thumb
instruction sets. Without this option the two instruction sets
cannot be reliably used inside one program. The default is
-mno-thumb-interwork, since slightly larger code is generated when
-mthumb-interwork is specified.
If this is related to a build configuration, you should be able to set it separately for each configuration "such as Release or Debug".
Why do you want to change these settings? I know using thumb instructions save some memory but will it save enough to matter in this case?
my application uses both, thumb and vfp code but i never specifically
set -thumb-interwork flag.. how is that possible?
According to man page, without that flag the two instructions sets
cannot be reliably used inside one program.
It says "reliably"; so without that option, it seems they still can be mixed within a single program but it might be "unreliably". I think normally mixing both instructions sets works, the compiler is smart enough to figure out when it has to switch from one set to another one. However, there might be border cases the compiler just doesn't understand correctly and it might fail to see that it should switch instruction sets here, causing the application to fail (most likely it will crash). This option generates special code, so that no matter what your code does, the switching always happens correctly and reliably; the downside is that this extra code is needed for every global visible function and thus increases the binary side (I have no idea if it also might slow down function calls a little bit, I personally would expect that).
Please also note the following two settings:
-mcallee-super-interworking
Gives all externally visible functions in the file being
compiled an ARM instruction set header
which switches to Thumb mode before executing the rest of
the function. This allows these
functions to be called from non-interworking code.
-mcaller-super-interworking
Allows calls via function pointers (including virtual
functions) to execute correctly regardless
of whether the target code has been compiled for
interworking or not. There is a small overhead
in the cost of executing a function pointer if this option
is enabled.
Though I think you only need those, when building libraries to be used with other projects; but I don't know for sure. The GCC thumb handling is definitely "underdocumented".