Due to license problem, few libraries cannot be linked statically.
So is it possible to prevent/detect preloaded library, either through LD_PRELOAD environment variable or through /etc/ld.so.preload configuration?
The previous can be detected through getenv(). But I have no idea about the other method.
I think there might be a general way to do it, is it?
is it possible to prevent/detect preloaded library, either through LD_PRELOAD environment variable or through /etc/ld.so.preload configuration?
You appear to be trying to implement some kind of anti-hacking protection. If so, it would be worth your while to study existing crackproofing techniques. This book describes a few.
Note that there are many other techniques to inject "foreign" code into your application, besides LD_PRELOAD and /etc/ld.so.preload. A couple that immediately come to mind are: LD_AUDIT, running under debugger, and renaming/replacing libc.so.
You have very little hope of stopping a moderately-sophisticated attacker. On Linux, I can build my own libc.so.6, and I can rename LD_PRELOAD to something else. I can also build my own kernel, and have it automatically inject myhack.so into your process without any user-space visible effects. Or I can simply make system calls do something else when executed by your application.
... the LD_PRELOAD ... can be detected through getenv()
That would stop only the least sophisticated attacker, for two reasons:
the preloaded library could itself interpose getenv(), and can hide LD_PRELOAD from your application, and
the LD_PRELOAD only matters at process startup. After the process has started, the preloaded library can easily remove LD_PRELOAD from the environment before your application has any chance to examine it.
Related
I'd like to get an indication about the context in which my process is running from. I'd like to distinguish between the following cases :
It runs as a persistent scheduled task (launchDaemon/launchAgent)
It was called on-demand and created by launchd using open command-line or double-click.
It was called directly from command-line terminal (i.e. > /bin/myProg from terminal )
Perhaps is there any indication about the process context using Objective-c/swift framework or any other way ? I wish to avoid inventing the wheel here :-)
thanks
There is definetely no simple public API or framework for doing this, and doing this is hard.
Some parts of this info possibly could be retreived by your process itslef with some side-ways which will work on some system versions:
There is a launchctl C-based API, which you can try to use to enumerate all
launch daemon/agent tasks and search for your app path/pid. You may
require a root rights for your process for doing this.
Using open command-line sometimes could be traced with environment
variables it sets for your process.
Running directly from command-line could leave responsible_pid filled correctly (which is private API from libquarantine, unless you are observing it with Endpoint Security starting from 11.smth version)
All this things, except launchctl API, are not public, not reliable, could be broken at any time by Apple, and may be not sufficient for your needs.
But it is worth to take them a try, because there is nothing better :)
You could potentially distinguish all cases you want using system events monitoring from some other (root-permitted) process you control, possibly adopting Endpoint Security Framework (requires an entitlement from Apple, can't be distributed via AppStore), calling a lot of private APIs and a doing bunch of reversing tricks.
The open resource I could suggest on this topic is here
The babel-node docs carry a stern warning:
Not meant for production use
You should not be using babel-node in production. It is unnecessarily heavy, with high memory usage due to the cache being stored in memory. You will also always experience a startup performance penalty as the entire app needs to be compiled on the fly.
Let's break this down:
Memory usage – huh? All modules are 'cached' in memory for the lifetime of your application anyway. What are they getting at here?
Startup penalty – how is this a performance problem? Deploying a web app already takes several seconds (or minutes if you're testing in CI). Adding half a second to startup means nothing. In fact if startup time matters anywhere, it matters more in development than production. If you're restarting your web server frequently enough that the startup time is an issue, you've got much bigger problems.
Also, there is no such warning about using Babel's require hook (require('babel-register')) in production, even though this presumably does pretty much exactly the same thing as babel-node. For example, you can do node -r babel-register server.js and get the same behaviour as babel-node server.js. (My company does exactly this in hundreds of microservices, with no problems.)
Is Babel's warning just FUD, or am I missing something? And if the warning is valid, why doesn't it also apply to the Babel require hook?
Related: Is it okay to use babel-node in production
– but that question just asks if production use is recommended, and the answers just quote the official advice, i.e. "No". In contrast, I am questioning the reasoning behind the official advice.
babel-node
The production warning was added to resolve this issue :
Without the kexec module, you can get into a really ugly situation where the child_process dies but its death or error never bubbles up. For more info see https://github.com/babel/babel/issues/2137.
It would be great if the docs on babel-node explained that it is not aimed for production and that without kexec installed that it has bad behaviour.
(emphasis mine)
The link for the original issue #2137 is dead, but you can find it here.
So there seems to be two problems here :
"very high memory usage on large apps"
"without kexec installed that it has bad behaviour"
These problems lead to the production warning.
babel-register
Also, there is no such warning about using Babel's require hook (require('babel-register')) in production
There may be no warning but it is not recommanded either. See this issue :
babel-register is primarily recommended for simple cases. If you're running into issues with it, it seems like changing your workflow to one built around a file watcher would be ideal. Note that we also never recommend babel-register for production cases.
I don't know enough about babel's and node's internals to give a full answer; some of this is speculation, but the caching babel-node would do is not the same thing as the cache node does.
babel-node's cache would be another cache on top of node's require cache, and it would have to, at best, cache the resulting source code (before it's fed to node).
I believe node's cache, after evaluating a module, will only cache things reachable from the exports, or, rather, the things that are not reachable anymore will be eventually GCed.
The startup penalty will depend on the contents of your .babelrc, but you're forcing babel to do the legwork to translate your entire source code every time it is executed. Even if you implement a persistent cache, babel-node would still need to do a cache fetch and validation for each file of your app.
In development, more appropriate tools like webpack in watch mode can, after the cold start, re-translate only modified files, which would be much faster than even a babel-node with perfectly optimized cache.
I have a catalyst web server. I can see every child server process load a lot of same .so files individually, which take a lot of memory.
Is there any possible Catalyst preload all .so file once for all child processes?
The specific behavior you are describing is a feature of mod_perl rather than Catalyst itself. But you can of course run your Catalyst application under a mod_perl environment.
Under mod_perl there can only load shared library files once and it is not possible to have different versions. Aside from the saving of loading for multiple children this would actually work over different applications on the same server. So two different web applications using the mod_perl interpreter would actually share a loaded instance of a lirbrary that they both used.
For this reason, most people generally prefer to not use mod_perl as a means of serving their application, because they actually want to maintain different library versions per application. For the reasons described above this would not be possible in this environment.
But if that is something that you believe suits your needs, then mod_perl may be the environment for you.
Support for mod_perl with Catalyst is in a slight state of flux. The generally preferred method is to use the Plack::Handler methods to bootstrap Catalyst as a PSGI application to the mod_perl environment. There are some additional notes on configuration here.
I don't know what options are available in built-in Catalyst server, but looking at the documentation for Starman shows this option:
--preload-app
This option lets Starman preload the specified PSGI application in the
master parent process before preforking children. This allows memory
savings with copy-on-write memory management. When not set (default),
forked children loads the application in the initialization hook.
Enabling this option can cause bad things happen when resources like
sockets or database connections are opened at load time by the master
process and shared by multiple children.
Since Starman 0.2000, this option defaults to false, and you should
explicitly set this option to preload the application in the master
process.
Alternatively, you can use -M command line option (plackup's common
option) to preload the modules rather than the
itself.starman -MCatalyst -MDBIx::Class myapp.psgi
will load the modules in the master process for memory savings with
CoW, but the actual loading of myapp.psgi is done per children,
allowing resource managements such as database connection safer.
If you enable this option, sending HUP signal to the master process
will not pick up any code changes you make. See "SIGNALS" for details.
The GNU page says :
Your program can arrange to run its own cleanup functions if normal termination happens. If you are writing a library for use in various application programs, then it is unreliable to insist that all applications call the library's cleanup functions explicitly before exiting. It is much more robust to make the cleanup invisible to the application, by setting up a cleanup function in the library itself using atexit or on_exit.
SDL in one of it's pages says :
You can use SDL_Quit() with atexit() to ensure that it is run when your application is shutdown, but it is not wise to do this from a library or other dynamically loaded code.
What I understood from the GNU page is that it encourages the usage of atexit() in programs.
Can someone elaborate on this, taken from the SDL page ? The meaning is not so obvious :
but it is not wise to do this from a library or other dynamically loaded code.
What are the pitfalls that are to be avoided while using atexit() ?
Are the above two quotes contradicting each other ?
No, SDL is saying don't call atexit(SDL_Quit) from within a library. GNU advise you set up atexit within your library to avoid cleanup problems, SDL is saying don't call it from a different library - due to the manner in which dynamic code gets unloaded you can't be certain of ordering (especially in multithreaded apps).
Short version: call atexit(SDL_Quit) inside your program's main. If you are using or writing a wrapper library around SDL, don't call atexit(SDL_Quit) inside that library, instead call atexit(YOURLIBRARY_Quit) inside the main function (presuming that YOURLIBRARY_Quit will handle the call to SDL_Quit.
atexit() is somehow like the dtor of c++ global/static object. One pitfall I have been seeing is, the atexit callbacks are called by exit() but the while the calling thread is running the callback other threads are also running so you need to make sure you don't have concurrency issue. Like in the dtor(of global/static obj), if you destroy something, other threads may still be using it so it might cause unexpected behavior.
But this is for Linux; not familiar with windows
I am trying to find a way to monitor the contents of a directory for changes. I have tried two approaches.
Use kqueue to monitor the directory
Use GCD to monitor the directory
The problem I am encountering is that I can't find a way to detect which file has changed. I am attempting to monitor a directory with potentially thousands of files in it and I do not want to call stat on every one of them to find out which ones changed. I also do not want to set up a separate dispatch source for every file in that directory. Is this currently possible?
Note: I have documented my experiments monitoring files with kqueue and GCD
My advice is to just bite the bullet and do a directory scan in another thread, even if you're talking about thousands of files. But if you insist, here's the answer:
There's no way to do this without rolling up your sleeves and going kernel-diving.
Your first option is to use the FSEvents framework, which sends out notifications when a file is created, edited or deleted (as well as things to do with attributes). Overview is here, and someone wrote an Objective C wrapper around the API, although I haven't tried it. But the overview doesn't mention the part about events surrounding file changes, just directories (like with kqueue). I ended up using the code from here along with the header file here to compile my own logger which I could use to get events at the individual file level. You'd have to write some code in your app to run the logger in the background and monitor it.
Alternatively, take a look at the "fs_usage" command, which constantly monitors all filesystem activity (and I do mean all). This comes with Darwin already, so you don't have to compile it yourself. You can use kqueue to listen for directory changes, while at the same time monitoring the output from "fs_usage". If you get a notification from kqueue that a directory has changed, you can look at the output from fs_usage, see which files were written to, and check the filenames against the directory that was modified. fs_usage is a firehose, so be prepared to use some options along with Grep to tame it.
To make things more fun, both your FSEvents logger and fs_usage require root access, so you'll have to get authorization from the user before you can use them in your OS X app (check out the Authorization Services Programming Guide for info on how to do it).
If this all sounds horribly complicated, that's because it is. But at least you didn't have to find out the hard way!