I have a catalyst web server. I can see every child server process load a lot of same .so files individually, which take a lot of memory.
Is there any possible Catalyst preload all .so file once for all child processes?
The specific behavior you are describing is a feature of mod_perl rather than Catalyst itself. But you can of course run your Catalyst application under a mod_perl environment.
Under mod_perl there can only load shared library files once and it is not possible to have different versions. Aside from the saving of loading for multiple children this would actually work over different applications on the same server. So two different web applications using the mod_perl interpreter would actually share a loaded instance of a lirbrary that they both used.
For this reason, most people generally prefer to not use mod_perl as a means of serving their application, because they actually want to maintain different library versions per application. For the reasons described above this would not be possible in this environment.
But if that is something that you believe suits your needs, then mod_perl may be the environment for you.
Support for mod_perl with Catalyst is in a slight state of flux. The generally preferred method is to use the Plack::Handler methods to bootstrap Catalyst as a PSGI application to the mod_perl environment. There are some additional notes on configuration here.
I don't know what options are available in built-in Catalyst server, but looking at the documentation for Starman shows this option:
--preload-app
This option lets Starman preload the specified PSGI application in the
master parent process before preforking children. This allows memory
savings with copy-on-write memory management. When not set (default),
forked children loads the application in the initialization hook.
Enabling this option can cause bad things happen when resources like
sockets or database connections are opened at load time by the master
process and shared by multiple children.
Since Starman 0.2000, this option defaults to false, and you should
explicitly set this option to preload the application in the master
process.
Alternatively, you can use -M command line option (plackup's common
option) to preload the modules rather than the
itself.starman -MCatalyst -MDBIx::Class myapp.psgi
will load the modules in the master process for memory savings with
CoW, but the actual loading of myapp.psgi is done per children,
allowing resource managements such as database connection safer.
If you enable this option, sending HUP signal to the master process
will not pick up any code changes you make. See "SIGNALS" for details.
Related
I'd like to get an indication about the context in which my process is running from. I'd like to distinguish between the following cases :
It runs as a persistent scheduled task (launchDaemon/launchAgent)
It was called on-demand and created by launchd using open command-line or double-click.
It was called directly from command-line terminal (i.e. > /bin/myProg from terminal )
Perhaps is there any indication about the process context using Objective-c/swift framework or any other way ? I wish to avoid inventing the wheel here :-)
thanks
There is definetely no simple public API or framework for doing this, and doing this is hard.
Some parts of this info possibly could be retreived by your process itslef with some side-ways which will work on some system versions:
There is a launchctl C-based API, which you can try to use to enumerate all
launch daemon/agent tasks and search for your app path/pid. You may
require a root rights for your process for doing this.
Using open command-line sometimes could be traced with environment
variables it sets for your process.
Running directly from command-line could leave responsible_pid filled correctly (which is private API from libquarantine, unless you are observing it with Endpoint Security starting from 11.smth version)
All this things, except launchctl API, are not public, not reliable, could be broken at any time by Apple, and may be not sufficient for your needs.
But it is worth to take them a try, because there is nothing better :)
You could potentially distinguish all cases you want using system events monitoring from some other (root-permitted) process you control, possibly adopting Endpoint Security Framework (requires an entitlement from Apple, can't be distributed via AppStore), calling a lot of private APIs and a doing bunch of reversing tricks.
The open resource I could suggest on this topic is here
I would like to share a blessed object between two or more Perl applications. The object in question is quite expensive to instantiate, but always the same (static). The idea is to instantiate it once in one application and use it in other applications. This particular object is basically an http client using HTTP::Tiny an a whole bunch of other modules. Instantiating it via new() can take more than 50% of the total run time. I think the only module which may be a problem is the HTTP::Tiny since it open sockets, but not really sure. Can I use IPC::Shareable or some other method to share this http client among other applications?
Follow-up, are there any significant security issues with IPC::Shareable?
It supports anything Storable can handle. So it can handle objects, but not file handles (incl sockets). File handles are process-specific anyway.
Access to the shared memory is controlled using the same permission system that is used to control access to files (via the mode option), so the security issues are the same as they are for files.
I have a perl app which processes text files from the local filesystem (think about it as an overly-complicated grep).
I want to design a webapp which allows remote users to invoke the perl app by setting the required parameters.
Once it's running it would be desirable some sort of communication between the perl app and the webapp about the status of the process (running, % done, finished).
Which would be a recommended way of communication between the two processes? I was thinking in a database table, but I'm not really sure it's a good idea.
any suggestions are appreciated.
Stackers, go ahead and edit this answer to add code examples or links to them.
DrNoone, two approaches come to mind.
callback
Your greppy app needs to offer a callback function that returns the status and which is periodically called by the Web app.
event
This makes sense if you are already using a Web server/app framework which exposes an event loop usable from external applications (rather unlikely in Perl land). The greppy app fires events on status changes and the Web app attaches/listens to them and acts accordingly.
For IPC as you envision it, a plain database is not so suitable. Look into message queues instead. For great interop, pick AMPQ compliant implementation.
If you run the process using open($handle, "cmd |") you can read the results in real time and print them straight to STDOUT while your response is open. That's probably the simplest approach.
I am using the wonderful AnyEvent for creating an asynchronous TCP server (specifically, a MUD server).
In order to keep everything running smoothly and with as few blocking/synchronous pieces of code possible, I have replaced some modules I was using with their asynchronous counterpart, for example AnyEvent::Memcached and AnyEvent::Gearman. This allows the main program to be quite speedy, which is desirable. I have coded around the need for some of these calls to be synchronous.
One problem I currently have, and the focus of this question, is logging.
Before turning to AnyEvent for this server program, I was using Log::Log4perl as it allows me to fine-tune which modules or subroutines should be logged, at which level and to which log output (screen, file, etc).
The problem here is that the Log4perl actions (warn, info, etc) are currently performed synchronously but I have no requirement for that as long as the log lines eventually end up on the screen / file (and in the correct order).
Is Log::Log4perl still the right choice when using an asynchronous event handler such as AnyEvent, or should I look at a different module? If so, which is recommended?
AnyEvent::Log, which comes with AnyEvent, uses AnyEvent::IO, which appends to files asynchronously when IO::AIO is available (and synchronously when not).
What you are trying to avoid? If it's synchronous file IO (writing to log files/stdout etc.) then your problem would probably be solved with an asynchronous and/or buffering appender(s) rather than replacing all use of Log4perl in your code.
Log::Log4perl::Appender::Buffer seems like it might be a good start, but a completely async appender doesn't appear to exist anymore.
I am writing a couple fo scripts that go and collect data from a number of servers, the number will grow and im trynig to future proof my scripts, but im a little stuck.
so to start off with I have a script that looks up an IP in a mysql database and then connects to each server grabs some information and then puts it into the database again.
What i have been thinknig is there is a limited amount of time to do this and if i have 100 servers it will take a little bit of time to go out to each server get the information and then push it to a db. So I have thought about either using forks or threads in perl?
Which would be the prefered option in my situation? And hs anyone got any examples?
Thanks!
Edit: Ok so a bit more inforamtion needed: Im running on Linux, and what I thought was i could get the master script to collect the db information, then send off each sub process / task to connect and gather information then push teh information back to the db.
Which is best depends a lot on your needs; but for what it's worth here's my experience:
Last time I used perl's threads, I found it was actually slower and more problematic for me than forking, because:
Threads copied all data anyway, as a thread would, but did it all upfront
Threads didn't always clean up complex resources on exit; causing a slow memory leak that wasn't acceptable in what was intended to be a server
Several modules didn't handle threads cleanly, including the database module I was using which got seriously confused.
One trap to watch for is the "forks" library, which emulates "threads" but uses real forking. The problem I faced here was many of the behaviours it emulated were exactly what I was trying to get away from. I ended up using a classic old-school "fork" and using sockets to communicate where needed.
Issues with forks (the library, not the fork command):
Still confused the database system
Shared variables still very limited
Overrode the 'fork' command, resulting in unexpected behaviour elsewhere in the software
Forking is more "resource safe" (think database modules and so on) than threading, so you might want to end up on that road.
Depending on your platform of choice, on the other hand, you might want to avoid fork()-ing in Perl. Quote from perlfork(1):
Perl provides a fork() keyword that
corresponds to the Unix system call of
the same name. On most Unix-like
platforms where the fork() system call
is available, Perl's fork() simply
calls it.
On some platforms such as Windows
where the fork() system call is not
available, Perl can be built to
emulate fork() at the interpreter
level. While the emulation is
designed to be as compatible as
possible with the real fork() at the
level of the Perl program, there are
certain important differences that
stem from the fact that all the pseudo
child "processes" created this way
live in the same real process as far
as the operating system is concerned.